Methods and apparatus for brain machine interface decoders

Abstract

The techniques described herein relate to systems, apparatus, articles of manufacture, and methods for decoding biophysical activity. An example method includes inputting biophysical activity data into an input module comprising one or more machine learning models and outputting, from the input module, one or more sets of inputs to change a state of one or more continuous attractor networks (CANs), inputting the one or more sets of inputs into a representation module comprising the one or more CANs and outputting, from the representation module, output representative of the change to the state of the one or more CANs, and causing control of one or more degrees of freedom of a device based on the output.

Description

METHODS AND APPARATUS FOR BRAIN MACHINE INTERFACE DECODERSRELATED APPLICATION

[0001] This application claims priority under 35 U.S.C. § 119(e) to U.S. Provisional Application No. 63 / 677,939, entitled “METHODS AND APPARATUS FOR BRAIN MACHINE INTERFACE DECODERS,” filed on July 31, 2024, which is herein incorporated by reference in its entirety.GOVERNMENT SUPPORT

[0002] This invention was made with government support under NS 123716 awarded by the National Institutes of Health, and IIS2151077 awarded by the National Science Foundation. The government has certain rights in this invention.FIELD

[0003] The techniques described herein relate generally to human computer interfaces and, more particularly, to systems, apparatus, articles of manufacture, and methods for brain machine interface decoders.BACKGROUND

[0004] Human computer interaction (HCI) disciplines, including brain machine interface (BMI) systems, convert biological signals into control signals to control a device. For instance, an HCI system may use muscle activity to control a prosthesis, or a BMI system may use neural signals from motor and / or language regions of the brain to manipulate devices, such as computers (e.g., computer cursors) or prosthetic devices. HCI systems, such as BMI systems, may allow users’ intentions to be directly enacted onto a device without physical movement by the user. Thus, BMI systems and / or, more generally, HCI systems, hold the potential to restore autonomy and independence to persons suffering from severe motor impairment by providing direct neural control of devices to carry out daily tasks.SUMMARY

[0005] In accordance with the disclosed subject matter, systems, apparatus, methods, and articles of manufacture are provided for brain machine interface decoders and / or, more generally, human computer interface decoders.

[0006] Some embodiments relate to a method for biophysical signal decoding, comprising inputting biophysical activity data into an input module comprising one or more machine learning models and outputting, from the input module, one or more sets of inputs to change a state of one or more continuous attractor networks (CANs), inputting the one or more sets of inputs into a representation module comprising the one or more CANs and outputting, from the representation module, output representative of the change to the state of the one or more CANs, and causing control of one or more degrees of freedom of a device based on the output.

[0007] Some embodiments relate to an apparatus for biophysical activity decoding comprising memory storing processor-executable instructions, and at least one hardware processor configured to execute the processor-executable instructions to perform a method comprising inputting biophysical activity data into an input module comprising one or more machine learning models and outputting, from the input module, one or more sets of inputs to change a state of one or more continuous attractor networks (CANs), inputting the one or more sets of inputs into a representation module comprising the one or more CANs and outputting, from the representation module, output representative of the change to the state of the one or more CANs, and causing control of one or more degrees of freedom of a device based on the output.

[0008] Some embodiments relate to at least one computer-readable storage medium storing processor-executable instructions that, when executed by at least one hardware processor, cause the at least one hardware processor to perform a method for biophysical activity decoding comprising inputting biophysical activity data into an input module comprising one or more machine learning models and outputting, from the input module, one or more sets of inputs to change a state of one or more continuous attractor networks (CANs), inputting the one or more sets of inputs into a representation module comprising the one or more CANs and outputting, from the representation module, output representative of the change to the state of the one or more CANs, and causing control of one or more degrees of freedom of a device based on the output.

[0009] Some embodiments relate a system for biophysical activity decoding comprising at least one hardware processor, and at least one computer-readable storage medium storing processor-executable instructions that, when executed by the at least one hardware processor, cause the at least one hardware processor to perform a method comprising inputting biophysical activity data into an input module comprising one or more machine learning models and outputting, from the input module, one or more sets of inputs to change a state ofone or more continuous attractor networks (CANs), inputting the one or more sets of inputs into a representation module comprising the one or more CANs and outputting, from the representation module, output representative of the change to the state of the one or more CANs, and causing control of one or more degrees of freedom of a device based on the output.

[0010] The foregoing summary is not intended to be limiting. Moreover, various aspects of the present disclosure may be implemented alone or in combination with other aspects.BRIEF DESCRIPTION OF FIGURES

[0011] Various aspects and embodiments will be described with reference to the following figures. It should be appreciated that the figures are not necessarily drawn to scale. Items appearing in multiple figures are indicated by the same or a similar reference number in all the figures in which they appear.

[0012] FIG. 1 is an illustration of an example brain machine interface system including a neural decoder to convert neural activity data into outputs for control of a device, according to some embodiments.

[0013] FIG. 2A is an illustration of decoding neural activity of a user into control of a device, according to some embodiments.

[0014] FIG. 2B is an illustration of converting an intention manifold into a brain machine interface control manifold via a neural activity manifold, according to some embodiments.

[0015] FIG. 3 is a block diagram of an example implementation of the neural decoder of FIG. 1, according to some embodiments.

[0016] FIG. 4A depicts an example implementation of the neural decoder of FIGS. 1 and / or 3 to convert neural activity data into a lattice position of a first brain machine interface manifold, according to some embodiments.

[0017] FIG. 4B depicts an example implementation of the neural decoder of FIGS. 1 and / or 3 to convert neural activity data into a lattice position of a second brain machine interface manifold, according to some embodiments.

[0018] FIG. 5A shows an example connectivity matrix for a line attractor, according to some embodiments.

[0019] FIG. 5B shows an example connectivity matrix for a ring attractor, according to some embodiments.

[0020] FIG. 6 shows mappings of states of a continuous attractor network activity manifold to respective bumps of activity, according to some embodiments.

[0021] FIG. 7A shows a first mapping of an input to a continuous attractor network to a lattice position of the continuous attractor network, according to some embodiments.

[0022] FIG. 7B shows a second mapping of an input to a continuous attractor network to a lattice position of the continuous attractor network, according to some embodiments.

[0023] FIG. 8 is a flowchart representative of an example process that may be performed and / or example machine-readable instructions that may be executed by processor circuitry to implement the neural decoder of FIGS. 1 and / or 3 to convert neural activity data into control of a device, according to some embodiments.

[0024] FIG. 9 is an example electronic platform structured to execute the machine -readable instructions of FIG. 8 to implement the neural decoder of FIGS. 1 and / or 3, according to some embodiments.DETAILED DESCRIPTION

[0025] The present application generally provides techniques for brain machine interfaces (BMIs) and / or, more generally, human computer interaction (HCI) systems, that use machine learning to convert a user’s biophysical signals (that may also be referred to as biosignals or biological signals) into control of a device. For example, an HCI system may convert a user’s muscle activity state (e.g., muscle activity signals) into control of a device, such as a prosthesis. In another example, a BMI system may convert a user’s neural activity state (e.g., neural signals) into control of a device, such as a computer screen cursor.

[0026] As disclosed herein, control of a device having one or more degrees of freedom can be represented by a manifold (e.g., an HCI control manifold, a BMI control manifold). The manifold can be decomposed into one or more one-dimensional attractor manifolds, and each attractor manifold can represent control of one of the degree(s) of freedom. As disclosed herein, one or more machine learning models can be trained to encode the user’s biosignals (e.g., muscle activity, neural activity state) onto the one or more one-dimensional attractor manifolds to effectuate control of the device in the one or more degrees of freedom with improved hardware computational efficiency with respect to conventional HCI / BMI techniques.

[0027] HCIs (that may also be referred to as human computer interfaces) seek to convert biophysical signals, such as muscle activity, into machine commands, allowing users’ intentions to control a device. BMIs (that may also be referred to as brain computer interfaces (BCIs)) also seek to convert biophysical signals, such as neural signals, into machine commands, allowing users’ intentions to be directly enacted without physical movement bythe user. Thus, BMIs and / or, more generally, HCIs (BMIs / HCIs), hold the potential to restore autonomy and independence to millions of individuals suffering from severe motor impairment. Further, BMIs and / or, more generally, HCIs, hold the potential to augment the life of millions of individuals not suffering from severe motor impairment, such as nondisabled individuals seeking to use BMIs / HCIs to improve control of machines such as augmented reality and / or virtual reality (AR / VR) devices.

[0028] The inventors have recognized multiple technical challenges for achieving widely usable BMIs / HCIs. First, the inventors have recognized that biophysical signal acquisition, such as brain signal acquisition, is limited by hardware constraints. Biophysical signal acquisition is ideally non-invasive, cheap, stable, noiseless, and of high spatial and temporal bandwidth. However, the inventors have recognized that these hardware desiderata are subject to inherent and fundamental tradeoffs dictated by physics and biology.

[0029] Second, the inventors have recognized that conventional BMI / HCI decoders used for biophysical signal interpretation (e.g., muscle activity interpretation, brain signal interpretation) fall short of the requirements necessary to enable truly effective and widely adopted BMI / HCI applications. Conventional BMI / HCI decoders may be implemented using machine learning. However, the inventors have recognized that decoding biophysical signals (e.g., muscle activity signals, brain signals) to generate appropriate machine outputs (e.g., computer outputs, robotic outputs) is a technically challenging machine learning problem because the transformation from intention to action is complex. For example, even in highly controlled and idealized laboratory settings, biophysical recordings (e.g., muscle activity recordings, neural recordings) are noisy and of low spatial or temporal bandwidth.

[0030] Further, the inventors have recognized that some types of biophysical signal interpretation, such as brain signal interpretation, is significantly affected by neural signal drifting. For example, neural signals drift due to factors intrinsic to the readout technology (e.g., changes across electrodes in different positions over time) and because of ongoing plasticity in the brain mutating the neural encoding of variables. Although specialized hardware and algorithms have been developed to reduce the disruption in neural decoding due to signal drift, drift cannot be entirely removed (e.g., due to neural plasticity) and decoders must be able to be effective in the face of signal drift.

[0031] The decoding challenge is compounded by several unique technical constraints of BMI / HCI applications. First, training data is typically generated by the individual BMI / HCI user, making it a very limited resource. To be most useful, BMI / HCI decoders must do well in the data-poor regime. Second, useful BMIs / HCIs will also require portability, lowbandwidth, and low power consumption, resulting in limited computer hardware that can be used by the decoder. Third, BMI / HCI decoders will also have to be reliable and robust prompting decoders to be effective under a wide range of conditions. These factors, especially data efficiency, are particularly important given that drift in neural recording require frequent re-training of the decoder algorithm, requiring the user to generate new calibration data each time, and model re-training. Finally, BMIs / HCIs used in medical devices must meet regulatory standards and have algorithmic interpretability, which may refer to an external observer’s ability to understand the process leading to the algorithm’s output.

[0032] The inventors have thereby recognized that the core desiderata for high-performance BMI / HCI are accuracy (e.g., ability to precisely reconstruct the user’s intention), reliability, robustness, data efficiency, power and compute efficiency, and interpretability. The inventors have recognized that conventional approaches to BMI / HCI do not achieve these core desiderata. For example, conventional approaches to BMI / HCI decoding may rely on linear machine learning algorithms and techniques like the Kalman Filter. However, these approaches fail to achieve sufficiently high decoding accuracy. Conventional BMI / HCI approaches may use modem machine learning models that have sophisticated architectures and training methods to achieve high accuracy, but the increase in model size and complexity resulted in increased demands for substantially large amounts of training data and are not interpretable. They also require substantial computational hardware resources for training and at inference time. Thus, the inventors have recognized that these modern machine learning models with their high complexity, demand for larger datasets, and compute resource consumption has rendered them incapable of jointly addressing the core set of BMI / HCI technical challenges. These limitations are particularly acute in models aiming to leverage larger BMI / HCI datasets to improve accuracy and data efficiency (for new users): the models’ size and complexity result in significant degradation of hardware efficiency and interpretability. The technical limitations of current decoders are likely to become increasingly more relevant as BMI / HCI is targeted towards applications requiring more sophisticated controls (e.g., robotic prosthetics, AR / VR controls). Thus, technical challenges inherent to the BMI / HCI decoding problem limit the use of modern machine learning architectures (as well as linear machine learning algorithms and techniques like the Kalman Filter) developed in other fields to implementing effective BMI / HCI decoders.

[0033] The inventors have developed technology that overcomes the technical limitations of current decoders for BMI applications and / or, more generally, HCI applications. Specifically,the inventors have developed a biophysical signal decoder (e.g., a muscle activity decoder, a neural decoder) configured to have a built-in structure that models a control manifold of a BMI / HCI application, such that the biophysical signal decoder can convert biophysical activity data (e.g., muscle activity data, neural activity data) of a user into outputs that conform to the structure of the commands used by the BMI / HCI application to control a device (e.g., a computer cursor, a prosthetic limb, a robotic arm, an AR / VR device). The built-in structure can constrain the biophysical signal decoder’s outputs to the BMI / HCI control manifold, which simultaneously achieves high decoding accuracy while minimizing and / or removing the need for learning such structure from data (e.g., training data, muscle activity data, neural activity data).

[0034] In some embodiments, the biophysical signal decoder includes and / or is implemented by a representation module. The representation module can be configured to constrain the biophysical signal decoder’s outputs to the BMI / HCI control manifold. In some embodiments, the representation module can be implemented by one or more machine learning models. The one or more machine learning models can be one or more continuous attractor networks (CANs). In CANs, bumps of activity are localized patterns that move together in response to inputs. The positions of these bumps along a manifold can represent the value of a continuous variable, such as a position or orientation. External stimuli can set the positions of these bumps, and their persistence can act as a memory of the stimulus value. The outputs of the one or more CANs can be converted into commands to control a device in accordance with a user’s intention to control the device.

[0035] In some embodiments, the representation module can be implemented using multiple CANs, each with a one-dimensional attractor manifold that, collectively, constructs a manifold equivalent to the BMI / HCI control manifold. For example, the one or more CANs can be configured with an attractor manifold topology that matches and / or corresponds to a topology of the BMI / HCI control manifold. Beneficially, despite the increased number of CANs, the lower dimensionality of each network’s manifold results in an overall reduction in number of parameters, which substantially improves computational hardware efficiency at inference time without negatively affecting accuracy of the biophysical signal decoder.

[0036] In some embodiments, the biophysical signal decoder includes and / or is implemented by an input module, whose output(s) is / are coupled to input(s) of the representation module. The input module can be trained to steer the bump(s) of the one or more CANs based on input biophysical activity data (e.g., muscle activity data, neural activity data) of a user.

[0037] In some embodiments, the input module can be implemented by one or more machine learning models. For example, the one or more machine learning models can be at least one neural network (e.g., at least one time convolutional neural network (TCNN) that may also be referred to as a temporal convolutional network (TCN)) that can be trained to learn a representation of the neural activity data conducive to accurate decoding. In some such embodiments, the one or more machine learning models can include individual, small, fully- connected networks (e.g., multilayer perceptrons (MLPs)) that are trained to map the neural network outputs to inputs to each of the one or more machine learning models of the representation module. By way of example, the neural network can learn a representation of the input neural activity data conducive to accurate decoding, while the small, fully- connected networks can steer the state(s) of the one or more CANs of the representation module to update their representation of the decoded variable to reconstruct the user’ s intention in controlling one or more devices based on the representation of the input neural activity generated by the neural network.

[0038] The techniques described herein may be implemented in any of numerous ways, as the techniques are not limited to any particular manner of implementation. Examples of details of implementation are provided herein solely for illustrative purposes. Furthermore, the techniques disclosed herein may be used individually or in any suitable combination, as aspects of the technology described herein are not limited to the use of any particular technique or combination of techniques.

[0039] Turning to the figures, the illustrated example of FIG. 1 is an illustration of an example brain machine interface (BMI) system 100 including a neural decoder 102 to convert neural activity data 104 into outputs 106 for control of a device. The BMI system 100 can allow a user’s intentions to be directly enacted onto a device without physical movement by the user. For example, the neural decoder 102 can be trained as a neural to kinematic mapping function with neural signals from a brain of a subject as input to the neural decoder 102 and kinematics to control the device as output of the neural decoder 102.

[0040] Alternatively, the BMI system 100 may be a human computer interaction (HCI) system. For example, the neural decoder 102 can be a biophysical signal decoder configured to convert biophysical activity data, such as accelerometer motion data and / or biophysical signals representative of a user’s muscle activity data, into the outputs 106 for control of a device. In such an example, the biophysical signal decoder and / or, more generally, the HCI system can be used to control a device, such as an augmented reality and / or virtual reality (AR / VR) device (e.g., an AR / VR headset).

[0041] Examples of a device that may be controlled by the BMI system 100 (and / or the HCI system described above) include a computer (e.g., a computer cursor, a display device, a speaker, an AR / VR device), a prosthetic limb, and a robotic arm. For example, the BMI system 100 can convert a user’s intentions, which are encoded in the neural activity data 104, into control of a computer cursor, a display device, a speaker, a prosthetic limb, and / or a robotic arm. In such an example, the BMI system 100 can generate the outputs 106 to control a grasp and / or movement of a robotic arm substantially in real time. In another example, the BMI system 100 can generate the outputs 106 to control cursor movement on a computer screen and / or type words on the computer screen. In yet another example, the BMI system 100 can generate the outputs 106 to cause a computer to synthesize artificial speech for output by a speaker. For example, the BMI system 100 can control a computer screen cursor to interact with software (e.g., a computer program, a software program) producing speech that may be output by the speaker.

[0042] In some embodiments, the terms “real time”, “substantially real time”, and “substantially real-time” may refer to occurrence in a near instantaneous manner recognizing there may be real-world delays for computing time, transmission, etc. Thus, unless otherwise specified, “real time”, “substantially real time”, and “substantially real-time” may refer to being within a 5-second time frame, a 1-second time frame, a 0.5-second time frame, a 250- millisecond time frame, a 100-millisecond time frame, etc., of real time.

[0043] The user of the BMI system 100 may be a person suffering from severe motor impairment. Alternatively, the user may be a person that uses the BMI system 100 to augment their daily life to carry out tasks using the BMI system 100. Beneficially, the BMI system 100 holds the potential to restore autonomy and independence to persons suffering from severe motor impairment and / or augment the daily life of users by providing direct neural control of devices to carry out daily tasks.

[0044] The neural activity data 104 of this example represents neural activity of a user (e.g., an animal, a human person). The neural activity data 104 may be measurements of user neural activity (e.g., neural activity measurements). As shown, the neural activity state xt(e.g., the recording device’s readout) can encode information about the user’s intended action. Examples of the neural activity data 104 include data representative of detections of neuronal action potentials (also known as spikes) generated from a single neuron or multiple neurons (e.g., spike data) and data representative of intracortical local field potentials (EFPs) (e.g., EPF data). For example, the input to the neural decoder 102 can be an array of spikecounts collected from implanted electrodes. Neural activity data 104 including detections of neuronal action potentials may be referred to as spiking activity data.

[0045] Additional examples of the neural activity data 104 include data recorded by utilizing non-invasive techniques such as electroencephalography (EEG), functional magnetic resonance imaging (fMRI), functional near-infrared spectroscopy (fNIRS), and magnetoencephalography (MEG). For example, EEG data may be referred to as non-spiking extracellular activity data.

[0046] Additional examples of the neural activity data 104 include data recorded by utilizing invasive techniques via surgically implanted electrodes close to the target neurons in the cortex and / or deep brain structures using techniques that use deep brain stimulation (DBS) electrodes, electrocorticography (ECoG) electrodes, microelectrode arrays (MEAs), and stereo-electroencephalography (sEEG) electrodes.

[0047] In some embodiments, the neural activity data 104 and / or, more generally, biophysical activity data may be provided to the neural decoder 102 as input. For example, the neural decoder 102 may receive the neural activity data 104 and / or non-neural activity data.

[0048] An example of non-neural activity data includes data recorded using an electromyography (EMG) technique, which can be used to evaluate and record the electrical activity produced by skeletal muscles. EMG is performed using an instrument called an electromyograph that can produce data in the form of an electromyogram. Another example of non-neural activity data includes motion data from one or more accelerometers. Yet another example of non-neural activity data includes one or more electrodes placed on a user and configured to measure non-neural activity data. Other examples of non-neural activity data include galvanic skin response measurements and oculomotor signal measurements. Galvanic skin response may also be referred to as electrodermal activity (EDA) or skin conductance.

[0049] Control of the device is represented in the illustrated example by a BMI control manifold 108. A manifold is a mathematical representation that can describe a complex geometric object, such as a folded structure or a curved surface, using intrinsic dimensions and local coordinates. In some manifolds, such as those that may be used to implement the BMI control manifold 108, high-dimensional data is considered points in a space that can be approximated by a low-dimensional variety. A manifold representation can capture the relationships and structures between data points.

[0050] Examples of a manifold include a line, a circle, a plane, a cylinder, a sphere, and a torus. For example, the BMI control manifold 108 can be implemented by a plane and a cylinder.

[0051] In the illustrated example, the neural decoder 102 is configured to confine the outputs 106 to the BMI control manifold 108, which is shown as a low dimensional manifold N. The dimensionality of the BMI control manifold 108 depends on the number of degrees of freedom of the BMI and its topology is determined by the choice of controls.

[0052] By way of example, the BMI control manifold 108 can be a cursor control BMI, such as converting a user’s neural activity data 104 into the outputs 106 for control of a cursor on a display device (e.g., a computer screen). A cursor control BMI can be represented by two degrees of freedom: speed and orientation. For example, in a cursor control BMI, a two dimensional manifold is used: a plane when using Cartesian coordinates and a cylinder for polar coordinates. As shown, the BMI control manifold 108 is a two dimensional manifold that can represent a user’ s intention to control a speed of the cursor and an orientation of the cursor that, collectively, form an output yt.

[0053] In the illustrated example, the neural decoder 102 includes an input module 110 to receive the neural activity data 104 and a representation module 112 to encode the user’s neural activity data 104 into the outputs 106. For example, the representation module 112 can encode the user’s neural activity data 104 into the outputs 106 by constraining the outputs 106 to the BMI control manifold 108.

[0054] The representation module 112 of this example includes and / or is implemented by multiple machine learning models 114, 116. The machine learning models 114, 116 may be deep learning models.

[0055] Examples of a deep learning model include a neural network. Examples of a neural network include an autoencoder, a convolutional neural network (CNN), a CTC-fitted neural network model, a graph neural network (GNN), a multi-layer perceptron, a recurrent neural network (RNN) (e.g., an artificial recurrent neural network, a Gated Recurrent Unit (GRU), Fong Short Term Memory (ESTM)), a generative adversarial network (GAN), and a transformer. Additionally and / or alternatively, the machine learning models 114, 116 may be and / or be implemented by a different type of machine learning model, such as a clustering model, a decision tree, a support vector machine (SVM), a Bayesian network, a hidden Markov model, and / or any combination(s) thereof.

[0056] The machine learning models 114, 116 of this example are artificial recurrent neural networks. The artificial recurrent neural networks of this example are continuous attractornetworks (CANs). For example, the machine learning models 114, 116 include a first CAN 114 and a second CAN 116. Alternatively, the representation module 112 may be implemented using a different number of CANs. Alternatively, the machine learning models 114, 116 may be implemented by a different type of recurrent neural network and / or, more generally, a different type of machine learning model.

[0057] In some embodiments, the representation module 112 is configured in accordance with d degrees of freedom of a BCI. For example, for a BMI with d degrees of freedom, the representation module 112 can include -many CANs, such as -many line or ring attractor CANs.

[0058] As shown, the CANs 114 116 each have a one-dimensional attractor manifold. For example, the first CAN 114 is a ring attractor CAN with an attractor manifold Jf1and the second CAN 116 is a line attractor CAN with an attractor manifold2. Alternatively, the first CAN 114 and / or the second CAN 116 may be a different type of attractor CAN with a different attractor manifold. Alternatively, the representation module 112 may be implemented using a single CAN whose attractor manifold matches N exactly.

[0059] Beneficially, using at least one CAN (e.g., at least one of the CANs 114, 116) with matching attractor manifold topology allows the neural decoder 102 to achieve high decoding accuracy while simultaneously minimize and / or eliminate the need for supervised learning. For example, the at least one CAN constructs a manifold equivalent to N to achieve these benefits. By way of example with respect to FIG. 1, using multiple CANs 114, 116 with matching attractor manifold topology allows the neural decoder 102 to achieve high decoding accuracy while simultaneously minimize and / or eliminate the need for supervised learning. Collectively, the CANs 114, 116 construct a manifold equivalent to N. Beneficially, despite the increased number of networks, the lower dimensionality of each network’s manifold results in an overall reduction in number of parameters, which greatly improves computational hardware efficiency at inference time without negatively affecting decoding accuracy.

[0060] In CANs, bumps of activity are localized patterns that move together in response to inputs. The positions of these bumps along a manifold can represent the value of a continuous variable, such as a position or orientation. External stimuli can set the positions of these bumps, and their persistence can act as a memory of the stimulus value. The outputs 106 of the CANs 114, 116 can be converted into commands to control a device in accordance with a user’s intention to control the device. As shown, the output 106 of the neural decoder 102 is ad-dimensional vector ytwhere the ithentry consists of the value encoded in the state of the ithCAN (e.g., the position of its activity bump on the CAN’s lattice). The representation module 112 can allow the neural decoder 102 to encode the neural activity data 104 on a prespecified attractor manifold.

[0061] The neural decoder 102 of this example includes the input module 110 to cause the encoded value to change based on the neural activity data 104 to reflect changes in the user’s intention. For example, to achieve accurate decoding, the activity bumps of the CANs 114, 116 in the representation module 112 are to move over the respective lattices to reflect changes in the user’s intention. As shown, the neural activity data 104 is an input available to the neural decoder 102 to effect such changes and the input module 110 is tasked with learning how to steer the CANs’ bumps based on the neural activity data 104. The input module 110 processes the neural activity data 104 into one or more sets of inputs 113 identified by ztfor the CANs 114, 116 of the representation module 112.

[0062] In some embodiments, the input module 110 can be implemented by an efficient machine learning architecture, which can be implemented by one or more machine learning models, to ensure that the input module 110 yields accurate decoding while ensuring high efficiency. In some embodiments, the one or more machine learning models include multiple machine learning models. For example, the input module 110 can include and / or be implemented by a first machine learning model and one or more second machine learning models.

[0063] In some embodiments, the first machine learning model of the BMI control manifold 110 can be a neural network. The neural network can be one or more time convolutional neural networks (TCNNs) configured to learn a representation of the neural activity data 104 conducive for accurate decoding.

[0064] In some embodiments, the one or more second machine learning models can be individual, small, fully-connected networks referred to as input heads. The individual, small, fully-connected networks can be multilayer perceptrons (MLPs) trained to map the TCNN outputs to inputs to each CAN 114, 116 in the representation module 112.

[0065] FIG. 2A is an illustration of decoding neural activity of a user into control of a device. The illustration of FIG. 2A represents the relationship between user intention 202, neural activity 204, and decoded intention 206 in greater detail. In some embodiments, the neural activity state xt(e.g., the recording device’s readout) encodes information about the user’s intended action yt. The neural decoder 102 of FIG. 1 can be configured to reconstruct theinformation represented by the neural activity state, yielding an estimate of the user’s intention yt.

[0066] In the context of BMI technology, intention can be expressed in terms of the inputs expected by the device being controlled. When controlling a computer cursor, for example, the same end result (e.g., moving the cursor to a specific location) can be achieved with different kinds of inputs, such as the cursor’s target position versus movement (e.g., a velocity vector indicating speed and direction of movement). A velocity vector can, in turn, be expressed in either Cartesian or polar coordinates. These approaches are conceptually equivalent and the choice between can be informed by external factors (e.g., effect on user learning). Once selected, the choice of inputs carries significant implication for any decoder: the decoder’ s outputs must conform with the structure of the commands used by the BMI application. The inventors have recognized that previous approaches did not make use of this observation and instead relied on the models’ ability to learn this structure given sufficient data and compute. The inventors have developed the neural decoder 102 of FIG. 1 to leverage this important observation.

[0067] FIG. 2B is an illustration of converting an intention manifold 210, via a neural activity manifold 212, into a BMI control manifold 214. For example, the neural decoder 102 of FIG.1 can be configured to decode a user’ s intention, which is represented by the intention manifold 210 and encoded in the neural activity manifold 212, into output on the BMI control manifold 214. In some embodiments, the neural activity manifold 212 can represent the neural activity data 104 of FIG. 1. In some embodiments, the BMI control manifold 214 can implement and / or correspond to the BMI control manifold 108 of FIG. 1.

[0068] The neural decoder 102 can be configured such that its output, yt, is confined to a low dimensional manifold N : the BMI control manifold 214. The manifold’s dimensionality depends on the number of degrees of freedom of the BMI and its topology is determined by the choice of controls. For example, in a cursor control BMI a two dimensional manifold is used: a plane when using Cartesian coordinates and a cylinder for polar coordinates.Explicitly modelling N predisposes the neural encoder 102 towards highly accurate decoding while minimizing the need for supervised learning. By analogy to the BMI control manifold 214, the user’s intention ytcan also be represented by an intention manifold 210 sharing the same topology as J\C.

[0069] In neural state space, neural activity is often confined to a low dimensional manifold JVC with xtE JVC. The position of xtE JVC encodes the user intention ytand decoders are tolearn the inverse mapping to determine the position of ytE N. The inventors have recognized that the observation that neural activity is confined to low dimensional manifolds can be used to improve decoder accuracy and efficiency. However, neural activity manifolds vary across BMI tasks and their geometry changes over time and across individuals. Thus, algorithms making use of ’s structure must learn its geometry from data, worsening data and hardware efficiency. Conversely, JAT’s structure is determined uniquely by the choice of controls for the BMI application and is constant across subjects and over time. Thus, the inventors have recognized that leveraging JAT’s structure affords a significant increase in decoding accuracy and efficiency.

[0070] As discussed above, the decoding process includes updating the value of a variable, yt, based on a stream of (potentially noisy) data xtcarrying information about a latent (e.g., unobserved) variable ytof interest. Thus formulated, the decoding problem is analogous with a challenge routinely faced by animal brains. Animals often need to keep track of variables (e.g., position, orientation) critical for behavior while these cannot be directly observed (e.g., in darkness). In these conditions, the latent variables’ values are estimated through integration of incoming sensory data, a process known as path integration. Animal brains evolved to excel at this task and often rely on hard-wired neural circuitry designed for path integration.

[0071] The inventors have recognized that neural networks, referred to as CANs (e.g., the CANS 114, 116 of FIG. 1), have properties that make them ideally suited for this task. CANs are characterized by a continuous, low dimensional, attractor manifold in state space making them ideally suited for representing continuous variables in the face of noisy inputs. CANs’ manifold topology reflects that of the variable being represented and other computational constraints and the attractor property of the CAN dynamics allows for their representations to be robust to noise and persist even in the absence of external stimuli. CANs’ structure is genetically encoded, minimizing the need for learning in individual animals. Thus, the inventors have recognized that integrating CAN-like artificial neural networks in a BMI decoder, such as the neural decoder 102 of FIG. 1, would afford it similar advantages. For example, the representation module 112 of FIG. 1 can be implemented with the CANs 114, 116 having attractor dynamics with a pre-specified attractor manifold topology. The CAN’s units may be conceptualized as laying on a lattice whose topology matches that of the target manifold, such as the BMI control manifold 108 of FIG. 1.

[0072] FIG. 3 is a block diagram of an example BMI system 300, which includes an example neural decoder 302. In some embodiments, the BMI system 300 of FIG. 3 can implement theBMI system 100 of FIG. 1. In some embodiments, the neural decoder 302 of FIG. 3 can implement the neural decoder 102 of FIG. 1. Alternatively, the BMI system 300 may more generally implement an HCI system such that the neural decoder 302 may more generally implement a biophysical decoder (e.g., a biophysical signal decoder, a biophysical activity decoder) that can decode neural activity data and / or non-neural activity data.

[0073] As shown, the neural decoder 302 includes an input module 304 and a representation module 306. In some embodiments, the input module 304 can implement the input module 110 of FIG. 1. In some embodiments, the representation module 306 implements the representation module 112 of FIG. 1.

[0074] As shown, the input module 304 can be configured to receive neural activity data 308 (identified by xt) from sensor(s) 310 via a sensor interface module 312. In some embodiments, the neural activity data 308 can implement the neural activity data 104 of FIG. 1. For example, xtof FIG. 3 can correspond to xtof FIGS. 1, 2A, and / or 2B.

[0075] Additionally and / or alternatively, the output from the sensor(s) 310 may be biophysical activity data. For example, the input module 304 can be configured to receive biophysical activity data (which may also be identified by xt) from sensor(s) 310 via the sensor interface module 312. In such an example, the biophysical activity data can include neural activity data and / or non-neural activity data (e.g., muscle activity data).

[0076] In some embodiments, the sensor(s) 310 can be one or more neural activity sensors configured to measure and / or output the neural activity data 308. Additionally and / or alternatively, the sensor(s) 310 can be one or more biophysical activity sensors configured to measure and / or output biophysical activity data, which may include the neural activity data 308.

[0077] In some embodiments, the sensor(s) 310 can be one or more of the same type of activity sensors. For example, the sensor(s) 310 can be one or more of the same type of neural activity sensor or one or more of the same type of non-neural activity sensor (e.g., an EMG sensor).

[0078] In some embodiments, the sensor(s) 310 can be a combination of different types of sensors. For example, the sensor(s) 310 can be one or more of a first type of neural activity sensor and one or more of a second type of neural activity sensor that is different from the first type of neural activity sensor. In another example, the sensor(s) 310 can be one or more of a first type of non-neural activity sensor and one or more of a second type of non-neural activity sensor that is different from the first type of non-neural activity sensor. In yet anotherexample, the sensor(s) 310 can be one or more neural activity sensors and one or more non- neural activity sensors.

[0079] Examples of the sensor(s) 310 include an EEG device, an fMRI device, an fNIRS device, a MEG device, an electromyograph, and one or more electrodes configured to measure neural activity data. Examples of the one or more electrodes include DBS electrodes, ECoG electrodes, MEAs, and sEEG electrodes. For example, the input module 304 may obtain spiking activity data and / or non-spiking extracellular activity data from the sensor(s) 310, which may be via the sensor interface module 312.

[0080] Examples of neural activity sensors include an EEG device, an fMRI device, an fNIRS device, a MEG device, and one or more electrodes. Examples of non-neural activity sensors include an accelerometer, an EMG device, and one or more electrodes configured to measure non-neural activity data (e.g., galvanic skin response). For example, the accelerometer can be configured to output motion data associated with a user.

[0081] As shown, input(s) of the sensor interface module 312 can be coupled to output(s) of the sensor(s) 310. In some embodiments, the sensor interface module 312 can be implemented by one or more wired interfaces. Examples of wired interfaces include an Ethernet interface, a Peripheral Component Interconnect (PCI) interface, a Serial Digital Interface (SDI), a Universal Serial Bus (USB) interface, and a High-Definition Multimedia Interface (HDMI).

[0082] In some embodiments, the sensor interface module 312 can be implemented by one or more wireless interfaces. Examples of wireless receivers include a Wireless Fidelity (Wi-Fi) receiver, a Bluetooth receiver, a near-field communication (NFC) receiver, a cellular receiver (e.g., a fifth generation cellular (5G) receiver), and a radio-frequency identification (RFID) receiver. Alternatively, the sensor interface module 312 may be implemented by one or more wired interfaces and / or one or more wireless interfaces.

[0083] In some embodiments, the sensor interface module 312 can be configured to process (e.g., pre-process) measurements from the sensor(s) 310 into biophysical activity data, such as the neural activity data 308. For example, the sensor interface module 312 can include one or more analog-to-digital converters (ADCs) to convert an analog measurement from the sensor(s) 310 into a digital output for processing by the neural decoder 302. Additionally and / or alternatively, the sensor interface module 312 can filter and / or scale the analog measurement and / or the digital output.

[0084] Additionally and / or alternatively, the input module 304 may include a data processing module (not shown), which may include input(s) coupled to output(s) of the sensor interfacemodule 312. For example, the input module 304 may include a data processing module, which may include one or more ADCs to convert an analog measurement received from the sensor(s) 310 via the sensor interface module 312 into a digital output for processing by the neural decoder 302. Additionally and / or alternatively, the data processing module may filter and / or scale the analog measurement and / or the digital output.

[0085] The input module 304 of this example includes multiple machine learning models 314, 316a, 316b, 316c. Alternatively, the input module 304 may be implemented using a single machine learning model. The machine learning models 314, 316a, 316b, 316c of this example include a first machine learning model 314 and second machine learning models 316a, 316b, 316c.

[0086] As shown, input(s) of the first machine learning model 314 is / are coupled to output(s) of the sensor interface module 312. Output(s) of the first machine learning model 314 is / are coupled to respective input(s) of the second machine learning models 316a, 316b, 316c. For example, first output(s) of the first machine learning model 314 is / are coupled to first input(s) of a first one of the second machine learning models 316a, second output(s) of the first machine learning model 314 is / are coupled to second input(s) of a first one of the second machine learning models 316a, and so on.

[0087] The first machine learning model 314 of the input module 304 can be implemented by a neural network. For example, as shown, the first machine learning model 314 can be a TCNN. Alternatively, the first machine learning model 314 may be implemented using a different type of neural network and / or, more generally, a different type of machine learning model. Alternatively, the first machine learning model 314 may be implemented using multiple machine learning models.

[0088] The second machine learning models 316a, 316b, 316c of the input module 304 can be implemented by a plurality of neural networks. For example, the second machine learning models 316a, 316b, 316c can be implemented by respective MLPs referred to as input heads. In such an example, the second machine learning models 316a, 316b, 316c include at least a first MLP 316a (identified by “INPUT HEAD 1”), a second MLP 316b (identified by “INPUT HEAD 2”), and a third MLP 316c (identified by “INPUT HEAD N”). Fewer or more than the number of MLPs shown may be used to implement the input module 304.

[0089] The representation module 306 of this example includes third multiple machine learning models 318a, 318b, 318c and an aggregator 320. Alternatively, the multiple third machine learning models 318a, 318b, 318c may be implemented by a single machine learning model or a different number of machine learning models shown.

[0090] The third machine learning models 318a, 318b, 318c of this example can be implemented by a plurality of neural networks. The plurality of neural networks may be a plurality of CANs. For example, the third machine learning models 318a, 318b, 318c can be implemented by respective CANs. In such an example, the third machine learning models 318a, 318b, 318c include at least a first CAN 318a (identified by “CONTINUOUS ATTRACTOR NETWORK (CAN) 1”), a second CAN 318b (identified by “CAN 2”), and a third CAN 318c (identified by “CAN N”). Alternatively, the third machine learning models 318a, 318b, 318c may be implemented by a different type of neural network and / or, more generally, a different type of machine learning model.

[0091] The CANs 318a, 318b, 318c of FIG. 3 are artificial recurrent neural networks whose connectivity structure, the connectivity matrix VF, is given by construction and not derived through training. In some embodiments, the construction procedure of the CANs 318a, 318b, 318c begins by the definition of the CAN’s lattice: a geometric arrangement of the CAN’s units. For example, the lattice can share the same manifold topology as the target manifold 3~C. Units can be equally spaced along the lattice, and each unit is assigned a coordinate 0(- to denote its position in the lattice. For the line attractor 0j G [0, 1], while for the ring 0(- G [0, 2K]. Next, a pairwise distance matrix A is constructed, whose entries are A^y = difit, 0j) where d is a distance metric over the lattice. For example, in the case of the ring attractor d must respect the ring’s boundary conditions. Next, the final connectivity matrix is given by I / F = k(A) where k is a kernel function symmetric around the origin (here, k is the Gaussian function for the line attractor and the vonMises function for the ring). Advantageously, this construction is guaranteed to yield an RNN with attractor dynamics and the desired attractor manifold topology. Points on the manifold correspond to states of the network’s activity characterized by localized bumps of activity over the CAN’s lattice (see FIGS. 4A, 4B, and / or 6 discussed in further detail below).

[0092] CANs have several parameters that can be tuned to alter their dynamics (e.g., width of the activity bump or how fast an external input causes the bump to move along the lattice). These include: r, the dynamic’s time constant, p, the input’s gain, and c, the kernel’s width parameter (e.g., the standard deviation for the Gaussian kernel). The hyperparameters p, y can be optimized for each CAN.

[0093] As depicted, output(s) of the second machine learning models 316a, 316b, 316c is / are coupled to respective input(s) of the third machine learning models 318a, 318b, 318c. Forexample, first output(s) of input head 1 is / are coupled to first input(s) of CAN 1, second output(s) of input head 2 is / are coupled to second input(s) of CAN 2, and so on.

[0094] The representation module 306 of the depicted example includes the aggregator 320 to process output(s) of the third machine learning models 318a, 318b, 318c into output 322 identified by yt. In some embodiments, the output 322 can implement the outputs 106 of FIG. 1. For example, ytof FIG. 3 can correspond to ytof FIGS. 1, 2A, and / or 2B.

[0095] As shown, the input module 304 is composed of a single TCNN 314, shared across all the CANs 318a, 318b, 318c, and a set of small MLPs (e.g., the input heads 316a, 316b, 316c), one for each of the CANs 318a, 318b, 318c. The TCNN 314 can be trained to learn a representation of the input neural activity data 308 conducive to accurate decoding, while the input heads 316a, 316b, 316c are responsible for steering the CANs’ 318a, 318b, 318c state to update their representation of the decoded variable. For example, the TCNN 314 can be trained to learn an embedding of the neural activity data 402 into a k-dimensional space, and the embedding can be optimized to improve performance of downstream networks (e.g., the input heads 316a, 316b, 316c, the CANs 318a, 318b, 318c). In such an example, each intermediate output 324 xtcan be implemented as a vector representing the embedding of the corresponding neural activity data 308 xt.

[0096] As shown, the representation module 306 is made of one or more CANs 318a, 318b, 318c. In some embodiments, the number and topology of the CANs 318a, 318b, 318c depends on the number and kind of decoded variables which, in turn, is determined by the BMI application design. For example, the choice of CANs can be based on the behavioral variables included in the dataset for the BMI application design.

[0097] By way of example, in a BMI application that seeks to convert spike sorted recordings from hippocampal neurons in mice running along a one-dimensional corridor, a single line attractor CAN can be used to represent position in a one dimensional corridor. For example, the CANs 318a, 318b, 318c in FIG. 3 can be implemented by a single line CAN.

[0098] By way of another example, in a BMI application that seeks to convert extracellular, spike sorted recordings from thalamic head-direction neurons in mice freely moving in a two- dimensional area, a single ring attractor CAN can be used to represent head direction. For example, the CANs 318a, 318b, 318c in FIG. 3 can be implemented by a single ring attractor CAN.

[0099] By way of yet another example, in a BMI application that seeks to convert extracellular, spike sorted recordings from hippocampal neurons in rats running on a W-shaped arena, two line attractor CANs can be used to represent a two dimensional arena. For example, the CANs 318a, 318b, 318c in FIG. 3 can be implemented by two line attractor CANs.

[0100] By way of another example, in a BMI application that seeks to convert extracellular, spike sorted recordings from the motor cortex of a human subject trained to perform a task involving the simultaneous control of two cursors, each moving in a single direction, two line attractor CANs can be used to represent hand / cursor position or speed. For example, the CANs 318a, 318b, 318c in FIG. 3 can be implemented by two line attractor CANs.

[0101] By way of yet another example, in a BMI application that seeks to convert extracellular, spike sorted recordings from the entorhinal formation in mice running in a Y- shaped arena, three line attractor CANs can be used to represent position in a two dimensional arena and running speed, and two ring attractor CANs for heading and orientation. For example, the CANs 318a, 318b, 318c in FIG. 3 can be implemented by three line attractor CANs and two ring attractor CANs.

[0102] In example operation, the input module 304 can map measurements of the BMI user’s neural activity xtonto a set of inputs for the CANs 318a, 318b, 318c in the representation module 306: {z^} and represented in FIG. 3 as z* as output from the first input head 316a, z^ as output from the second input head 316b, and z as output from the third input head 316c. In some embodiments, the BMI user’s neural activity xtis a vector with one value for each measurement channel (e.g., for each electrode, each voxel in an fMRI, each optical sensor in fNIRS). In some such embodiments, a range of the values for each measurement channel may be bound when (i) the input module 304 and / or the sensor interface module 312 performs pre-processing that includes normalization operation(s) and / or (ii) by the sensor(s) 310 own sensing range (e.g., a voltage range that the sensor(s) 310 can detect).

[0103] In some embodiments, the input module 304 and / or the sensor interface module 312 can rescale the values for each measurement channel to a range suitable for the first machine learning model 314. An example range is [-1, 1] but any other range may be used.

[0104] In example operation, the input module 304 inputs the BMI user’s neural activity xtinto the first machine learning model 314 and outputs from the first machine learning model 314 an intermediate output xt324. The intermediate output xt324 is arepresentation of the BMI user’s neural activity xtthat is conducive to accurate decoding. In some embodiments, the first machine learning model 314 uses historical data (e.g., sensor measurements from the last 0.1 seconds) to improve performance.

[0105] As shown, all the second machine learning models 316a, 316b, 316c receive the same input xt. In some embodiments, the intermediate output xt324 is a vector whose dimensionality is arbitrary. In some such embodiments, the dimensionality of the intermediate output xt324 can be a hyperparameter of the neural decoder 302.

[0106] In some embodiments, values of the intermediate output xt324 are bound by the non-linearity used by the first machine learning model 314, which is a TCNN in the shown example. For example, the TCNN 314 can use a rectified linear unit (ReLU) as its non-linear activation function. In such an example, ReLU can result in positive unbounded values for the intermediate output xt324. Alternatively, the TCNN 314 may use tanh as its non-linear activation function. For example, tanh can result in bounded values in a range. An example range is [-1, 1]. Alternatively, a different range may be used.

[0107] In example operation, the input module 304 outputs one or more sets of inputs represented by ztl326. For example, the second machine learning models 316a, 316b, 316c can output a respective one of zf1, z^ , and z . In some embodiments, the sets of inputs z 326 are vectors of size equal to the number of units in each CAN 318a, 318b, 318c.

[0108] The one or more sets of inputs represented by ztl326 can change a state of one or more of the CANs 318a, 318b, 318c. For example, the values of the sets of inputs z 326 can be shaped like a bump on the CAN’s lattice where the CAN’s activity bump should be located. In such an example, the values of the sets of inputs ztl326 are therefore bounded values in a range. An example range is [0, 1]. Alternatively, a different range may be used.

[0109] As shown, respective ones of the sets of inputs represented by ztl326 are different between pairs of input heads and CANs. By way of example, assume that two of the CANs 318a, 318b are used for a BMI application to control a computer cursor. The first CAN 318a can correspond to control of the computer cursor for the horizontal direction and the second CAN 318b can correspond to control of the computer cursor for the vertical direction. Furthering the example, if the user of the BMI system 300 intends to move the cursor only in the horizontal direction, z* for the first CAN 318a will change to reflect that intention, while Zj for the second CAN 318b will remain constant because there is no vertical movement.

[0110] In example operation, the input module 304 inputs the one or more sets of inputs represented by z 326 into the representation module 306 and outputs, from therepresentation module 306, output htl328 representative of the change to the state of one(s) of the CANs 318a, 318b, 318c. For example, output htl328 are CAN output(s). As shown, the first CAN 318a can process Zf into htl, which represents the state of the first CAN 318a. Further shown, the second CAN 318b can process z into, which represents the state of the second CAN 318b. Also shown, the third CAN 318c can process z” into / i , which represents the state of the third CAN 318c.

[0111] In some embodiments, the output htl328 of each CAN 318a, 318b, 318c is a scalar. In some such embodiments, the output htl328 changes for each CAN 318a, 318b, 318c independently.

[0112] In some embodiments, the value of each output htl328 is bound by the lattice coordinate system of the corresponding CAN. For example, the value of h is bound by the lattice coordinate system of the first CAN 318a, the value ofis bound by the lattice coordinate system of the second CAN 318b, and so on.

[0113] By way of example in which the first CAN 318a is a line attractor CAN, the value of h is bound by [0, 1] for the line. By way of another example in which the second CAN 318b is a ring attractor CAN, the value of hl is bound by [0, 2n] for the ring.

[0114] In the illustrated example, each CAN output (e.g.,, h , htl) can be the readout for the control of one degree of freedom in the BMI application implemented by the BMI system 300. For example, the BMI system 300 can be configured with three CANs (e.g., CAN 1, CAN 2, CAN 3) to control a device 330 having three degrees of freedom. In such an example, h output from CAN 1 318a can be the read-out for control of a first degree of freedom of the device 330, h output from CAN 2 318b can be the read-out for control of a second degree of freedom of the device 330, and hl output from CAN 3 (e.g., the third CAN 318c when N = 3) can be the read-out for control of a third degree of freedom of the device 330.

[0115] As depicted, the CAN outputs (e.g., hl, hl, h ') are aggregated by the aggregator 320 for output to control the device 330 via a device control interface 332. The aggregator 320 of this example is a CAN output aggregator because the aggregator 320 can be configured to aggregate and / or combine the CAN outputs into the output 322.

[0116] In some embodiments, the output 322 is a vector. For example, the output 322 can be a ( / -dimensional vector ytwhere the ithentry consists of the value encoded in the state of the ithCAN 318a, 318b, 318c (e.g., the position of its activity bump on the CAN’s lattice) (see FIGS. 4A and / or 4B described in further detail below).

[0117] In some embodiments, the aggregator 320 processes the CAN outputs by converting them from the CAN’s variable range (e.g., [0, 1] for a line attractor CAN, [0, 2K] for a ring attractor CAN) to a variable range of the device 330. For example, the device 330 can be a computer cursor and the variable range of the computer cursor can be [0, 3000] pixels for the computer cursor position.

[0118] As shown, the output 322 is output and / or transmitted to the device 330 via the device control interface 332. For example, the neural decoder 302 can cause control of one or more degrees of freedom of the device 330 based on the output 322.

[0119] Examples of the device 330 include physical systems and digital systems. Examples of physical systems include a prosthetic limb, a robotic arm, and a wheelchair. Examples of digital systems include a computer cursor and an artificial speech synthesizer. Movement of the computer cursor can be rendered into display graphics by at least one display device. Output from the artificial speech synthesizer can be rendered into audio by at least one speaker.

[0120] In some embodiments in which the device 330 is a physical system (e.g., a wheelchair), the device 330 can perform further processing on the output 322 such that the output 322 can be converting to analog signals for the end effectors (e.g., actuators of the wheelchair). In some embodiments in which the device 330 is a digital system (e.g., computer cursor control), the output 322 can be directly translated into action by the device 330 (e.g., a computer cursor moves in accordance with the user’s intended computer cursor movement).

[0121] In some embodiments, portion(s) of the neural decoder 302 can be trained in accordance with the CANs’ functions to improve the training efficiency of the neural decoder 302 and reduce demands for data and computational hardware resources required to fit the model. A ground truth decoded variable y (for the IthBMI control) can correspond to a location 0ton the ithCAN’s lattice in the representation module 306. Here, 9 can be a coordinate system over a CAN’s lattice. For example, 9 can be a coordinate system over the lattice of the first CAN 318a, a coordinate system over the lattice of the second CAN 318b, and so on.

[0122] The inventors have recognized that, because of the precisely defined relationship between CAN states and outputs, accurate decoding occurs when the CAN’s activity bump is centered at 0t. Thus, inputs (e.g., z ) to the CAN should steer its bump from its current location, 0 , towards 0t. Conventional approaches to providing inputs to CANs made use of a velocity vector prescribing the movement of the activity bump on the lattice. Inprinciple, the input module 304 could be trained to produce such velocity vectors. However, since the precise vector depends on both the goal location 0tand the current location 0 , CANs dynamics must be updated during such training. This, however, results in substantial inefficiencies (e.g., CANs parameters are not updated during training) and can result in other common technical issues in supervised learning with recurrent neural networks (e.g., vanishing gradients).

[0123] Here, the neural decoder 302 is configured such that it leverages the mechanistic understanding of CAN function and voids the need to consider CAN dynamics during training. In some embodiments, the neural decoder 302 configures the input to the ithCAN, Zj, to be vector-valued such that it can be used to provide independent inputs to each unit in the CAN. For example, z can be implemented as an n-dimensional vector where n is the number of units in the CAN. In such an example, the status of the CAN can be perturbed by providing an input to each unit in the CAN.

[0124] In some embodiments, the neural decoder 302 configures ztlto take the form of a localized bump of inputs on the CAN’s lattice centered at 0t(see FIGS. 7 A and / or 7B described in further detail below). When the CAN’s activity bump is close to 0t, the interaction between the CAN’s state htland the inputs z disrupts the bump’s symmetry, causing it to shift in the direction of 0tachieving the desired objective and yielding accurate decoding. In some embodiments, z , the desired output of the input module 304, depends only on through 0tthe CAN’s own dynamic need not be considered and the input module 304 can be trained as an independent component with substantial computational efficiency gains. In some embodiments, when z^’s bump is distant from the CAN’s activity bump, the CAN’s winner-take-all dynamics effectively cause it to ignore this input (see FIGS. 7A and / or 7B described in further detail below). Using this approach, the neural decoder 302 can be trained extremely fast without sacrificing decoding accuracy.

[0125] In some embodiments, one(s) of the machine learning models of the neural decoder 302 can be trained using stochastic gradient descent and / or with the Adaptive Moment Estimation optimizer (referred to as the Adam optimizer). For example, the neural decoder 302 can evaluate mean squared error (MSE) over the training data.

[0126] In some embodiments, a bespoke training scheme can be used to improve the neural decoder 302 efficiency. In some such embodiments, under this scheme, only networks in the input module 304 are part of the training, and the representation module 306 is not included in the process. For each sample in the training set, the expected output of each inputhead 316a, 316b, 316c in the input module 304, ztl, was given by y , as a localized bump on the CAN’s lattice centered at 0l(after scaling from yl’s range to the CAN’s lattice range). While the location of the bump is entirely dictated by the ground truth data, it’s width can be one of the neural decoder 302 hyperparameters and can be optimized for each dataset. The entire input module 304 (e.g., the TCNN 314 and all input heads 316a, 316b, 316c) can be trained simultaneously using the mean-squared error between each input head’s output, z , and its expected output z as an error signal. The number of epochs, learning rate, and early stopping threshold (patience) can be varied across datasets and can be optimized using a same hyperparameter optimization scheme. Before training, neural activity data for training can be min-max scaled to the [0, 1] range for each input channel.

[0127] In some embodiments, when fitting across multiple sessions, input neural activity data xtcan be mapped to a shared embedding space of fixed dimensionality through a learnable linear map. By way of example, with Xkas the n x T matrix with all neural activity training data for session k, the embedding data can be given by AkXkwith A being a q x n matrix, to embed neural activity in a q -dimensional space. Conceptually, this embedding step is akin to token embedding in natural language processing. Each unit can be assigned an embedding vector capturing its “semantic” meaning relative to other units in the dataset. The parameters of each Akmatrix are trainable and the matrices can be trained simultaneously to the rest of the input module 304 for all sessions included during training. When decoding a new experimental session, a new matrix Akcan be fitted to that session. Additionally and / or alternatively, any other technique for fitting input neural activity data xtmay be used.

[0128] In some embodiments, hyperparameters of the neural decoder 302 can be tuned to improve decoding accuracy and computational efficiency during training and / or at inference. Examples of the hyperparameters include number of epochs, learning rate, batch size, patience, kernel size and number of kernels for the time-convolutional layer of the TCNN 314, number and size of the TCNN hidden layers, size of the TCNN’s output, number of layers in the input heads MLPs, choice of non-linear activation function for the TCNN and input heads and the width of the CAN input bump z^. In some embodiments, when the neural decoder 302 is fitted on data from multiple training sessions, the size of the shared unit embedding space can also be a variable hyperparameter.

[0129] In some embodiments, the CANs 318a, 318b, 318c can have one or more tunable hyperparameters. For example, the CANs 318a, 318b, 318c can have two tunablehyperparameters, P and y that affect the speed at which the activity bump moves on the CANs’ lattice in response to inputs. These hyperparameters can be tuned in accordance with the neural activity data 308. However, to further improve decoding accuracy, can be further fine-tuned for each experimental session. With ?*, y* being the values arrived at during the hyperparameter sweep, 10 alternative values ranging in [ ?* / 4, 4 C | (and equivalently for y*) can be tested. For each hyperparameter, the value yielding the most accurate decoding can be selected for deployment at inference. Advantageously, this can be done very rapidly (e.g., less than 10 seconds per CAN), because such fine tuning does not require re-training of the networks of the input module 304. Additionally and / or alternatively, other hyperparameter(s) of the CANs 318a, 318b, 318c may be tuned and / or tuned using any other suitable technique.

[0130] In some embodiments, the value of the latent variable represented by each CAN depends on the location of its activity bump on the CAN’s lattice: 6* . However, while 0*6 [0, 1] for the line attractor CAN and 0*6 [0, 2K] for the ring attractor CAN, the ranges of the decoded variables may vary across datasets and variables. For example, position in the dataset for the one-dimensional corridor example provided above was a number in range [0, 1.6]. Thus, during training, a mapping can be defined to scale the range of the jthvariable to the jthCAN’s lattice range. This mapping can be used to generate the training labels for the input module 304. During decoding, the inverse mapping can be used to reconstruct the decoded variable’s value from the bumps position on the lattice.

[0131] Alternatively, in some embodiments, the neural decoder 302 may be trained end-to-end to minimize and / or otherwise reduce decoding error (e.g., the difference between ytand ytgiven ground truth data).

[0132] The neural decoder 302 and / or, more generally, the BMI system 300 of FIG. 3 affords several benefits. An example benefit is interpretability, which is important when the BMI system 300 is used in and / or in connection with medical devices. For example, CAN function is achieved through a precisely defined and mathematically grounded construction. This renders their function entirely transparent to an external observer, in stark contrast with the “black-box” nature of conventional neural networks that undergo extensive learning procedures. The neural decoder’s 302 use of CANs 318a, 318b, 318c in the representation module 306 endows it with a similar degree of interpretability. Outputs of the neural decoder 302 are readily explainable by analysis of the states of the CANs 318a, 318b, 318c. These, in turn, depend on the history of inputs produced by the input module 304. Advantageously, the mechanism by which the inputs z and the CANs’ states interact is also well understood. 1Only the mapping xt— z is less readily interpretable since it depends on the behavior of trained networks in the input module. However, the use of relatively small and / or simple neural networks minimizes and / or otherwise reduces this effect. In addition, the neural decoder 302 modular architecture allows for changes to the input module 304 without affecting the configuration and / or design of the representation module 306. Thus, the input module 304 could easily be replaced with a more interpretable architecture without affecting other aspects of the neural decoder 302 function.

[0133] Another example benefit is the neural decoder’ s 302 ability to leverage substantially large datasets. For example, while the neural decoder 302 can be configured to maximize data efficiency to operate in the data-poor regime common to BMI applications, the neural decoder 302 can be configured to leverage larger datasets when they are available. Advantageously, the neural decoder 302 may incur only a minimal penalty in terms of computational hardware efficiency and no reduction in interpretability when leveraging substantially large datasets.

[0134] Yet another example benefit is the computational hardware accuracy of the neural decoder 302 with respect to conventional decoder architectures. Hardware efficiency can refer to an algorithm’ s ability to fit the data and do inference without requiring large, power hungry computer hardware which is not compatible with the design of portable BMI devices. Large, foundational, decoder models typically rely on significant hardware resources to speed up training and allow for real-time inference. The design of the neural decoder 302 allows it to be highly efficient during both training and inference. Indeed, the inventors have recognized that the neural decoder’s 302 training is significantly faster than that of other recurrent architectures of similar size when trained on a single GPU.

[0135] In addition, the neural decoder’s 302 data efficiency allows it to achieve high decoding accuracy using only a small amount of data, suggesting that the neural decoder’s 302 training can be made even more efficient by reducing the size of the dataset without loss in accuracy. Resources required for training a decoder are an important aspect of hardware efficiency, especially considering the need for periodic retraining. Also important is an algorithm’s ability to support real-time BMI applications by enabling fast inference with limited hardware requirement. The inventors have recognized that the neural decoder’ s 302 inference step to be fast, taking approximately one millisecond per sample on a central processing unit (CPU). The architecture of the neural decoder 302 therefore is more hardware efficient than other deep-learning approaches while also achieving superior decodingaccuracy, enabling real-time BMI applications with minimal and / or otherwise reduced hardware requirements.

[0136] Another example benefit is data efficiency. Data efficiency refers to the ability to achieve a certain level of performance (e.g., accuracy) with minimal requirements for training data. The neural decoder 302 design leverages CANs and a modular structure to greatly increase data efficiency. The inventors have recognized that, in the context of BMI / HCI applications, data efficiently greatly improves user experience and, thus, the neural decoder 302 can deliver an improved user experience than that of other recurrent architectures.

[0137] Yet another example benefit is the decoding accuracy of the neural decoder 302 with respect conventional decoder architectures, which is shown in Table 1 below. In Table 1 below, five conventional datasets were evaluated using conventional machine learning models (e.g., RNN, GRU, LSTM) and the neural decoder as disclosed herein, such as the neural decoder 102 of FIG. 1 and / or the neural decoder 302 of FIG. 3. The values in Table 1 represent the decoding accuracy in terms of median MSE. The methodology used to generate the below values in Table 1 is merely an example and other dataset(s) and / or technique(s) to benchmark the neural decoder 102 of FIG. 1 and / or the neural decoder 302 of FIG. 3 may be used.

[0138] The evaluated datasets included hippocampal recordings in mice and rats navigating one or two dimensional arenas (Place ID, Place2D), recordings of thalamic head direction cells in mice freely moving in a two dimensional arena (Head Direction), entorhinal cortex recording in mice navigating a Y-shaped maze (Navigation) and recordings from a human subject engaged in a BMI cursor control task (Bimanual Cursor). The evaluated datasets are merely examples and any other dataset and / or technique for benchmarking may be used.

[0139] The example datasets varied in the number and nature of behavioral variables being decoded, and each dataset was composed of multiple experimental sessions varying in length and number of simultaneously recorded units. After optimization of each decoder’s hyperparameters for each dataset, decoding accuracy on held-out validation data was assessed across multiple repeats of each experiment. As illustrated in Table 1 below, the neural decoder as disclosed herein was the most accurate decoder in four out of five datasets. In Bimanual Cursor, the neural decoder as disclosed herein was second best performing only marginally worse than LSTM. Compared to the next best architecture, the neural decoder as disclosed herein was 55% more accurate in PlacelD, 25% in Place2D and 82% inHeadDirection (in terms of reduction in MSE) while being roughly as accurate in Navigation and only 2% less accurate in Bimanual Cursor. Advantageously, the neural decoder as disclosed herein reliably exhibits superior decoding accuracy to all other decoder architectures commonly used in BMI applications.Table 1

[0140] While an example implementation of the neural decoder 102 of FIG. 1 is depicted in FIG. 3, other implementations are contemplated. For example, one or more blocks, components, functions, etc., of the neural decoder 302 may be combined or divided in any other way. The neural decoder 302 of the illustrated example may be implemented by hardware alone, or by a combination of hardware, software, and / or firmware. For example, the neural decoder 302 may be implemented by one or more analog or digital circuits (e.g., comparators, operational amplifiers, etc.), one or more hardware-implemented state machines, one or more programmable processors (e.g., central processing units (CPUs), digital signal processors (DSPs), field programmable gate arrays (FPGAs), graphics processing units (GPUs), etc.), one or more network interfaces (e.g., network interface circuitry, network interface cards (NICs), smart NICs, etc.), one or more application specific integrated circuits (ASICs), one or more memories (e.g., non-volatile memory, volatile memory, etc.), one or more mass storage disks or devices (e.g., hard-disk drives (HDDs), solid-state disk (SSD) drives, etc.), etc., and / or any combination(s) thereof.

[0141] In some embodiments, the neural decoder 302, or portion(s) thereof, may be implemented by a system on a chip or system-on-chip (SoC). An SoC is an integrated circuit design that combines elements of an electronic device onto a single chip instead of using separate components. For example, an SoC may include and / or incorporate within itself one or more programmable processors, input and output (I / O) ports, memory, analog inputblocks, analog output blocks, etc., and / or any combination(s) thereof. For example, the neural decoder 302 may be implemented by a single platform and integrates an entire electronic device (or portion(s) thereof), such as a receiver device, onto the platform.

[0142] In some embodiments, the BMI system 300 of FIG. 3 or portion(s) thereof can be implemented by one or more wearable devices. For example, one(s) of the sensor(s) 310, the sensor interface module 312, the input module 304, the representation module 306, the device control interface 332, and / or the device 330 can be implemented by one or more wearable devices. Examples of a wearable device include an augmented reality and / or virtual reality (AR / VR) device, a heads-up display (HUD) device, a fitness tracker, a smartwatch, smart glasses, smart goggles, a medical device patch, and a medical bracelet.

[0143] FIG. 4A depicts an example implementation of a neural decoder 400 to convert neural activity data 402 into a lattice position yton a BMI control manifold 404. In some embodiments, the neural activity data 402 can correspond to the neural activity data 104 of FIG. 1 and / or the neural activity data 308 of FIG. 3.

[0144] In some embodiments, the neural decoder 302 of FIG. 3 can implement the neural decoder 400 of this example. For example, the neural decoder 400 includes an input module 406 and a representation module 408 that can correspond to the input module 304 and the representation module 306 of FIG. 3, respectively.

[0145] In some embodiments, the neural decoder 102 of FIG. 1 and / or the neural decoder 302 of FIG. 3 can be configured in accordance with a BMI control manifold topology. For example, as shown in FIG. 4A, the representation module 408 can include two line attractor CANs 410, 412 that, collectively, construct a manifold equivalent to the BMI control manifold 404. In some embodiments, the line attractor CANs 410, 412 can correspond to the first CAN 318a and the second CAN 318b of FIG. 3, respectively.

[0146] The BMI control manifold 404 of this example has 2 degrees of freedom. Accordingly, the representation module 408 can include 2 line attractor CANs as shown such that the output of the representation module 408 is a 2-dimensional vector ytwhere the ithentry consists of the value encoded in the state of the ithCAN (e.g., the position of its activity bump on the CAN’s lattice). The representation module 408 can allow the neural decoder 400 to encode the neural activity data 402 on the pre- specified attractor manifolds represented by the line attractor CANs 410, 412.

[0147] FIG. 4B depicts an example implementation of a neural decoder 420 to convert neural activity data 422 into a lattice position yton a BMI control manifold 424. Insome embodiments, the neural activity data 422 can correspond to the neural activity data 104 of FIG. 1 and / or the neural activity data 308 of FIG. 3.

[0148] In some embodiments, the neural decoder 302 of FIG. 3 can implement the neural decoder 420 of this example. For example, the neural decoder 420 includes an input module 426 and a representation module 428 that can correspond to the input module 304 and the representation module 306 of FIG. 3, respectively.

[0149] In some embodiments, the neural decoder 102 of FIG. 1 and / or the neural decoder 302 of FIG. 3 can be configured in accordance with a BMI control manifold topology. For example, as shown in FIG. 4B, the representation module 428 can include two ring attractor CANs 430, 432 that, collectively, construct a manifold equivalent to the BMI control manifold 424. In some embodiments, the ring attractor CANs 430, 432 can correspond to the first CAN 318a and the second CAN 318b of FIG. 3, respectively.

[0150] The BMI control manifold 424 of this example has 2 degrees of freedom. Accordingly, the representation module 428 can include 2 ring attractor CANs as shown such that the output of the representation module 428 is a 2-dimensional vector ytwhere the ithentry consists of the value encoded in the state of the ithCAN (e.g., the position of its activity bump on the CAN’s lattice). The representation module 428 can allow the neural decoder 420 to encode the neural activity data 422 on the pre- specified attractor manifolds represented by the ring attractor CANs 430, 432.

[0151] FIG. 5A shows an example connectivity matrix 500 for a line attractor 502. For example, the second CAN 116 of FIG. 1, one(s) of the CANs 318a, 318b, 318c of FIG. 3, and / or the line attractor CANs 410, 412 can have the precisely defined connectivity structure shown in the connectivity matrix 500.

[0152] FIG. 5B shows an example connectivity matrix 510 for a ring attractor 512. As shown, the connectivity for the ring attractor 512 demonstrates periodic symmetric structure, which is depicted by the off-diagonal elements indicated by arrows. For example, the first CAN 114 of FIG. 1, one(s) of the CANs 318a, 318b, 318c of FIG. 3, and / or the ring attractor CANs 430, 432 can have the precisely defined connectivity structure shown in the connectivity matrix 510.

[0153] FIG. 6 shows mappings of states 602, 604 of a CAN activity manifold 600 to respective bumps 606, 608 of activity. The CAN activity manifold 600 shown is a lower dimensional space arrived through Principal Component Analysis (PCA). The CAN activity manifold 600 is a line attractor activity manifold with the two states 602, 604. In someembodiments, one(s) of the CANs 318a, 318b, 318c of FIG. 3 can have the CAN activity manifold 600 shown in FIG. 6.

[0154] In some embodiments, a CAN’s neural states are points on its activity manifold. For example, the neural states of the CANs 318a, 318b, 318c can be represented as points on their respective activity manifolds. As shown in FIG. 6, the neural states 602, 604 are points on the CAN activity manifold 600.

[0155] In some embodiments, a CAN’s neural states can be defined by localized bumps of activity on the CAN’s lattice. For example, the locations of such bumps on the lattice can correspond to different values of the latent variable represented by the CAN. As shown in FIG. 6, the two states 602, 604 are visualized as bumps 606, 608 of activity on the line attractor’s lattice.

[0156] By way of example in which the first CAN 318a has the CAN activity manifold 600 of FIG. 6, the neural states of the first CAN 318a can be defined by localized bumps of activity on the CAN’s lattice, which are shown in FIG. 6 as the bumps 606, 608. For example, the locations of the bumps 606, 608 on the CAN activity manifold 600 can correspond to different values of the latent variable represented by the first CAN 318a. Also shown in FIG. 6 are dotted lines 610, 612, which indicate the value of the decoded variable based on the locations of the bumps 606, 608.

[0157] FIG. 7A is a schematic illustration of the CAN input scheme, which shows a first mapping of an input zt702 to one or more CANs (e.g., one(s) of the CANs 318a, 318b, 318c of FIG. 3). For example, the input 702 can correspond to the one or more sets of inputs 113 of FIG. 1 and / or the one or more sets of inputs 326 of FIG. 3. In such an example, the one or more sets of inputs 113 and / or the one or more sets of inputs 326 can correspond to an activation at each lattice position, where each of the CANs 318a, 318b, 318c is at a respective lattice position.

[0158] As shown in FIG. 7A, the input zt702 is a valid input that causes an activity bump ht704 to move along the lattice as indicated by the direction of the arrow 706. As shown, the activity bump 704 moves along the lattice by ht+1. For example, the input zt702 to one of the CANs 318a, 318b, 318c can specify which units in the one of the CANs 318a, 318b, 318c should receive any activation. In such an example, the CAN’s activity bump will move towards the activated units if these units are close to the activity bump’s original location. Across CANs, different inputs ztcause each CAN’s bump to move independently from those of other CANs.

[0159] FIG. 7B is another schematic illustration of the CAN input scheme, which shows a second mapping of an input zt712 to a CAN (e.g., one(s) of the CANs 318a, 318b, 318c of FIG. 3) to a lattice position of the CAN. For example, the input 712 can correspond to the one or more sets of inputs 113 of FIG. 1 and / or the one or more sets of inputs 326 of FIG. 3.

[0160] As shown in FIG. 7B, the input zt712 is an invalid input that does not cause an activity bump ht714 to move along the lattice. Advantageously, an invalid input is ignored through the winner-take-all CAN dynamics.

[0161] FIG. 8 is a flowchart 800 representative of an example process that may be performed and / or example machine-readable instructions that may be executed by processor circuitry to implement a neural decoder as disclosed herein, such as the neural decoder 102 of FIG. 1 and / or the neural decoder 302 of FIG. 3, to convert neural activity data into control of a device. Additionally or alternatively, block(s) of the flowchart of FIG. 8 may be representative of state(s) of one or more hardware-implemented state machines, algorithm(s) that may be implemented by hardware alone such as an ASIC, etc., and / or any combination(s) thereof.

[0162] The flowchart 800 of FIG. 8 begins at block 802, at which the neural decoder 102, 302 may obtain neural activity data. For example, the input module 304 of FIG. 3 can receive the neural activity data 308 from the sensor(s) 310 via the sensor interface module 312. Additionally and / or alternatively, the neural decoder 102, 302 may more generally implement a biophysical decoder that can be configured to receive biophysical activity data from the sensor(s) 310 via the sensor interface module 310. In such an example, the biophysical activity data may include neural activity data and / or non-neural activity data.

[0163] At block 804, the neural decoder 102, 302 may input the neural activity data into an input module. For example, the first machine learning model 314 of FIG. 3 can process the neural activity data 308 into the intermediate output 324. Additionally and / or alternatively, the neural decoder 102, 302 may more generally implement a biophysical decoder that can be configured to process, using the first machine learning model 314, the biophysical activity data into the intermediate output 324.

[0164] At block 806, the neural decoder 102, 302 may output from the input module an input for respective continuous attractor network(s). For example, the second machine learning models 316a, 316b, 316c can process the intermediate output 324 into the one or more sets of inputs represented by ztl326. Additionally and / or alternatively, the neuraldecoder 102, 302 may more generally implement a biophysical decoder that can be configured to process, using the second machine learning models 316a, 316b, 316c, the intermediate output 324 into the one or more sets of inputs represented by z 326.

[0165] At block 808, the neural decoder 102, 302 may input the input(s) into the respective continuous attractor network(s). For example, the CANs 318a, 318b, 318c can process the one or more sets of inputs represented by z 326 into the output htl328 representative of the change to the state of one(s) of the CANs 318a, 318b, 318c. Additionally and / or alternatively, the neural decoder 102, 302 may more generally implement a biophysical decoder that can be configured to process, using the CANs 318a, 318b, 318c, the one or more sets of inputs represented by z 326 into the output htl328 representative of the change to the state of one(s) of the CANs 318a, 318b, 318c.

[0166] At block 810, the neural decoder 102, 302 may output from the continuous attractor network(s) an output for control of a device. For example, the CANs 318a, 318b, 318c can output the output 328 to the aggregator 320. In such an example, the aggregator 320 can process the change to the state of one(s) of the CANs 318a, 318b, 318c into the output 322, which can be a value in a variable range associated with control of the device 330 (e.g., a value in a range of [0, 3000] pixels for computer cursor position control). Additionally and / or alternatively, the neural decoder 102, 302 may more generally implement a biophysical decoder that can be configured to output the output 328 to the aggregator 320.

[0167] At block 812, the neural decoder 102, 302 may control the device using the output. For example, the device control interface 332 can process the output 322 into one or more commands, directions, instructions, etc., for control of the device 330 in accordance with an intention of a user from which the neural activity data 308 is obtained.

[0168] At block 814, the neural decoder 102, 302 may determine whether to continue to monitor for new neural activity data. For example, the input module 304 can determine whether new neural activity data 308 is received from the sensor(s) 310 via the sensor interface module 312. Additionally and / or alternatively, the neural decoder 102, 302 may more generally implement a biophysical decoder that can be configured to determine whether new biophysical activity data is received from the sensor(s) 310 via the sensor interface module 312.

[0169] If, at block 814, the neural decoder 102, 302 determines to continue to monitor for new neural activity data, control returns to block 802 to obtain new neural activity data.Otherwise, the example flowchart 800 of FIG. 8 concludes.

[0170] FIG. 9 is an example implementation of an electronic platform 900 structured to execute the machine -readable instructions of FIG. 8 to implement a neural decoder, such as the neural decoder 102 of FIG. 1 and / or the neural decoder 302 of FIG. 3 and / or, more generally, a BMI system, such as the BMI system 100 of FIG. 1 and / or the BMI system 300 of FIG. 3. Additionally and / or alternatively, the electronic platform 900 may be structured to execute the machine -readable instructions of FIG. 8 to implement a biophysical decoder. It should be appreciated that FIG. 9 is intended neither to be a description of necessary components for an electronic and / or computing device to operate as a neural decoder and / or a BMI system, in accordance with the techniques described herein, nor a comprehensive depiction.

[0171] The electronic platform 900 of this example may be an electronic device, such as a handset device (e.g., a cellular network device, a smartphone, etc.), a desktop computer, a laptop computer, a tablet computer, a server (e.g., a computer server, a blade server, a rackmounted server, etc.), a wearable device (e.g., an AR / VR device, a HUD device, a fitness tracker, a smartwatch, smart glasses, smart goggles, a medical device patch, a medical bracelet, etc.), a workstation, or any other type of computing and / or electronic device.

[0172] The electronic platform 900 of the illustrated example includes processor circuitry 902, which may be implemented by one or more programmable processors, one or more hardware-implemented state machines, one or more ASICs, etc., and / or any combination(s) thereof. For example, the one or more programmable processors may include one or more CPUs, one or more DSPs, one or more FPGAs, one or more GPUs, etc., and / or any combination(s) thereof. The processor circuitry 902 includes processor memory 904, which may be volatile memory, such as random-access memory (RAM) of any type. The processor circuitry 902 of this example implements the input module 304 and the representation module 306 of FIG. 3. For example, the processor circuitry 902 may implement one(s) of the first machine learning model 314, the second machine learning models 316, the CANs 318a, 318b, 318c, and / or the aggregator 320 of FIG. 3.

[0173] The processor circuitry 902 may execute machine -readable instructions 906 (identified by INSTRUCTIONS), which are stored in the processor memory 904, to implement at least one of the input module 304 or the representation module 306 of FIG. 3. The machine-readable instructions 906 may include data representative of computerexecutable and / or machine-executable instructions implementing techniques that operate according to the techniques described herein. For example, the machine-readable instructions906 may include data (e.g., code, embedded software (e.g., firmware), software, etc.) representative of the flowchart of FIG. 8, or portion(s) thereof.

[0174] The electronic platform 900 includes memory 908, which may include the instructions 906. The memory 908 of this example may be controlled by a memory controller 910. For example, the memory controller 910 may control reads, writes, and / or, more generally, access(es) to the memory 908 by other component(s) of the electronic platform 900. The memory 908 of this example may be implemented by volatile memory, non-volatile memory, etc., and / or any combination(s) thereof. For example, the volatile memory may include static random-access memory (SRAM), dynamic random-access memory (DRAM), cache memory (e.g., Level 1 (LI) cache memory, Level 2 (L2) cache memory, Level 3 (L3) cache memory, etc.), etc., and / or any combination(s) thereof. In some examples, the nonvolatile memory may include Flash memory, electrically erasable programmable read-only memory (EEPROM), magnetoresistive random-access memory (MRAM), ferroelectric random-access memory (FeRAM, F-RAM, or FRAM), etc., and / or any combination(s) thereof.

[0175] The electronic platform 900 includes input device(s) 912 to enable data and / or commands to be entered into the processor circuitry 902. For example, the input device(s) 912 may include an audio sensor, a camera (e.g., a still camera, a video camera, etc.), a keyboard, a microphone, a mouse, a touchscreen, a voice recognition system, etc., and / or any combination(s) thereof.

[0176] The electronic platform 900 includes output device(s) 914 to convey, display, and / or present information to a user (e.g., a human user, a machine user, etc.). For example, the output device(s) 914 may include one or more display devices, speakers, etc. The one or more display devices may include an augmented reality (AR) and / or virtual reality (VR) display, a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED) display, a quantum dot (QLED) display, a thin-film transistor (TFT) LCD, a touchscreen, etc., and / or any combination(s) thereof. The output device(s) 914 can be used, among other things, to generate, launch, and / or present a user interface. For example, the user interface may be generated and / or implemented by the output device(s) 914 for visual presentation of output and speakers or other sound generating devices for audible presentation of output.

[0177] The electronic platform 900 includes accelerators 916, which are hardware devices to which the processor circuitry 902 may offload compute tasks to accelerate their processing. For example, the accelerators 916 may include artificial intelligence / machine-leaming( AI / ML) processors, ASICs, FPGAs, graphics processing units (GPUs), neural network (NN) processors, systems-on-chip (SoCs), vision processing units (VPUs), etc., and / or any combination(s) thereof.

[0178] In some embodiments, the input module 304 and / or the representation module 306 circuitry may be implemented by one(s) of the accelerators 916 instead of the processor circuitry 902. For example, one(s) of the first machine learning model 314, the second machine learning models 316, the CANs 318a, 318b, 318c, and / or the aggregator 320 of FIG. 3 may be implemented by one(s) of the accelerators 916 instead of the processor circuitry 902.

[0179] In some embodiments, the input module 304 and / or the representation module 306 may be executed concurrently (e.g., in parallel, substantially in parallel, etc.) by the processor circuitry 902 and the accelerators 916. For example, the processor circuitry 902 and one(s) of the accelerators 916 may execute in parallel function(s) corresponding to the input module 304. In another example, the processor circuitry 902 and one(s) of the accelerators 916 may execute in parallel function(s) corresponding to the representation module 306. In yet another example, the processor circuitry 902 and one(s) of the accelerators 916 may execute in parallel function(s) corresponding to one(s) of the first machine learning model 314, the second machine learning models 316, the CANs 318a, 318b, 318c, or the aggregator 320.

[0180] The electronic platform 900 includes storage 918 to record and / or control access to data, such as the machine-readable instructions 906. The storage 918 may be implemented by one or more mass storage disks or devices, such as HDDs, SSDs, etc., and / or any combination(s) thereof.

[0181] The electronic platform 900 includes interface(s) 920 to effectuate exchange of data with external devices (e.g., computing and / or electronic devices of any kind) via a network 922. In this example, the interface(s) 920 may optionally implement(s) the sensor interface module 312 and / or the device control interface 332 of FIG. 3.

[0182] The interface(s) 920 of the illustrated example may be implemented by an interface device, such as network interface circuitry (e.g., a NIC, a smart NIC, etc.), a gateway, a router, a switch, etc., and / or any combination(s) thereof. The interface(s) 920 may implement any type of communication interface, such as BLUETOOTH®, a cellular telephone system (e.g., a 4G LTE interface, a 5G interface, a future generation 6G interface, etc.), an Ethernet interface, a near-field communication (NFC) interface, an optical disc interface (e.g., a Blu- ray disc drive, a Compact Disk (CD) drive, a Digital Versatile Disk (DVD) drive, etc.), an optical fiber interface, a satellite interface (e.g., a BLOS satellite interface, a LOS satelliteinterface, etc.), a Universal Serial Bus (USB) interface (e.g., USB Type-A, USB Type-B, USB TYPE-C™ or USB-C™, etc.), etc., and / or any combination(s) thereof.

[0183] The electronic platform 900 includes a power supply 924 to store energy and provide power to components of the electronic platform 900. The power supply 924 may be implemented by a power converter, such as an alternating current-to-direct-current (AC / DC) power converter, a direct current- to-direct current (DC / DC) power converter, etc., and / or any combination(s) thereof. For example, the power supply 924 may be powered by an external power source, such as an alternating current (AC) power source (e.g., an electrical grid), a direct current (DC) power source (e.g., a battery, a battery backup system, etc.), etc., and the power supply 924 may convert the AC input or the DC input into a suitable voltage for use by the electronic platform 900. In some examples, the power supply 924 may be a limited duration power source, such as a battery (e.g., a rechargeable battery such as a lithium-ion battery).

[0184] Component(s) of the electronic platform 900 may be in communication with one(s) of each other via a bus 926. For example, the bus 926 may be any type of computing and / or electrical bus, such as an I2C bus, a PCI bus, a PCIe bus, a SPI bus, and / or the like.

[0185] The network 922 may be implemented by any wired and / or wireless network(s) such as one or more cellular networks (e.g., 4G ETE cellular networks, 5G cellular networks, future generation 6G cellular networks, etc.), one or more data buses, one or more local area networks (LANs), one or more optical fiber networks, one or more private networks, one or more public networks, one or more wireless local area networks (WLANs), etc., and / or any combination(s) thereof. For example, the network 922 may be the Internet, but any other type of private and / or public network is contemplated.

[0186] The network 922 of the illustrated example facilitates communication between the interface(s) 920 and a central facility 928. The central facility 928 in this example may be an entity associated with one or more servers, such as one or more physical hardware servers and / or virtualizations of the one or more physical hardware servers. For example, the central facility 928 may be implemented by a public cloud provider, a private cloud provider, etc., and / or any combination(s) thereof. In this example, the central facility 928 may compile, generate, update, etc., the machine-readable instructions 906 and store the machine-readable instructions 906 for access (e.g., download) via the network 922. For example, the electronic platform 900 may transmit a request, via the interface(s) 920, to the central facility 928 for the machine-readable instructions 906 and receive the machine-readable instructions 906 from the central facility 928 via the network 922 in response to the request.

[0187] Additionally or alternatively, the interface(s) 920 may receive the machine-readable instructions 906 via non-transitory machine-readable storage media, such as an optical disc 930 (e.g., a Blu-ray disc, a CD, a DVD, etc.) or any other type of removable non-transitory machine-readable storage media such as a USB drive 932. For example, the optical disc 930 and / or the USB drive 932 may store the machine-readable instructions 906 thereon and provide the machine -readable instructions 906 to the electronic platform 900 via the interface(s) 920. Further shown, the interface(s) 920 may receive data from and / or transmit data to the sensor(s) 310 and / or the device 330 of FIG. 3.

[0188] Techniques operating according to the principles described herein may be implemented in any suitable manner. The processing and decision blocks of the flowcharts above represent steps and acts that may be included in algorithms that carry out these various processes. Algorithms derived from these processes may be implemented as software integrated with and directing the operation of one or more single- or multi-purpose processors, may be implemented as functionally equivalent circuits such as a DSP circuit or an ASIC, or may be implemented in any other suitable manner. It should be appreciated that the flowcharts included herein do not depict the syntax or operation of any particular circuit or of any particular programming language or type of programming language. Rather, the flowcharts illustrate the functional information one skilled in the art may use to fabricate circuits or to implement computer software algorithms to perform the processing of a particular apparatus carrying out the types of techniques described herein. For example, the flowcharts, or portion(s) thereof, may be implemented by hardware alone (e.g., one or more analog or digital circuits, one or more hardware-implemented state machines, etc., and / or any combination(s) thereof) that is configured or structured to carry out the various processes of the flowcharts. In some examples, the flowcharts, or portion(s) thereof, may be implemented by machine-executable instructions (e.g., machine-readable instructions, computer-readable instructions, computer-executable instructions, etc.) that, when executed by one or more single- or multi-purpose processors, carry out the various processes of the flowcharts. It should also be appreciated that, unless otherwise indicated herein, the particular sequence of steps and / or acts described in each flowchart is merely illustrative of the algorithms that may be implemented and can be varied in implementations and embodiments of the principles described herein.

[0189] Accordingly, in some embodiments, the techniques described herein may be embodied in machine-executable instructions implemented as software, including as application software, system software, firmware, middleware, embedded code, or any othersuitable type of computer code. Such machine-executable instructions may be generated, written, etc., using any of a number of suitable programming languages and / or programming or scripting tools, and also may be compiled as executable machine language code or intermediate code that is executed on a framework, virtual machine, or container.

[0190] When techniques described herein are embodied as machine-executable instructions, these machine-executable instructions may be implemented in any suitable manner, including as a number of functional facilities, each providing one or more operations to complete execution of algorithms operating according to these techniques. A “functional facility,” however instantiated, is a structural component of a computer system that, when integrated with and executed by one or more computers, causes the one or more computers to perform a specific operational role. A functional facility may be a portion of or an entire software element. For example, a functional facility may be implemented as a function of a process, or as a discrete process, or as any other suitable unit of processing. If techniques described herein are implemented as multiple functional facilities, each functional facility may be implemented in its own way; all need not be implemented the same way.Additionally, these functional facilities may be executed in parallel and / or serially, as appropriate, and may pass information between one another using a shared memory on the computer(s) on which they are executing, using a message passing protocol, or in any other suitable way.

[0191] Generally, functional facilities include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. Typically, the functionality of the functional facilities may be combined or distributed as desired in the systems in which they operate. In some implementations, one or more functional facilities carrying out techniques herein may together form a complete software package. These functional facilities may, in alternative embodiments, be adapted to interact with other, unrelated functional facilities and / or processes, to implement a software program application.

[0192] Some exemplary functional facilities have been described herein for carrying out one or more tasks. It should be appreciated, though, that the functional facilities and division of tasks described is merely illustrative of the type of functional facilities that may implement using the exemplary techniques described herein, and that embodiments are not limited to being implemented in any specific number, division, or type of functional facilities. In some implementations, all functionalities may be implemented in a single functional facility. It should also be appreciated that, in some implementations, some of the functional facilitiesdescribed herein may be implemented together with or separately from others (e.g., as a single unit or separate units), or some of these functional facilities may not be implemented.

[0193] Machine-executable instructions (e.g., processor-executable instructions) implementing the techniques described herein (when implemented as one or more functional facilities or in any other manner) may, in some embodiments, be encoded on one or more computer-readable media, machine-readable media, etc., to provide functionality to the media. Computer-readable media, machine -readable media, etc., include magnetic media such as a hard disk drive, optical media such as a CD or a DVD, a persistent or non-persistent solid-state memory (e.g., Flash memory, Magnetic RAM, etc.), or any other suitable storage media. Such a computer-readable medium, a machine-readable medium, etc., may be implemented in any suitable manner. As used herein, the terms “computer-readable media” (also called “computer-readable storage media”), “computer-readable medium” (also called “computer-readable storage medium”), “machine-readable media” (also called “machine- readable storage media”), and “machine-readable medium” (also called “machine-readable storage medium”) refer to tangible storage media. Tangible storage media are non-transitory and have at least one physical, structural component. In a “computer-readable medium” and “machine-readable medium” as used herein, at least one physical, structural component has at least one physical property that may be altered in some way during a process of creating the medium with embedded information, a process of recording information thereon, or any other process of encoding the medium with information. For example, a magnetization state of a portion of a physical structure of a computer-readable medium, a machine-readable medium, etc., may be altered during a recording process.

[0194] Further, some techniques described above comprise acts of storing information (e.g., data and / or instructions) in certain ways for use by these techniques. In some implementations of these techniques — such as implementations where the techniques are implemented as machine-executable instructions — the information may be encoded on a computer-readable storage media. Where specific structures are described herein as advantageous formats in which to store this information, these structures may be used to impart a physical organization of the information when encoded on the storage medium. These advantageous structures may then provide functionality to the storage medium by affecting operations of one or more processors interacting with the information; for example, by increasing the efficiency of computer operations performed by the processor(s).

[0195] In some, but not all, implementations in which the techniques may be embodied as machine-executable instructions, these instructions may be executed on one or more suitablecomputing device(s) and / or electronic device(s) operating in any suitable computer and / or electronic system, or one or more computing devices (or one or more processors of one or more computing devices) and / or one or more electronic devices (or one or more processors of one or more electronic devices) may be programmed to execute the machine-executable instructions. A computing device, electronic device, or processor (e.g., processor circuitry) may be programmed to execute instructions when the instructions are stored in a manner accessible to the computing device, electronic device, or processor, such as in a data store (e.g., an on-chip cache or instruction register, a computer-readable storage medium and / or a machine-readable storage medium accessible via a bus, a computer-readable storage medium and / or a machine-readable storage medium accessible via one or more networks and accessible by the device / processor, etc.). Functional facilities comprising these machineexecutable instructions may be integrated with and direct the operation of a single multipurpose programmable digital computing device, a coordinated system of two or more multipurpose computing device sharing processing power and jointly carrying out the techniques described herein, a single computing device or coordinated system of computing device (colocated or geographically distributed) dedicated to executing the techniques described herein, one or more FPGAs for carrying out the techniques described herein, or any other suitable system.

[0196] Embodiments have been described where the techniques are implemented in circuitry and / or machine-executable instructions. It should be appreciated that some embodiments may be in the form of a method, of which at least one example has been provided. The acts performed as part of the method may be ordered in any suitable way. Accordingly, embodiments may be constructed in which acts are performed in an order different than illustrated, which may include performing some acts simultaneously, even though shown as sequential acts in illustrative embodiments.

[0197] Various aspects of the embodiments described above may be used alone, in combination, or in a variety of arrangements not specifically discussed in the embodiments described in the foregoing and is therefore not limited in its application to the details and arrangement of components set forth in the foregoing description or illustrated in the drawings. For example, aspects described in one embodiment may be combined in any manner with aspects described in other embodiments.

[0198] The phrase “and / or,” as used herein in the specification and in the claims, should be understood to mean “either or both,” of the elements so conjoined, e.g., elements that are conjunctively present in some cases and disjunctively present in other cases. Multipleelements listed with “and / or” should be construed in the same fashion, e.g., “one or more” of the elements so conjoined. Other elements may optionally be present other than the elements specifically identified by the “and / or” clause, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, a reference to “A and / or B,” when used in conjunction with open-ended language such as “comprising” can refer, in one embodiment, to A only (optionally including elements other than B); in another embodiment, to B only (optionally including elements other than A); in yet another embodiment, to both A and B (optionally including other elements); etc.

[0199] The indefinite articles “a” and “an,” as used herein in the specification and in the claims, unless clearly indicated to the contrary, should be understood to mean “at least one.”

[0200] As used herein in the specification and in the claims, the phrase, “at least one,” in reference to a list of one or more elements, should be understood to mean at least one element selected from any one or more of the elements in the list of elements, but not necessarily including at least one of each and every element specifically listed within the list of elements and not excluding any combinations of elements in the list of elements. This definition also allows that elements may optionally be present other than the elements specifically identified within the list of elements to which the phrase “at least one” refers, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, “at least one of A and B” (or, equivalently, “at least one of A or B,” or, equivalently, “at least one of A and / or B”) can refer, in one embodiment, to at least one, optionally including more than one, A, with no B present (and optionally including elements other than B); in another embodiment, to at least one, optionally including more than one, B, with no A present (and optionally including elements other than A); in yet another embodiment, to at least one, optionally including more than one, A, ,and at least one, optionally including more than one, B (and optionally including other elements); etc.

[0201] Use of ordinal terms such as “first,” “second,” “third,” etc., in the claims to modify a claim element does not by itself connote any priority, precedence, or order of one claim element over another or the temporal order in which acts of a method are performed, but are used merely as labels to distinguish one claim element having a certain name from another element having a same name (but for use of the ordinal term) to distinguish the claim elements.

[0202] Also, the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of “including,” “comprising,” “having,”“containing,” “involving,” and variations thereof herein, is meant to encompass the items listed thereafter and equivalents thereof as well as additional items.

[0203] All definitions, as defined and used herein, should be understood to control over dictionary definitions, definitions in documents incorporated by reference, and / or ordinary meanings of the defined terms.

[0204] The word “exemplary” is used herein to mean serving as an example, instance, or illustration. Any embodiment, implementation, process, feature, etc., described herein as exemplary should therefore be understood to be an illustrative example and should not be understood to be a preferred or advantageous example unless otherwise indicated.

[0205] Having thus described several aspects of at least one embodiment, it is to be appreciated that various alterations, modifications, and improvements will readily occur to those skilled in the art. Such alterations, modifications, and improvements are intended to be part of this disclosure and are intended to be within the spirit and scope of the principles described herein. Accordingly, the foregoing description and drawings are by way of example only.

Claims

CLAIMS1. A method for biophysical signal decoding, comprising: inputting biophysical activity data into an input module comprising one or more machine learning models and outputting, from the input module, one or more sets of inputs to change a state of one or more continuous attractor networks (CANs); inputting the one or more sets of inputs into a representation module comprising the one or more CANs and outputting, from the representation module, output representative of the change to the state of the one or more CANs; and causing control of one or more degrees of freedom of a device based on the output.

2. The method of claim 1, wherein the biophysical activity data comprises neural activity data comprising at least one of spiking activity data or non-spiking extracellular activity data, and further comprising obtaining the at least one of the spiking activity data or the non- spiking extracellular activity data from at least one sensor.

3. The method of claim 1, wherein the biophysical activity data comprises non-neural activity data comprising data from at least one of an accelerometer, an electromyograph, or one or more electrodes and further comprising obtaining the data from the at least one of the accelerometer, the electromyograph, or the one or more electrodes.

4. The method of any one of claims 1-3, wherein the one or more machine learning models comprise a first machine learning model and one or more second machine learning models, and inputting the biophysical activity data into the input module comprises: inputting the biophysical activity data into the first machine learning model and outputting, from the first machine learning model, an encoding of the biophysical activity data that is conducive to accurate decoding; and inputting the encoding of the biophysical activity data into the one or more second machine learning models and outputting, from the one or more second machine learning models, the one or more sets of inputs.

5. The method of claim 4, wherein the first machine learning model is a time convolutional neural network.

6. The method of claim 4, wherein the one or more second machine learning models are one or more multilayer perceptrons.

7. The method of claim 4, wherein the first machine learning model is trained to map the biophysical activity data onto a brain machine interface manifold.

8. The method of any one of claims 1-3, wherein the one or more sets of inputs comprise an activation for the one or more CANs at one or more respective lattice positions.

9. The method of any one of claims 1-3, wherein the one or more sets of inputs comprise a first set of inputs and a second set of inputs, the one or more CANs comprise a first CAN and a second CAN, the output comprises a first output and a second output, the one or more degrees of freedom comprise a first degree of freedom and a second degree of freedom, and further comprising: converting, using the first CAN, the first set of inputs into the first output, the first output representative of controlling the first degree of freedom of the device; and converting, using the second CAN, the second set of inputs into the second output, the second output representative of controlling the second degree of freedom of the device.

10. The method of claim 9, further comprising: aggregating the first output and the second output into a third output, and wherein causing control of the one or more degrees of freedom of the device is based on the third output.

11. The method of any one of claims 1-3, wherein the one or more CANs comprise a first CAN and a second CAN, the first CAN is configured to map a first one of the one or more sets of inputs onto a first attractor manifold, and the second CAN is configured to map a second one of the one or more sets of inputs onto a second attractor manifold.

12. The method of claim 11, wherein at least one of the first attractor manifold or the second attractor manifold is a line attractor manifold.

13. The method of claim 11, wherein at least one of the first attractor manifold or the second attractor manifold is a ring attractor manifold.

14. The method of claim 11, wherein the first attractor manifold is a line attractor manifold and the second attractor manifold is a ring attractor manifold.

15. The method of any one of claims 1-3, wherein the device is a cursor on a computer screen, a prosthetic device, a robotic arm, a speaker, or a wheelchair.

16. An apparatus for biophysical activity decoding comprising: memory storing processor-executable instructions; and at least one hardware processor configured to execute the processor-executable instructions to perform a method comprising: inputting biophysical activity data into an input module comprising one or more machine learning models and outputting, from the input module, one or more sets of inputs to change a state of one or more continuous attractor networks (CANs); inputting the one or more sets of inputs into a representation module comprising the one or more CANs and outputting, from the representation module, output representative of the change to the state of the one or more CANs; and causing control of one or more degrees of freedom of a device based on the output.

17. The apparatus of claim 16, wherein the biophysical activity data comprises neural activity data comprising at least one of spiking activity data or non-spiking extracellular activity data, and further comprising obtaining the at least one of the spiking activity data or the non-spiking extracellular activity data from at least one sensor.

18. The apparatus of claim 16, wherein the biophysical activity data comprises non-neural activity data comprising data from at least one of an accelerometer, an electromyograph, or one or more electrodes and further comprising obtaining the data from the at least one of the accelerometer, the electromyograph, or the one or more electrodes.

19. The apparatus of any one of claims 16-18, wherein the one or more machine learning models comprise a first machine learning model and one or more second machine learning models, and inputting the biophysical activity data into the input module comprises:inputting the biophysical activity data into the first machine learning model and outputting, from the first machine learning model, an encoding of the biophysical activity data that is conducive to accurate decoding; and inputting the encoding of the biophysical activity data into the one or more second machine learning models and outputting, from the one or more second machine learning models, the one or more sets of inputs.

20. The apparatus of claim 19, wherein the first machine learning model is a time convolutional neural network.

21. The apparatus of claim 19, wherein the one or more second machine learning models are one or more multilayer perceptrons.

22. The apparatus of claim 19, wherein the first machine learning model is trained to map the biophysical activity data onto a brain machine interface manifold.

23. The apparatus of any one of claims 16-18, wherein the one or more sets of inputs comprise an activation for the one or more CANs at one or more respective lattice positions.

24. The apparatus of any one of claims 16-18, wherein the one or more sets of inputs comprise a first set of inputs and a second set of inputs, the one or more CANs comprise a first CAN and a second CAN, the output comprises a first output and a second output, the one or more degrees of freedom comprise a first degree of freedom and a second degree of freedom, and the method further comprising: converting, using the first CAN, the first set of inputs into the first output, the first output representative of controlling the first degree of freedom of the device; and converting, using the second CAN, the second set of inputs into the second output, the second output representative of controlling the second degree of freedom of the device.

25. The apparatus of claim 24, the method further comprising: aggregating the first output and the second output into a third output, and wherein causing control of the one or more degrees of freedom of the device is based on the third output.

26. The apparatus of any one of claims 16-18, wherein the one or more CANs comprise a first CAN and a second CAN, the first CAN is configured to map a first one of the one or more sets of inputs onto a first attractor manifold, and the second CAN is configured to map a second one of the one or more sets of inputs onto a second attractor manifold.

27. The apparatus of claim 26, wherein at least one of the first attractor manifold or the second attractor manifold is a line attractor manifold.

28. The apparatus of claim 26, wherein at least one of the first attractor manifold or the second attractor manifold is a ring attractor manifold.

29. The apparatus of claim 26, wherein the first attractor manifold is a line attractor manifold and the second attractor manifold is a ring attractor manifold.

30. The apparatus of any one of claims 16-18, wherein the device is a cursor on a computer screen, a prosthetic device, a robotic arm, a speaker, or a wheelchair.

31. At least one computer-readable storage medium storing processor-executable instructions that, when executed by at least one hardware processor, cause the at least one hardware processor to perform a method for biophysical signal decoding comprising: inputting biophysical activity data into an input module comprising one or more machine learning models and outputting, from the input module, one or more sets of inputs to change a state of one or more continuous attractor networks (CANs); inputting the one or more sets of inputs into a representation module comprising the one or more CANs and outputting, from the representation module, output representative of the change to the state of the one or more CANs; and causing control of one or more degrees of freedom of a device based on the output.

32. The at least one computer-readable storage medium of claim 31, wherein the biophysical activity data comprises neural activity data comprising at least one of spiking activity data or non-spiking extracellular activity data, and further comprising obtaining the at least one of the spiking activity data or the non- spiking extracellular activity data from at least one sensor.

33. The at least one computer-readable storage medium of claim 31, wherein the biophysical activity data comprises non-neural activity data comprising data from at least one of an accelerometer, an electromyograph, or one or more electrodes and further comprising obtaining the data from the at least one of the accelerometer, the electromyograph, or the one or more electrodes.

34. The at least one computer-readable storage medium of any one of claims 31-33, wherein the one or more machine learning models comprise a first machine learning model and one or more second machine learning models, and inputting the biophysical activity data into the input module comprises: inputting the biophysical activity data into the first machine learning model and outputting, from the first machine learning model, an encoding of the biophysical activity data that is conducive to accurate decoding; and inputting the encoding of the biophysical activity data into the one or more second machine learning models and outputting, from the one or more second machine learning models, the one or more sets of inputs.

35. The at least one computer-readable storage medium of claim 34, wherein the first machine learning model is a time convolutional neural network.

36. The at least one computer-readable storage medium of claim 34, wherein the one or more second machine learning models are one or more multilayer perceptrons.

37. The at least one computer-readable storage medium of claim 34, wherein the first machine learning model is trained to map the biophysical activity data onto a brain machine interface manifold.

38. The at least one computer-readable storage medium of any one of claims 31-33, wherein the one or more sets of inputs comprise an activation for the one or more CANs at one or more respective lattice positions.

39. The at least one computer-readable storage medium of any one of claims 31-33, wherein the one or more sets of inputs comprise a first set of inputs and a second set of inputs, the one or more CANs comprise a first CAN and a second CAN, the output comprisesa first output and a second output, the one or more degrees of freedom comprise a first degree of freedom and a second degree of freedom, and the method further comprising: converting, using the first CAN, the first set of inputs into the first output, the first output representative of controlling the first degree of freedom of the device; and converting, using the second CAN, the second set of inputs into the second output, the second output representative of controlling the second degree of freedom of the device.

40. The at least one computer-readable storage medium of claim 39, the method further comprising: aggregating the first output and the second output into a third output, and wherein causing control of the one or more degrees of freedom of the device is based on the third output.

41. The at least one computer-readable storage medium of any one of claims 31-33, wherein the one or more CANs comprise a first CAN and a second CAN, the first CAN is configured to map a first one of the one or more sets of inputs onto a first attractor manifold, and the second CAN is configured to map a second one of the one or more sets of inputs onto a second attractor manifold.

42. The at least one computer-readable storage medium of claim 41, wherein at least one of the first attractor manifold or the second attractor manifold is a line attractor manifold.

43. The at least one computer-readable storage medium of claim 41, wherein at least one of the first attractor manifold or the second attractor manifold is a ring attractor manifold.

44. The at least one computer-readable storage medium of claim 41, wherein the first attractor manifold is a line attractor manifold and the second attractor manifold is a ring attractor manifold.

45. The at least one computer-readable storage medium of any one of claims 31-33, wherein the device is a cursor on a computer screen, a prosthetic device, a robotic arm, a speaker, or a wheelchair.

46. A system for biophysical signal decoding comprising: at least one hardware processor; and at least one computer-readable storage medium storing processor-executable instructions that, when executed by the at least one hardware processor, cause the at least one hardware processor to perform a method comprising: inputting biophysical activity data into an input module comprising one or more machine learning models and outputting, from the input module, one or more sets of inputs to change a state of one or more continuous attractor networks (CANs); inputting the one or more sets of inputs into a representation module comprising the one or more CANs and outputting, from the representation module, output representative of the change to the state of the one or more CANs; and causing control of one or more degrees of freedom of a device based on the output.

47. The system of claim 46, wherein the biophysical activity data comprises neural activity data comprising at least one of spiking activity data or non-spiking extracellular activity data, and further comprising obtaining the at least one of the spiking activity data or the non- spiking extracellular activity data from at least one sensor.

48. The system of claim 46, wherein the biophysical activity data comprises non-neural activity data comprising data from at least one of an accelerometer, an electromyograph, or one or more electrodes and further comprising obtaining the data from the at least one of the accelerometer, the electromyograph, or the one or more electrodes.

49. The system of any one of claims 46-48, wherein the one or more machine learning models comprise a first machine learning model and one or more second machine learning models, and inputting the biophysical activity data into the input module comprises: inputting the biophysical activity data into the first machine learning model and outputting, from the first machine learning model, an encoding of the biophysical activity data that is conducive to accurate decoding; and inputting the encoding of the biophysical activity data into the one or more second machine learning models and outputting, from the one or more second machine learning models, the one or more sets of inputs.

50. The system of claim 49, wherein the first machine learning model is a time convolutional neural network.

51. The system of claim 49, wherein the one or more second machine learning models are one or more multilayer perceptrons.

52. The system of claim 49, wherein the first machine learning model is trained to map the biophysical activity data onto a brain machine interface manifold.

53. The system of any one of claims 46-48, wherein the one or more sets of inputs comprise an activation for the one or more CANs at one or more respective lattice positions.

54. The system of any one of claims 46-48, wherein the one or more sets of inputs comprise a first set of inputs and a second set of inputs, the one or more CANs comprise a first CAN and a second CAN, the output comprises a first output and a second output, the one or more degrees of freedom comprise a first degree of freedom and a second degree of freedom, and the method further comprising: converting, using the first CAN, the first set of inputs into the first output, the first output representative of controlling the first degree of freedom of the device; and converting, using the second CAN, the second set of inputs into the second output, the second output representative of controlling the second degree of freedom of the device.

55. The system of claim 54, the method further comprising: aggregating the first output and the second output into a third output, and wherein causing control of the one or more degrees of freedom of the device is based on the third output.

56. The system of any one of claims 46-48, wherein the one or more CANs comprise a first CAN and a second CAN, the first CAN is configured to map a first one of the one or more sets of inputs onto a first attractor manifold, and the second CAN is configured to map a second one of the one or more sets of inputs onto a second attractor manifold.

57. The system of claim 56, wherein at least one of the first attractor manifold or the second attractor manifold is a line attractor manifold.

58. The system of claim 56, wherein at least one of the first attractor manifold or the second attractor manifold is a ring attractor manifold.

59. The system of claim 56, wherein the first attractor manifold is a line attractor manifold and the second attractor manifold is a ring attractor manifold.

60. The system of any one of claims 46-48, wherein the device is a cursor on a computer screen, a prosthetic device, a robotic arm, a speaker, or a wheelchair.

Images