Closed-loop brain stimulation framework to modulate neural population activity

Abstract

Disclosed herein are two embodiments of a closed-loop stimulation framework capable of driving neural population activity toward specified states by optimizing over a large stimulation parameter space.

Description

Attorney Docket: 8350.2025-009WO Closed-Loop Brain Stimulation Framework to Modulate Neural Population Activity Related Applications

[0001] This application claims the benefit of U.S. Provisional Patent Application Nos. 63 / 697,896, filed September 23, 2024 and 63 / 795,999, filed April 28, 2025, the contents of which are incorporated herein in their entireties. Government Interest

[0002] This invention was made with U.S. Government support under contract MH118929, awarded by the National Institutes of Health (NIH). The U.S. Government has certain rights in the invention. Background

[0003] Brain stimulation techniques are crucial tools for treating brain diseases and for causally probing neural activity to understand brain function. Because most complex brain functions are realized through the coordinated activity of populations of neurons, brain stimulation techniques must control neural population activity to manipulate brain function. Different combinations of stimulation parameters make it possible to produce diverse neural population activity patterns. Causally perturbing this activity has the potential to modulate such functions, which could lead to major scientific and medical breakthroughs. 2Attorney Docket: 8350.2025-009WO

[0004] The state of the brain affects how neural populations respond to incoming sensory stimuli. Thus, taking into account pre-stimulation neural population activity may be crucial to achieve a desired causal manipulation using stimulation.

[0005] Prior art efforts have developed online closed-loop stimulation methods to efficiently search over the stimulation parameter space. These methods adaptively learn the optimal parameter set to drive a neural population activity toward a desired state by iteratively testing different stimulation parameter configurations over the course of a session. While these methods work with a small parameter search space, they do not translate well into a larger search space because they rely on limited observations acquired during the online optimization procedure.

[0006] Further, it is challenging to identify the optimal parameter set to achieve a desired activity manipulation of an entire neural population, given the large number of possible stimulation parameter combinations. The large stimulation parameter space cannot be explored within one session, requiring multiple sessions. It is challenging to merge neural activity across sessions to learn the mapping between stimulation patterns and brain responses. This is because recording instabilities and real changes in neural circuits make it unclear how to correspond neurons across days, or to account for changes in response patterns. To merge neural activity across sessions, recent studies leveraged the finding that the coordinated activity of populations of neurons lives within a constrained low-dimensional (low-d) latent manifold in neural activity space over time, even when experiencing recording instabilities, making it possible to align neural activity across sessions. The latent representations of neural population activity 3Attorney Docket: 8350.2025-009WO within this low-d space are known to be stable over time even when experiencing recording instabilities, making it possible to align neural activity across sessions. Summary of the Invention

[0007] In a first embodiment of the invention, using such alignment methods, a prediction model is trained that maps stimulation parameters to stimulation-induced neural responses recorded across sessions. Disclosed herein is a closed-loop stimulation framework to drive neural population activity toward specified states by optimizing over a large stimulation parameter space (referred to herein as MicroStimulation Optimization (MiSO)). The framework of the first embodiment leverages two key innovations: 1) performing latent space alignment to create a large training data set by statistically merging stimulus-response samples across sessions; and 2) using a convolutional neural network (CNN) to predict the effects of stimulation for unseen stimulation parameters. The predictions of the CNN are used to guide closed-loop optimization by suggesting stimulation parameter configurations that are likely to generate the specified neural activity state.

[0008] In a second embodiment of the invention, disclosed herein is a brain stimulation framework involving two methodological advancements: 1) using the neural population activity state prior to the stimulation with a statistical model to choose the stimulation parameters; and 2) adaptively updating the model during a stimulation experiment to improve its prediction (referred to herein as Online MicroStimulation Optimization (OMiSO)). Specifically, the framework of the second embodiment first fits stimulus- response models to predict brain responses to different stimulation configurations 4Attorney Docket: 8350.2025-009WO given brain states prior to the stimulation. It then inverts the trained stimulus-response models to output an optimal stimulation configuration for creating a target brain state during a brain stimulation experiment. The framework updates the inverse model online during a stimulation experiment to adapt to changes in neural population activity within and across sessions.

[0009] The embodiments disclosed herein may guide development of neural perturbations and / or modulation. The disclosed embodiments may enable the design of custom ways to modulate the nervous system while searching for the optimal parameters for that modulation that achieves a desired result. Most complex brain functions are realized through the coordinated activity of populations of neurons, and the disclosed embodiments may be used to find the stimulation parameters to achieve the precise manipulation of neural population activity, which may lead to the manipulation of complex brain functions. The disclosed embodiments may be adapted to for use in medical applications, including, but not limited to, treating various brain diseases by optimizing brain stimulation. 5Attorney Docket: 8350.2025-009WO Brief Description of the Drawings

[0010] FIG.1 is an illustration in the first embodiment of identifying optimal stimulation parameters to produce a specified neural population activity.

[0011] FIG.2 is an illustration in the first embodiment of the extraction of a low-d subspace of the high-d population activity where the target population activity is specified.

[0012] FIG.3 is a schematic depiction of the framework of the first embodiment.

[0013] FIG.4 is an illustration, in the second embodiment, of identifying the best stimulation parameters to create a target neural population activity state using neural population activity state prior to the stimulation early and late in each session respectively.

[0014] FIG.5 is a schematic depiction of the framework of the second embodiment.

[0015] FIG.6 is an illustration of latent activities in the pre-stimulation and post-stimulation periods in the second embodiment.

[0016] FIG.7 is a schematic depiction of the behavior cloning approach used to invert the stimulation-response model.

[0017] FIGS.8(A-D) show various exemplary model architectures for use with OMiSO. Detailed Description

[0018] MiSO - MicroStimulation Optimization Framework

[0019] The goal of MiSO is to identify the optimal stimulation parameters to produce one or more specified neural population activity states, as shown in FIG.1. In one embodiment, the neural activity consists of spiking responses recorded with a 2D grid of electrodes implanted in the brain. As shown in FIG.2, in one embodiment, spiking 6Attorney Docket: 8350.2025-009WO activity is recorded from a multi-electrode array or other modality of recording, implanted in pre-frontal cortex. In other embodiments, other areas of the brain may be targeted. During fixation, microsimulation (uStim) is applied (in one embodiment, for 150ms) to induce a specified neural population activity state in a post-uStim period (MiSO can be readily applied to other stimulation protocols with a spatial component as well, for example, holographic optogenetics). Although one embodiment describes use in pre-frontal cortex, the methods disclosed herein are not limited to this brain region and may be applied to neural recordings and stimulation in any brain area.

[0020] The uStim effect is evaluated within a low-dimensional (low-d) latent subspace (e.g., 2D) identified from high-dimensional (high-d) multi-electrode spiking activity. MiSO first extracts the low-d subspace of the high-d population activity where the target population activity is specified. MiSO then collects stimulation-response samples across multiple sessions by using latent space alignment.

[0021] The samples are used to fit a CNN to predict the brain response to all potential stimulation patterns within a set of parameters defined by the user. The CNN predictions are used to guide the closed-loop optimization search. At the beginning of each closed-loop session, MiSO identifies the latent space for the session and aligns it to the common latent space from past sessions, which are used to generate the CNN predictions. Although one embodiment uses a CNN, the invention is not limited to this model class; other machine learning models may similarly be employed to predict brain responses and guide closed-loop optimization. 7Attorney Docket: 8350.2025-009WO

[0022] The closed-loop optimization is run with an epsilon greedy algorithm, as shown in the schematic representation of the closed-loop framework shown in FIG.3. At 302 the predicted optimal stimulation parameters are chosen. uStim is then performed at 304 using those parameters. On each trial, MiSO tests the predicted parameters at 304, analyzes the results at 306 and updates the predictions based on the responses measured online at 308. MiSO iteratively runs this optimization over a plurality of trials to produce the specified latent activity. The details of each step of the framework are explained in more detail below.

[0023] In one embodiment, MiSO is implemented using a 96 electrode Utah array implanted in the brain, for example, in the pre-frontal cortex area. As would be realized, many different numbers and configurations of electrodes are intended to be within the scope of the invention.

[0024] Latent Space Identification Using Factor Analysis

[0025] In each session, MiSO uses factor analysis (FA) to identify a low-d latent space spanned by the neural population activity covariance. Each session consists of two types of trials, “stimulation trials”, in which stimulation is applied, and “non-stimulation trials”, in which no stimulation is applied. Importantly, to capture the latent space representing the natural activity covariance, only non-stimulation trials are used. For the ^^௧^session, MiSO extracts a list of usable electrodes ^^^, which record enough good spiking signals to be used in FA.

[0026] Response vectors are then computed containing the binned spike counts sampled from each electrode ^^^ ^,^^ௌ௧^^ ∈ ^^^^, where ^^ ൌ 1, … ,^^^^^ௌ௧^^and ^^^^^ௌ௧^^is the total 8Attorney Docket: 8350.2025-009WO number of spike count vectors, and ^^^is the number of usable electrodes. MiSO fits the following FA model using the Expectation Maximization (EM) algorithm: ^^^ ^,^^ௌ௧^^ ^^^0, ^^^^^^ ^,^^ௌ௧^^ ∨ ^^^^,^^ௌ௧^^^^൫^^^^^^ ^,^^ௌ௧^^ ^ ^^^ , ^^^൯(1)

[0027] where ^^^ ^,^^ௌ௧^^ ∈ ^^^,spike count vector, ^^^ ∈ ^^^^ൈ^ is the loading matrix whose columns define the low-dlatent space, ^^ ∈contains the mean spike counts for each electrode, and ^^^^^^^ൈ^^is amatrix capturing the independent variance of the spikefor each electrode.

[0028] In one embodiment, to identify the latent dimensions using FA, binned (50 ms, 24 bins) spike counts were computed in a 1.2 s period of fixation for each non-stimulation trial. Thus, ^^^^^ௌ௧^^for the ^^௧^session was the number of non-stimulation trials x 24. These non-stimulation trials were also used to extract a list of usable electrodes ^^^for the ^^௧^session based on three criteria: a firing rate >1 Hz, a Fano factor <8, and <20% coincident spikes indicating no signal corruption from other electrodes. The latent dimensionality for each session is a hyperparameter chosen based on the cross- validated data likelihood (m = 4). In other embodiments, different parameters may be used.

[0029] Latent Space Alignment

[0030] To align the neural activity across sessions, MiSO uses a method that solves the Procrustes problem to find the orthogonal transformation matrix to maximize the 9Attorney Docket: 8350.2025-009WO alignment between two FA loading matrices. First, MiSO identifies a reference latent subspace spanning the natural covariance of the data ^^^ ∈ ^^^బൈ^. MiSO aligns thedata from all sessions to this latent subspace. ^^^is the number of usable electrodes in ^^^. This is computed from a single session run exclusively with non-stimulation trials. MiSO aligns the latent space identified in the ^^௧^session ^^^to the reference latent space ^^^. Specifically, for the ^^௧^session, MiSO identifies an orthogonal transformation matrix ^^^ ∈ ^^^ൈ^ that fulfills:^^^ ൌ ^ை^^:^ை^^ை^⊺^ୀ^^ூ^^ ‖^^^^^^^^^, : ^ െ ^^^^^^^^^, : ^^^⊺‖ଶி(2)

[0031] where ^^^^^^ ൌ ^^^and the ^^௧^session, and‖∙‖ிis the Frobenius Norm. This optimization can be solved in closed-form. The ^^^found is applied to ^^^to obtain the aligned latent space ^^^∈ ^^^^ൈ^:^^^ ൌ ^^^^^^⊺(3)

[0032] MiSO defines a one-time bin after the stimulation period as the targetthe neural population activity in each trial, as shown in FIG.2. The latent activity at this bin is estimated using the posterior mean from the FA model defined in Eq. (1): ^^^ௌ,^௧^^ ൌ ^^^൫^^^ௌ,^௧^^ െ ^^^൯(4)are 10Attorney Docket: 8350.2025-009WO

[0034] Stimulation-Response Sample Collection and CNN Fitting

[0035] The stimulation-response data collection is performed in two phases. In the first phase, MiSO applies randomly selected stimulation patterns throughout the course of ^^ sessions. The random selection process allows MiSO to test a wide variety of stimulation patterns within a limited number of trials. Each stimulation pattern specifies the parameters of the stimulation, given by the location of the electrodes used for stimulation within the multi-electrode grid, and one or more additional stimulation properties that can be changed for each electrode, for example, stimulation amplitude, frequency, duration and waveforms. The stimulation patterns are parameterized with a ℎ ൈ ^^ grid with non-zero values for stimulated electrodes and 0for other electrodes ^^^,^ ∈ ^^^ൈ௩, where ^^ ൌ 1, … , ^^^ௌ௧^^, and ^^^ௌ௧^^is the number of stimulation trials in the ^^௧^session. Latent activity estimates ^^^ௌ,^௧^^are then computed in response to the different stimulations in each trial, aligned to the reference session using Eq. (4).

[0036] Next, the user defines target dimensions within a latent subspace where the activity will be controlled. The entries of the latent activity vectors ^^^ௌ,^௧^^corresponding to the target dimensions are subselected and collected in new vectors ^̌^^ௌ,^௧^^ ∈ ^^௧, where ^^ ^^^ is the number of target latent dimensions. To create the training data for the CNN, all observed stimulation patterns ^^^,^and latent responses ^̌^^ௌ,^௧^^across ^^ sessions, where ^^ ൌ 1, … , ^^ are appended in ^^^ ∈ ^^^ൈ௩ and ^̌^^ௌ௧^^ ∈ ^^௧, and where ^^ ൌ1, … , ^^ௌ௧^^ and ^^ௌ௧^^ is the total number of stimulation trials across sessions.11Attorney Docket: 8350.2025-009WO

[0037] The totality of the stimulation-response samples ^^^and ^̌^^ௌ௧^^are used to train the CNN model, which maps the stimulation patterns ^^^to the induced brain responses ^̌^^ௌ௧^^. Because the CNN predictions are noisy due to the limited number of stimulation- response samples, MiSO uses bagging to train the CNN. MiSO fits ^^ CNN models with bootstrapping and obtains the average prediction of the top ^^ performing models in testing. MiSO generates the average predicted stimulation responses ^^^ ∈ ^^௧ for allpotential patterns within the set of parameters defined by the user, where^^ ൌ 1, … ,^^, and ^^ is the total number of potential stimulation patterns.

[0038] In the second phase, MiSO collects stimulation-response samples by using a guided sample collection strategy to increase the diversity of the stimulation responses observed within limited trials. MiSO uses the predicted stimulation responses ^^^obtained from the first phase to extract stimulation patterns expected to produce unique neural responses. MiSO then computes the distance measure: ^^^ൌ 1 ^^^ ฮ^^^ െଶ (5)

[0039] where ^^^is the set of K-^^^. MiSO then chooses the top ^^^stimulation patterns maximizing this criterion. MiSO also adds ^^^randomly chosen stimulation patterns to sample from the entire response distribution. Selected ^^^ ^ ^^^ patterns are then tested, and newly collectedstimulation-response samples are appended to ^^^and ^̌^^ௌ௧^^. These are used to train another CNN to update the stimulation patterns expected to produce unique responses 12Attorney Docket: 8350.2025-009WO (tested in the next session). MiSO repeats this process for ^^ sessions. The CNN predictions ^^^from the last iteration are used during initialization of the closed-loop optimization.

[0040] Latent Space Calibration and Closed-Loop Optimization

[0041] Each online closed-loop session starts with ^^^^^ௌ௧^^calibration trials, where no stimulation is applied. MiSO extracts a list of usable electrodes ^^^using these trials and finds the common electrodes ^^^^^ ൌ ^^^ ∩ ^^^ between this session and the referencesession. The observed spike count vectors ^^^ ^,^^ௌ௧^^ ∈ ^^^^ (where ^^ ൌ 1, … , ^^^^ௌ௧^^ and^^^is the number of usable electrodes in ^^^) are used to fit the FA parameters ^^^∈ ^^^^ൈ^, ^^^ ∈ ^^^^, and ^^^ ∈ ^^^^ൈ^^. The identified latent space ^^^ is aligned to thediscussed latent space alignment method, yielding ^~^^ ∈ ^^^^ൈ^.

[0042] The closed-loop stimulation optimization starts by the user defining a target latent state ^̌^௧^^^ ∈ ^^௧ and loading the predicted stimulation responses ^^^ obtained from thepreviously discussed stimulation-response sample collection and CNN fitting method. The goal of MiSO is to find the ^^௧^stimulation pattern which achieves ^̌^௧^^^by adaptively updating ^^^. MiSO performs the closed-loop optimization procedure by choosing the optimal stimulation patterns trial by trial, rather than updating them within a trial. During the closed-loop optimization, the selected ^^௧^stimulation pattern ^^^is applied and the induced response during the post-stimulation period ^^^ௌ,௧^^^ ∈ ^^^^where ^^ ൌ 1, … ,^^^ௌ௧^^ is measured. MiSO estimates the latent response ^^^ௌ,௧^^^ ∈ ^^^ inreal time:13Attorney Docket: 8350.2025-009WO ^^^ௌ,௧^^^ ൌ ^^^൫^^^ௌ,௧^^^ െ ^^^൯(6) where ^^^ ൌ ^~^⊺^ ^^~^^^~^⊺^ ^prediction of theeffect is updated as follows: ^^^ ൌ ^^^ ^ ^^^^(7)

[0043] where 0 ^ ^^ ^ 1 is the learning rate, and ^^ ൌ ^^^ െ ^̌^^ௌ,௧^^^is the prediction error based on the current observation. To implement ascheduling, MiSO uses a clipped learning rate: ^^ ൌ ^^^^^^ ൬^^^^^^,1 ^^^^^^^ (8)

[0044] where ^^^^^^is the number ofcurrent closed loop session. The clipped learning rate allows MiSO to make larger updates at the beginning of the session, when the framework tries to adapt to the online data, and smaller updates as the framework becomes more confident about the prediction.

[0045] After updating the prediction, MiSO chooses the pattern to test at the next trial using an epsilon greedy algorithm. This balances exploitation and exploration during the optimization to maximize the chance of finding the optimal stimulation pattern. With a probability of 1 െ ^^, MiSO chooses the ^^௧^ pattern predicted to minimize the error forthe target latent state 14Attorney Docket: 8350.2025-009WO ^^ ൌ ^^^^^^^^^^^^^ ฮ^^^ െ ^̌^௧^^^ฮ^(9) and applies stimulationchooses a random stimulation pattern from ^^^. MiSO then iterates until the session is terminated.

[0046] Model Architecture

[0047] In preferred embodiments, a convolutional neural network is used as the model because it captures spatial smoothness by optimizing multiple 2D convolutional filters that learn the structure of the stimulation effect over the array. However, as would be realized, other machine learning models to predict brain responses may be substituted for the CNN.

[0048] In one embodiment, the CNN architecture consists of (1) a convolution layer with 32 channels, spatial kernel size 3x3, and stride size 1, followed by ReLU activation; (2) a convolution layer with 64 channels, spatial kernel size 3x3, and stride size 1, followed by a ReLU activation (3) two linear layers with ReLU activation and (4) a final output linear layer of size 2 without any activation. The MLP architecture had: (1) an input layer of size 96; (2) A hidden layer of size 10, followed by a ReLU activation; and (3) an output layer of size 2. The CNN and MLP can be implemented in PyTorch and fitted using the Adam optimizer with a mean squared error loss and a learning rate of 0.001. The GP model can be fitted using the GPyTorch library. A Radial Basis Function kernel with length scale was used. The length scale hyperparameter was chosen by maximizing the data marginal log likelihood with Adam optimizer with a learning rate of 15Attorney Docket: 8350.2025-009WO 0.001. All models can be trained on a local computer cluster using 4 NVIDIA GeForce RTX GPUs and 11GB of RAM.

[0049] OMiSO – Online MicroStimulation Optimization Framework

[0050] The goal of OMiSO is to shift neural population activity states toward a targeted state by choosing stimulation parameters based on time-varying neural activity states. To choose stimulation parameters, OMiSO predicts the best stimulation parameters to create a target neural population activity state using neural population activity state prior to the stimulation, as shown in FIG.4. To account for changes in neural population activity states and stimulus-response relationship over time, OMiSO updates the prediction of the best stimulation parameters online using new online observations, also shown in FIG.4. Specifically, OMiSOfirst identifies a reference low-d latent space of the high-d population activity, to which the neural activity from all the other sessions are aligned. OMiSO then merges collected neural activity across multiple sessions by using latent space alignment. The merged datasets are used tofit statistical models to predict the post-stimulation latent state given stimulation parameters and pre-stimulation latent state. OMiSO inverts the trained statistical models to obtain a model that outputs stimulation parameters to create a target state. The trained inverse model is then used to choose stimulation parameters in a closed-loop brain stimulation experiment.

[0051] At the beginning of each closed-loop session, OMiSO identifies the latent space for the session. OMiSO then aligns it to the reference latent space, which was used to train the models. On each trial, OMiSO analyzes a pre-stimulation latent state and chooses 16Attorney Docket: 8350.2025-009WO stimulation parameters by passing the analyzed pre-stimulation latent state and user defined target state to the inverse model. Using new online observations, OMiSO adaptively updates the inverse model to improve the optimization performance during a closed-loop experiment.

[0052] While the neural activity used in this study consists of spiking responses recorded with a planar grid of electrodes implanted in the brain and each stimulation pattern specifies the location of the electrodes used for stimulation within the grid, OMiSO can be readily applied to other stimulation / recording protocols (e.g., holographic optogenetics). Note that, herein, a specific stimulation parameter configuration is referred to as a “stimulation pattern”.

[0053] A schematic depiction of the framework of the second embodiment is shown in FIG.5. OMiSO trains stimulus-response models and stimulus-response inverse model using previously collected neural activity. The trained stimulus-response inverse model is loaded to initialize a closed-loop stimulation procedure. In each brain stimulation trial, OMiSO performs the five steps shown in FIG.5.

[0054] Latent Space Identification

[0055] To merge neural activity across multiple sessions, OMiSO uses the same methodology as the MiSO embodiment previously discussed, wherein a low-d latent space of the high-d population activity is identified in each session. The factor analysis of Eq. (1) using the EM algorithm is used to identify the intrinsic latent space only using non- stimulation trials.

[0056] Latent space alignment 17Attorney Docket: 8350.2025-009WO

[0057] OMiSO aligns identified latent spaces to merge neural activity across sessions. Like MiSO, OMiSO uses a method that solves the Procrustes problem to find the orthogonal transformation matrix to maximize the alignment between two FA loading matrices. The main difference is the OMiSO identifies a list of usable and stable electrodes. Thus Eq. (2) becomes: ^^^ ൌ ^ ⊺ை^^:^ை^^ை^⊺^ୀ^^ூ^^ ‖^^^^^^^௧^^^^, : ^ െ ^^^^^^^௧^^^^, : ^^^ ‖ଶிwhere ^^^௧^^^^is aobtain the aligned latent space ^^^using Eq. (3).

[0058] The latent activities in the pre-stimulation and post-stimulation periods, shown in FIG. 5, are estimated as the posterior mean from the FA model (Eq. (1)) using the loading matrix ^~^^:^^^,^ ൌ ^^^൫^^^,^ െ ^^^൯(11)

[0059] where ^^^,^ ∈ ^^^, ^^^,^ ∈ ^^^^ and ^^^ ൌ ^~^⊺^^^~^^^~^⊺^ ^ ^^^^ି^. By using ^~^^, the induced latentactivity ^^^,^residesspace across all

[0060] Stimulus-Response Model Fitting

[0061] Using the merged neural activity across sessions, OMiSOfits statistical models (referred to herein as stimulus-response models) to predict post-stimulation latent neural activity states. As in the first embodiment (MiSO), the stimulus-response models use a convolutional neural network (CNN) to extract the spatial features of stimulation patterns. To extract the temporal features from the pre-stimulation latent activity 18Attorney Docket: 8350.2025-009WO state, the stimulus-response models use a long short-term memory (LSTM). The extracted spatial and temporal features are concatenated to predict the post- stimulation neural activity state.

[0062] To train the stimulus-response models, OMiSO uses stimulation trials of ^^ sessions collected as part of the latent space alignment procedure, which involves stimulation with randomly chosen patterns among ^^ possible stimulation patterns defined by the user. With latent space alignment, OMiSO creates datasets including pre-stimulation and post-stimulation latent activity states, and stimulation patterns. The pre- stimulation latent activity state over ^^ time bins ^^^ ^,^^^ ∈ ^^^ൈ௧ and induced post-stimulation latent activity state ^^^ ^,^^^௧ ∈ ^^^ are computed using Eq. (11). The entries of^^^^,^^^and ^^^^,^^^௧corresponding to the user defined ^^ target dimensions within the ^^ dimensional aligned latent space ^^^ ^ ^^^ are subselected and collected in new vectors^̌^^ ^,^^^ ∈ ^^^ൈ௧ and ^̌^^ ^,^^^௧ ∈ ^^^. With a planar grid of electrodes of size ℎ ൈ ^^, thestimulation pattern tested in the ^^௧^ trial during the ^^௧^ session ^^^,^ ∈ ^^^ൈ௩ is encodedusing a value of 1 for stimulated electrodes and 0 for all other electrodes. To train the stimulus-response models ^^൫^^^ ^,^^^௧ห^^^,^, ^̌^^^,^^^൯, which aims to map the stimulation pattern ^^^,^and pre-stimulation state ^̌^^^,^^^to the post-stimulation state ^̌^^^,^^^௧, OMiSO runs ^^-folds cross validation with mean squared error where every one of the sessions is used as a validation set once. The final prediction ^^^ ^,^^^௧ ∈ ^^^ is obtained by averagingthe predictions of the ^^ models.

[0063] Inverting Stimulus-Response Models 19Attorney Docket: 8350.2025-009WO

[0064] Using the trained stimulus-response models, OMiSO estimates a stimulus-response inverse model. The goal of the stimulus-response inverse model is to identify a stimulation pattern that can reach a target neural activity state given the pre- stimulation neural activity state. More specifically, a stimulus-response inverse model ^^ఏ൫^^^,^ห^̌^௧^^^, ^̌^^^,^^^൯, with model parameters ^^, receives the latent target state ^̌^௧^^^∈^̌^^ ^,^^^ ∈ ^^^ൈ௧. Then, it returns a ^^-dimensional vector ofstimulation probabilities for each electrode ^^^,^ ∈ ^^௨ where ^^ is the total number ofcandidate electrodes to be stimulated (termed stimulation candidate electrodes), with the number being determined by the user. Each entry of the vector represents the likelihood of each stimulation candidate electrode being stimulated to create a target state.

[0065] OMiSO uses Behavior Cloning (BC) to train a 5-layered Multi-Layered Perceptron (MLP) as a stimulus-response inverse model ^^ఏ. This MLP model was chosen because it is computationally fast and could return the uStim pattern online within only 50 milliseconds after observing the pre-uStim states, as shown in FIG.6. BC is a widely used approach in reinforcement learning (RL) to learn an optimal action at different states and in particular when expert data is easily accessible. An action in this case is a selected uStim pattern to perform and a state is a combination of a pre-stimulation state and target state. Expert data in this case teaches a stimulus-response inverse model which stimulation pattern to perform to create a target state given a pre- stimulation state. To create the expert data, OMiSO uses the trained stimulus-response models. The trained stimulus-response models output the expected average post- 20Attorney Docket: 8350.2025-009WO stimulation state ^^^^^௧ ∈ ^^^ given combination of a stimulation pattern ^^ ∈ ^^^ൈ௩ andpre-stimulation state ^̌^^^^ ∈ ^^^ൈ௧. OMiSO constructs expert data by defining the targetstate as ^̌^௧^^^ ൌ ^^^^^௧. Seen in this way, the stimulus-response model provides thecombinations of stimulation patterns and pre-stimulation states that give rise to certain activity targets. This results in an expert dataset ^^ ∈ ^^̌^^^^, ^̌^௧^^^, ^^^, where ^^ ∈ ^^௨ is avector representation of ^^ only including the ^^ stimulation candidate electrodes. The value of the ^^௧^stimulation candidate electrode (^^ ൌ 1, … ,^^) in ^^ is 1 if stimulatedand 0 if not. OMiSO trains a stimulus-response inverseby minimizing the binary cross entropy over multiple epochs with respect to ^^: െ1 ே ^^^^^^^^^^^^^^^^ ^ ^1 ^^^^^^^^^^1 ^^^^where ^^ ∈ ^^௨.

[0066] FIG.7 is a schematic depiction of the behavior cloning. The trained stimulus-response model (pink circle, top) predicts the post-uStim state (pink dot, top) given the pre- uStim states (gray dots, top) and a uStim pattern (array map, top). By re-arranging this data and setting a predicted post-uStim state as a target state (pink dot, bottom), a stimulus-response inverse model is trained to output a uStim pattern given pre-uStim states and target state (bottom).

[0067] To train the stimulus-response inverse model with BC, OMiSO iterates the expert data generation and model parameter update over multiple epochs. At each epoch, OMiSO 21Attorney Docket: 8350.2025-009WO generates ^^ expert samples, each of which consist of a pre-stimulation state ^̌^^^^∈ ^^^ൈ௧, a stimulation pattern ^^ ∈ ^^௨ and a predicted post-stimulation state used as atarget state ^^௧^^^. One of the observed pre-stimulation states ^̌^^^,^^^as ^̌^^^^selected and combined with a randomly chosen stimulation pattern among ^^ possible stimulation patterns to obtain the predicted target post-stimulation state ^^௧^^^. OMiSO performs a gradient descent parameter with Eq. (12) to update the model parameter ^^ at each epoch. To avoid overfitting, OMiSO evaluates Eq. (12) with 1,000 validation samples based on the user defined frequency. OMiSO stops the model training when Eq. (12) does not improve for 20 consecutive performance evaluations with the validation dataset.

[0068] Latent Space Calibration and Closed-Loop Optimization Initialization

[0069] To run closed-loop stimulation optimization using the stimulus-response inverse model, each online closed-loop session starts with ^^^^^ௌ௧^^calibration trials, where no stimulation is applied. OMiSO extracts a list of usable electrodes ^^^, which includes ^^^electrodes, using these trials. The observed spike count vectors ^^^ ^,^^ௌ௧^^ ∈ ^^^^ are usedto fit the FA parameters ^^^ ∈ ^^^^ൈ^, ^^^ ∈ ^^^^, and ^^^ ∈ ^^^^ൈ^^ . The identified latentspace ^~^^is aligned to the reference latent spacethe latent space alignment methods previously described, yielding ^~^^ ∈ ^^^^ൈ^. To start a closed-loopoptimization, OMiSO loads the user defined target state ^̌^௧^^^ ∈ ^^^, the number ofelectrodes used in stimulation ^^^ ∈ ^^^^^^, and the trained stimulus-response inversemodel ^^ఏ.

[0070] Stimulation Pattern Selection in a Closed-Loop Optimization 22Attorney Docket: 8350.2025-009WO

[0071] By using the trained stimulus-response inverse model ^^ఏand observed pre-stimulation neural activity states at 502 on each trial, OMiSO selects the stimulation pattern at 504 to perform. uStim is then performed at 506 using the selected parameters. On the ^^௧^trial during the closed-loop optimization, OMiSO estimates the pre-stimulation latent activity state ^^^ ^,^^^ ∈ ^^^ൈ௧in real time using Eq. (11). The entries of ^^^^,^^^corresponding to the target dimensions are subselected as ^̌^^ ^,^^^ ∈ ^^^ൈ௧ and ^̌^^^,^^̌^௧^^^are passed to a stimulus-response inverse model to get a ^^-dimensional vector of stimulation probabilities for each electrode ^^^,^. Using ^^^,^, OMiSO chooses a stimulation pattern with an epsilon greedy algorithm. With probability 1 െ ^^^,^ where (0 ^ ^^^,^ ^ 1), itchooses ^^^electrodes with the highest predicted probabilities in ^^^,^. With probability ^^^,^, it chooses ^^^electrodes stochastically following the softmax transformed probability ^^^௪,^ ∈ ^^ computed for each ^^௧^ electrode (^^ ൌ 1, … , ^^):^^௪^^^^^^൫^^^௪,^൯ where ^^௪ ∈ ^^ is theuses a time-varying ^^^,^to initially encourage exploration and gradually shift to exploitation: ^^^,^ ൌ ^^^^^^൫^^^^^௧ , ^^^ , ^^^^^^^൯(14) 23Attorney Docket: 8350.2025-009WO where 0 ^ ^^^^^௧ ^ 1 is an initial value of ^^^,^, 0 ^ ^^ ^ 1 is a discount factor, 0 ^^^^^^^^ ^ 1 is a floor value of ^^^,^൫^^^^^^^ ^ ^^^^^௧൯, and ^^ is the current closed-loopoptimization trial index.

[0072] OMiSO uses ^^^^^^^to maintain some amount of exploration throughout a session. Once the stimulation pattern is selected with the epsilon greedy algorithm, OMiSO performs stimulation with the selected ^^^electrodes and measures post-stimulation spike counts ^^^ ^,^^^௧ ∈ ^^^^ to compute ^̌^^ ^,^^^௧ ∈ ^^^ at 508 (Eq. (11)). The set of ^̌^^^,^^^, ^̌^௧^^^, ^̌^^ ^,^^^௧ and the tested stimulation pattern ^^^,^ ∈ ^^௨ are stored to update the stimulus-response inverse model in a batched manner, as explained in the next section.

[0073] Online Updating of the Stimulus-Response Inverse Model

[0074] During the closed-loop experiment, OMiSO updates the stimulus-response inverse model ^^ఏusing the stored online observations at 510. OMiSO’s online model update was implemented using a clipped policy gradient objective function used in Proximal Policy Optimization (PPO) methods. At each update, OMiSO increases the probability of the stimulated electrodes that produced a post-uStim state closer to the target state and decreases it for electrodes that did not. Specifically, for ^^௧^^^^ ൌ 1, … , ^^^^electrodes among ^^^electrodes used for the stimulation pattern ^^^,^, OMiSO updates the parameters ^^ in the stimulus-response inverse model ^^ఏby performing gradient ascent over multiple epochs on the following objective function: ^ೞ^^^ ^^^^^^ ^ ^^^^ ^^ ^^ ൯Attorney Docket: 8350.2025-009WO ^ ^^^^^^,^ ^^^^ ൌ^^ఏ൫^^^,^ ห^̌^௧^^^, ^̌^^,^^൯ ^^̌^^^^^൯ where 0 ^ ^^^^^^ ^ 1 is anto avoid a too large updatestep of the model. ^^ఏ൫^^^ ^,^ ห^̌^௧^^^, ^̌^^^,^^^൯ represents the predicted stimulation probability for the ^^௧^stimulated electrode under the updated stimulus-response inverse model. Similarly, ^^ఏ^^^൫^^^ ^,^ ห^̌^௧^^^, ^̌^^^,^^^൯ is the predicted probability under the original stimulus-before updating ^^. The ratio ^^^^,^^^^^is the change inprobability for selecting the ^^௧^ electrode. ^^^,^ ∈ ^^ is the advantage for the stimulationpattern used in the ^^௧^ stored trial where ^^ ∈ ^^ is a fixed baseline value to control adesired performance by OMiSO. The advantage ^^^,^reflects the effectiveness of the performed stimulation pattern at the ^^௧^trial ^^^,^. If the pattern achieves an absolute error smaller than the baseline, the advantage is positive, otherwise, it becomes negative. The PPO update increases the selection probability of the stimulation electrodes with positive advantage while decreasing the probability of the stimulation electrodes with negative advantage. OMiSO performs gradient ascent on ^^^^ைin a batched manner after every 5 trials as determined by the user.

[0075] Model Architecture

[0076] FIGS.8(A-D) show various exemplary model architectures for use with OMiSO. FIG.8A is a CNN architecture (uStim pattern only). FIG.8B is a stimulus-response model architecture with MLP (uStim pattern + Pre-stim{1,3,5} (MLP)). FIG.8C is a stimulus- 25Attorney Docket: 8350.2025-009WO response model architecture with LSTM (uStim pattern + Pre-stim5 (LSTM)). FIG.8D is a stimulus-response inverse model architecture.

[0077] Disclosed herein as a first embodiment of the invention is MiSO, which enables optimization of stimulation parameters over a large parameter search space guided by a CNN model trained on merged neural activity across sessions. While the MiSO framework is disclosed with a specific stimulation modality (uStim with a Utah array) and prediction model (CNN), the closed-loop framework is generalizable across stimulation modalities and modeling approaches. The framework provides a significant advancement in neuromodulation technologies to treat brain diseases and better understand brain functions.

[0078] Disclosed herein as a second embodiment of the invention is OMiSO, a brain stimulation framework which involves two major methodological advancements: incorporation of pre-stimulation brain state and online adaptive model update to choose optimal stimulation parameters for shaping neural population activity states. OMiSO achieves a significant performance improvement compared to the methods that do not incorporate these advancements. OMiSO provides a novel framework to integrate neural population state changing over time into a closed-loop optimization procedure.

[0079] The framework may be implemented as software executing in a general-purpose computer having an interface to the stimulation modality. The framework is meant to be modality agnostic and could be used with any modality capable of delivering the required stimulation pattern and observe the required neural activity states (e.g., EEG). 26Attorney Docket: 8350.2025-009WO

[0080] Many possible features of and variations of the methods have been disclosed herein. As would be realized, many of these features or variations may be used in any combination or are described in a generalized manner such as to encompass specific embodiments. All possible combinations of features disclosed herein are contemplated to be within the scope of the invention. 27

Claims

Attorney Docket: 8350.2025-009WO Claims 1. A method comprising: creating a dataset comprising a plurality of data points, each data point comprising a pre-stimulation latent activity state, a post-stimulation latent activity state and a stimulation pattern; training a stimulus-response model using the training dataset, stimulus- response model predicting a post-stimulation latent activity state, given a pre- stimulation latent activity state and a stimulation pattern; and creating a stimulus-response inverse model from the stimulus-response model, the stimulus-response inverse model predicting a stimulation pattern given a pre- stimulation latent activity state and a target post-stimulation latent activity state. 2 The method of claim 1 wherein the latent space identified from neural activity is aligned with a reference latent space. 3 The method of claim 1 wherein the stimulation patterns in the training dataset are chosen from a set of user-defined stimulation patterns. 4 The method of claim 1 wherein the stimulation-response model is a neural network. 28Attorney Docket: 8350.2025-009WO 5. The method of claim 1 wherein the stimulation-response inverse model is a multi-layer perceptron.

6. The method of claim 1 wherein the stimulation-response inverse model is created using behavior cloning. 7 The method of claim 1 wherein the pre-stimulation latent activity state and the post- stimulation latent activity state are collected using a multi-electrode array. 8 A method comprising: iterating, over a plurality of sessions: observing a pre-stimulation latent activity state; predicting, using a stimulus-response inverse model, a stimulation pattern that will produce a desired target post-stimulation latent activity state, given the observed pre-stimulation latent activity state; performing stimulation using the predicted stimulation pattern; observing a post-stimulation latent activity state; determining an error between the observed post-stimulation latent activity state and the target post-stimulation latent activity state; and updating the stimulus-response inverse model using the error. 29Attorney Docket: 8350.2025-009WO 9. The method of claim 8 wherein samples from the plurality of iterations are accumulated before the stimulus-response inverse model is updated using the accumulated samples.

10. The method of claim 8 wherein the stimulus-response inverse model predicts the stimulation pattern given the pre-stimulation latent activity state and the target post- stimulation latent activity state.

11. The method of claim 10 wherein the stimulus-response inverse model is derived from a stimulus-response model.

12. The method of claim 11 wherein the latent space identified from neural activity is aligned with a reference latent space and used to train the stimulus-response model from which the stimulus-response inverse model is derived.

13. The method of claim 8 wherein, for a plurality of initial iterations in each session, the method comprises: performing a plurality of non-stimulation iterations; determining a common list of usable electrodes based on the non-stimulation iterations between a reference session and the current session; and identifying a low-dimensional latent space using the common electrodes and aligning the low-dimensional latent space with a reference latent space. 30Attorney Docket: 8350.2025-009WO 14. The method of claim 13 wherein the usable electrodes are determined based on firing rate, Fano factor and signal corruption from other electrodes.

15. The method of claim 13 wherein the low-dimensional latent space is determined using factor analysis using an expectation-maximization algorithm, given a high-dimensional neural activity space.

16. The method of claim 13 wherein the high-dimensional neural activity space comprises binned spike counts sampled from each electrode.

17. The method of claim 8 wherein the pre-stimulation latent activity state and post- stimulation latent activity state are observed and wherein the stimulation is delivered using a multi-electrode array.

18. The method of claim 17 wherein the pre- and post-stimulation latent activity states comprise spiking activity recorded from the multi-electrode array.

19. The method of claim 17 wherein the multi-electrode array is implanted in the pre-frontal cortex of a subject.

20. The method of claim 17 wherein the multi-electrode array is a 96 electrode Utah array. 31Attorney Docket: 8350.2025-009WO 21. The method of claim 8 wherein stimulation patterns comprise: a location of each electrode in the electrode array; and one or more additional electrode-specific stimulation parameters.

22. The method of claim 21 wherein the additional electrode-specific stimulation parameter is selected from one or more of amplitude, frequency, duration and waveform.

23. A system comprising: a processor; a stimulation / recording modality, coupled to the processor; and software, executing on the processor, implementing the framework of claim 8.

24. The system of claim 23 wherein the stimulation / recording modality comprises a multi- electrode array implanted in the brain of a subject. 32

Images