Predicting neural activity at depth from surface using multimodal experiments and machine learning models

Abstract

Methods, systems, and devices related to predicting brain neural activity at depth from surface recordings obtained using a graphene-based, transparent electrode array are disclosed. In an example embodiment, a method for measuring activity in a brain at multiple depths is disclosed. The method includes simultaneously obtaining a first dataset comprising first signals from a first depth within a brain and a second dataset comprising second signals from one or more second depths within the brain using a flexible transparent electrode array comprising graphene. The method further includes determining activity at the one or more second depths based on cross modality inferencing results between the first dataset and the second dataset that are obtained using a trained neural network, wherein the trained neural network is trained using the first dataset to learn nonlinear relationships between the first signals and the second signals.

Description

PREDICTING NEURAL ACTIVITY AT DEPTH FROM SURFACE USING MULTIMODAL EXPERIMENTS AND MACHINE LEARNING MODELS

[0001] This invention was made with government support under ECCS-2024776 awarded by the National Science Foundation and EB030992 awarded by the National Institutes of Health. The government has certain rights in the invention.CROSS-REFERENCE TO RELATED APPLICATION

[0002] This patent document claims priority to and benefits of U.S. Provisional Application No. 63 / 606,408, entitled “PREDICTING NEURAL ACTIVITY AT DEPTH FROM SURFACE USING MULTIMODAL EXPERIMENTS AND MACHINE LEARNING MODELS,” and filed on December 5, 2023. The entire content of the above noted patent application is incorporated by reference as part of the disclosure of this patent document.TECHNICAL FIELD

[0003] This patent document is generally related to techniques to predict brain neural activity.BACKGROUND

[0004] Predicting brain neural activity at depth through non-invasive means is of interest to a variety of technical fields including neuroscience, neural engineering, medicine, and others. Techniques that enable such predicting have the potential to lead to new developments in minimally invasive neural prosthetics and targeted treatments for various neurological disorders.SUMMARY

[0005] The disclosed embodiments, among other features and benefits, relate to methods, systems, and apparatus to predict brain neural activity at depth from surface potential recordings obtained using a graphene-based, transparent microelectrode array. The disclosed embodiments can be implemented in cross modality inferencing applications to enable the development of neural network computational techniques which learn nonlinear dynamics between the different modalities.BRIEF DESCRIPTION OF THE DRAWINGS

[0006] FIG. 1 shows a table of example characteristics of a transparent graphene electrode array based on the disclosed technology in comparison to existing array technologies.

[0007] FIG. 2 shows an example plot of optical transmittance as a function of normalized impedance for the array technologies listed in the table shown in FIG. 1.

[0008] FIGS. 3A-B show examples of conventional gold electrode arrays to facilitate comparison with transparent graphene electrode arrays that can be achieved in accordance with implementations of the disclosed technology.

[0009] FIG. 3C shows an example embodiment of a transparent graphene electrode array that can be achieved in accordance with implementations of the disclosed technology.

[0010] FIG. 3D shows example images of shadows created by opaque gold wires in the field of view (FOV) during two-photon imaging using conventional gold electrodes in order to facilitate comparison with transparent graphene electrode arrays that can be achieved in accordance with implementations of the disclosed technology.

[0011] FIG. 3E shows examples of signals recorded by the gold electrodes shown in FIG. 3D and by the graphene electrodes shown in FIG. 3C during two-photon imaging.

[0012] FIG. 3F shows examples of the power spectral density of signals recorded by the gold electrodes shown in FIG. 3D and by the graphene electrodes shown in FIG. 3C during an example two-photon Z-scan.

[0013] FIG. 4A shows another example embodiment of a transparent graphene electrode array that can be achieved in accordance with implementations of the disclosed technology.

[0014] FIG. 4B shows a schematic illustrating an example of HNO3 doping of interlayer doped double layer graphene (id-DLG) in accordance with embodiments of the disclosed technology.

[0015] FIG. 4C shows an image of a graphene microwire made of single layer graphene (SLG; top) in comparison to an image of an example microwire comprising id-DLG (bottom) that can be achieved in accordance with implementations of the disclosed technology.

[0016] FIG. 4D shows example wire resistances as a function of wire length for SLG and double layer graphene (DLG) wires in comparison to id-DLG wires that can be achieved in accordance with implementations of the disclosed technology.

[0017] FIG. 4E an optical image of another example embodiment of a transparent graphene electrode array that can be achieved in accordance with implementations of the disclosedtechnology.

[0018] FIGS. 5A-C show images of SLG wires with 20 gm, 30 gm, and 40 gm widths, respectively, to facilitate comparison to transparent graphene electrode arrays that can be achieved in accordance with implementations of the disclosed technology.

[0019] FIGS. 5D-F show images of DLG wires with 20 pm, 30 gm, and 40 gm widths, respectively, to facilitate comparison to transparent graphene electrode arrays that can be achieved in accordance with implementations of the disclosed technology.

[0020] FIG. 6A shows an image of a recording interface board connected to an example embodiment of a transparent graphene electrode array that can be achieved in accordance with implementations of the disclosed technology.

[0021] FIG. 6B shows an example impedance histogram of the transparent graphene electrode array shown in FIG. 6A.

[0022] FIG. 6C shows an image of an example embodiment of a transparent graphene electrode array, including various opening sizes, that can be achieved in accordance with implementations of the disclosed technology.

[0023] FIG. 6D shows another image of an example embodiment of a transparent graphene electrode array that can be achieved in accordance with implementations of the disclosed technology.

[0024] FIG. 6E shows yet another image of an example embodiment of a transparent graphene electrode array that can be achieved in accordance with implementations of the disclosed technology.

[0025] FIG. 7A shows an example plot of capacitance versus voltage for SLG, DLG, and Helmholtz electrical double layer (EDL) in comparison to id-DLG that can be implemented in example embodiments based on the disclosed technology.

[0026] FIG. 7B shows a microscope image (left) and a scanning electron microscopy image (right) of an example embodiment of an id-DLG electrode based on the disclosed technology before and after the deposition of Pt nanoparticles (PtNPs).

[0027] FIG. 7C shows an example impedance distribution of an example embodiment of a transparent graphene electrode array, before and after the deposition of PtNPs, based on the disclosed technology.

[0028] FIG. 7D shows a diagram of an example equivalent circuit model for an id-DLG electrode with and without PtNPs in accordance with implementations of the disclosed technology.

[0029] FIG. 7E shows electrochemical impedance spectroscopy (EIS) data of an example embodiment of a PtNP / id-DLG electrode based on the disclosed technology and exemplary modeled results for the PtNP / id-DLG electrode obtained from the fitted equivalent circuit model shown in FIG. 7D.

[0030] FIG. 7F shows exemplary transmission versus wavelength data for different material stacks in an example embodiment of a transparent graphene electrode array that can be achieved in accordance with implementations of the disclosed technology.

[0031] FIG. 8 shows a table of example parameters of a PtNP / id-DLG electrode and an id- DLG model for an example embodiment of an id-DLG electrode based on the disclosed technology.

[0032] FIG. 9A shows EIS data of an example embodiment of an id-DLG electrode based on the disclosed technology.

[0033] FIG. 9B shows impedance data for example PtNP / id-DLG electrodes fabricated with various PtNP deposition times in accordance with implementations of the disclosed technology.

[0034] FIG. 9C shows SEM images of example PtNP / id-DLG electrodes fabricated with various PtNP deposition times in accordance with implementations of the disclosed technology.

[0035] FIG. 9D cyclic voltammetry (CV) data before and after 150 s of PtNP deposition for example electrodes based on the disclosed technology.

[0036] FIG. 9E shows a plot of the ratio of graphene wire resistance to electrode impedance versus wire length for example embodiments of the disclosed technology.

[0037] FIG. 10A shows a schematic of an example multimodal experimental setup that can be implemented in accordance with embodiments based on the disclosed technology.

[0038] FIG. 10B shows images of exposed cortex area and the field of view of an example transparent graphene electrode array based on the disclosed technology that can be implemented in the multimodal experiment shown in FIG. 10A.

[0039] FIG. 10C shows cortical potentials recorded by an example transparent grapheneelectrode array based on the disclosed technology.

[0040] FIG. 10E shows example fluorescence activity data of neurons detected in accordance with implementations of the disclosed technology.

[0041] FIG. 10F shows a plot of the example fluorescence activity data of the neurons shown in FIG. 10E after normalization and obtained in accordance with implementations of the disclosed technology.

[0042] FIG. 10G shows an example of pixel-level average calcium activity obtained in accordance with implementations of the disclosed technology.

[0043] FIG. 11 A shows example noise levels of channels of an example transparent graphene electrode array based on the disclosed technology for different graphene wire lengths.

[0044] FIG. 1 IB shows an example heatmap of the noise levels in FIG. 11 A overlaid with a microscope image of an example transparent graphene electrode array based on the disclosed technology.

[0045] FIG. 11C shows an example image of pyramidal cells around and underneath a PtNP / id-DLG electrode based on the disclosed technology.

[0046] FIG. 1 ID shows example fluorescence change signals of the pyramidal cells shown in FIG. 11C which can be obtained in accordance with implementations of the disclosed technology.

[0047] FIG. 12A shows a schematic of cortical regions covered by an example embodiment of a transparent graphene electrode array that can be achieved in accordance with implementations of the disclosed technology.

[0048] FIG. 12B shows examples of peak-to-peak amplitude data (left) and a delay map (right) that can be obtained in accordance with implementations of the disclosed technology.

[0049] FIG. 12C shows example spatial maps of powers at different frequency bands across a transparent graphene electrode array based on the disclosed technology.

[0050] FIG. 12D shows example waveforms of brain recordings obtained using a transparent graphene electrode array based on the disclosed technology.

[0051] FIG. 12E shows example correlation data between cell-averaged calcium peaks and MUA power obtained using a transparent graphene electrode array based on the disclosedtechnology.

[0052] FIG. 12F shows an example plot of a cell-averaged calcium signal AF / F and MUA power obtained using a transparent graphene electrode array based on the disclosed technology.

[0053] FIG. 12G shows an example scatter plot of cell-averaged calcium peaks and corresponding MUA powers obtained using a transparent graphene electrode array based on the disclosed technology.

[0054] FIG. 13 A shows example auto correlograms of surface recordings obtained in accordance with implementations of the disclosed technology.

[0055] FIG. 13B shows an example plot of cell-averaged AF / F and MUA power obtained using a transparent graphene electrode array based on the disclosed technology.

[0056] FIG. 13C shows an example scatter plot of calcium peaks and MUA powers obtained using a transparent graphene electrode array based on the disclosed technology.

[0057] FIG. 13D shows an example plot of cell-averaged AF / F and MUA power obtained using a transparent graphene electrode array based on the disclosed technology.

[0058] FIG. 13E shows an example scatter plot of calcium peaks and MUA powers obtained using a transparent graphene electrode array based on the disclosed technology.

[0059] FIG. 14A shows a schematic of an example neural network model which can be used to decode calcium activity from recorded surface potentials in accordance with implementations of the disclosed technology.

[0060] FIG. 14B shows examples of decoded and ground truth activities for LI and L2 / 3 that can be obtained in accordance with implementations of the disclosed technology.

[0061] FIG. 14C shows a plot of an example decoding performance for LI and L2 / 3 that can be achieved in accordance with implementations of the disclosed technology.

[0062] FIG. 14D shows another plot of an example decoding performance for LI and L2 / 3 that can be achieved in accordance with implementations of the disclosed technology.

[0063] FIG. 15A shows a diagram of an example order of channels, in a transparent graphene electrode array, that can be used for decoding calcium signals in accordance with implementations of the disclosed technology.

[0064] FIG. 15B shows example correlation data between cell-averaged calcium peaks and surface potential powers obtained using a transparent graphene electrode array based on the disclosed technology.

[0065] FIG. 16A shows example results of singe-cell decoding of calcium signals obtained in accordance with implementations of the disclosed technology.

[0066] FIG. 16B shows a schematic diagram of an example model that can be used for singlecell decoding of calcium signals in accordance with implementations of the disclosed technology.

[0067] FIG. 16C shows exemplary decoding results obtained using a neural network model based on the disclosed technology.

[0068] FIG. 17A a schematic of an example single-cell decoding model based on the disclosed technology.

[0069] FIG. 17B shows examples of decoded and ground truth single-cell activities that can be obtained in accordance with implementations of the disclosed technology.

[0070] FIG. 17C shows an example plot of results obtained based on single-cell decoding performed in accordance with implementations of the disclosed technology.

[0071] FIG. 18A shows example single-cell decoding results that can be obtained in accordance with implementations of the disclosed technology.

[0072] FIG. 18B shows examples of reconstructed calcium signals that can be obtained in accordance with implementations of the disclosed technology.

[0073] FIG. 18C shows another example of single-cell decoding results that can be obtained in accordance with implementations of the disclosed technology.

[0074] FIGS. 18D-E show additional examples of single-cell decoding results that can be obtained in accordance with implementations of the disclosed technology.

[0075] FIGS. 19A-B show an example comparison of population coupling across cells in spontaneous and evoked sessions in accordance with implementations of the disclosed technology.

[0076] FIGS. 19C-D show example decoding results as a function of population coupling for the spontaneous (FIG. 19C) and evoked (FIG. 19D) sessions that can be obtained in accordance with implementations of the disclosed technology.

[0077] FIG. 19E-H shows example plots of population coupling calculated for different numbers of cells and single-cell decoding results that can be obtained in accordance with implementations of the disclosed technology.

[0078] FIG. 20 shows a flow diagram of an example method based on the disclosed technology.

[0079] FIG. 21 shows a flow diagram of another example method based on the disclosed technology.

[0080] FIG. 22 shows a flow diagram illustrating operations that can be performed in yet another example method based on the disclosed technology.DETAILED DESCRIPTION

[0081] The disclosed embodiments, among other features and benefits, enable non-invasive prediction of brain neural activity at depth through cross modality inference. Cross modality inferencing generally refers to the process of using information obtained from one modality (e.g., sound) to make inferences about another modality (e.g., image), thereby allowing predictions to be made across different types of data. The disclosed embodiments include high-density, optically transparent microelectrode arrays which are scalable in size and include graphene. Such arrays can be used in multimodal experiments to combine simultaneous recordings from the brain surface with optical imaging and stimulation of neural activity. The transparent graphene microelectrode arrays can be used to obtain recordings of brain activity via a first modality (e.g., electrophysiological, electroencephalographic, etc.) and additional recordings of brain activity via a second modality (e.g., two-photon imaging, fluorescence imaging, optical coherence tomography, optogenetic stimulation, magnetic resonance imaging, etc.) in a simultaneous manner. In some example embodiments, the transparent graphene microelectrode arrays are fabricated with ultra-small openings and a large transparent recording area to achieve a clear field of view (FOV) for recording brain activity. Additionally, the embodiments disclosed herein enable the development of neural network techniques which learn the nonlinear dynamics between different modalities to facilitate predictions of brain neural activity. In some implementations, the neural network techniques include training neural networks to learn a nonlinear relationship between datasets obtained at different depths within the brain. In some implementations, the nonlinear relationship is based in-part on features identified in surface recordings of the brain andtheir spatial localizations within the brain.

[0082] The disclosed embodiments can be used to extend the lifetime of neural implants and improve the longevity of brain-computer interface (BCI) technologies. Further, the disclosed embodiments can enhance the interpretation of electrophysiology studies by reducing the damage to brain tissue and the consequent alteration of neuronal activities. In some example use cases, the disclosed embodiments are implemented in multimodal applications to localize the sources of distinct features in surface recordings. In some implementations, the distinct features from the surface recordings can be used to predict a global activity within the brain. The disclosed embodiments can lead to enhancing and expanding the capabilities of existing BCI technologies in tackling complex motor and behavioral tasks. The disclosed technology enables the predicting of brain activity at layers deep under the brain surface to localize sources of distinct features found in surface recordings.

[0083] In the current state of the art, electrical and optical neural recording technologies, such as a laminar probe, have been used to understand neural dynamics. However, it is currently a challenge to scale down electrode dimensions to single-cell size and increase the density to record neural activity with high spatial resolution across large areas in order to capture nonlinear neural dynamics at multiple spatial and temporal scales. Among other features and benefits, the disclosed embodiments can be implemented in various applications to address these limitations.

[0084] Recording brain activity with high spatial and high temporal resolution across deeper layers of cortex has been a long-sought methodology to study how neural information is coded, stored, and processed by neural circuits and how it leads to cognition and behavior. Electrical and optical neural recording technologies have been the key tools in neurophysiology studies toward a comprehensive understanding of neural dynamics. Optically transparent neural microelectrodes based on the disclosed technology can facilitate multimodal experiments by combining simultaneous electrophysiological recordings from the brain surface with optical imaging and stimulation of neural activity.

[0085] Disclosed herein are fabrication techniques to create transparent graphene microelectrodes with ultra-small openings and a large, completely transparent recording area. This can be achieved by using long graphene microwires without any gold extensions in the FOV In some implementations, Pt nanoparticles are used to overcome the quantum capacitance limit ofgraphene and scale down the microelectrode diameter to 20 .m. In some implementations, the disclosed embodiments employ id-DLG to prevent open circuit failure due to defects and disconnections in long graphene wires. As will be discussed further in the description that follows, centimeter-scale long transparent graphene wires with microscale width and low resistance can be achieved in accordance with implementations of the disclosed technology. By overcoming the quantum capacitance limit of graphene and utilizing id-DLG, high-density microelectrode arrays (e g., 256 channels) can be fabricated.

[0086] In an example use case to be described in further detail, a multimodal experiment was performed which combined recordings of cortical potentials obtained from high-density transparent arrays based on the disclosed technology with two-photon calcium imaging from layer 1 (LI) and layer 2 / 3 (L2 / 3) of the VI area of mouse visual cortex. Recordings using the high- density arrays showed that the visual evoked responses are more spatially localized for high- frequency bands, particularly for the MUA band. The MUA power was found to be strongly correlated with the cellular calcium activity. Leveraging this strong correlation, dimensionality reduction techniques and neural networks were employed to demonstrate that single-cell (L2 / 3) and average (LI and L2 / 3) calcium activities can be decoded from surface potentials recorded by the high-density transparent graphene arrays. High-density transparent graphene electrodes based on the disclosed technology, in combination with multimodal experiments and computational methods, could lead to the development of minimally invasive neural interfaces capable of recording neural activity from deeper layers without requiring depth electrodes that cause damage to the tissue. This could potentially improve brain computer interfaces and enable less invasive treatments for neurological disorders.

[0087] Understanding complex dynamics of the brain and the central nervous system requires studying mechanisms and functions in a diverse set of spatial and temporal scales. For example, studying neural circuits involves millimeter or centimeter spatial scales, studying single neurons involves micron spatial scales, studying synapses involves submicron spatial scales, and studying proteins such as ion channels and receptors involves nanometer spatial scales. This spatial diversity also cultivates temporal diversity where some molecular processes are taking place in microseconds, action potentials in milliseconds, neurotransmitter or hormone release in minutes and learning and behavioral changes in hours to days. In the existing state of the art, monitoring neural dynamics and interrogating neural function across these diverse spatial and temporal scalesis not possible using a single tool or technology. Therefore, integration of multiple tools and sensing and stimulation modalities in the same experiment are typically employed to link mechanisms and functions operating at these different spatiotemporal scales such that a more comprehensive understanding of the brain can be achieved.

[0088] To date, multimodal experiments have been used to investigate neural dynamics with applications ranging from studies of neural circuits or pathophysiology of brain disorders such as Parkinson’s disease, Alzheimer’s disease, and Schizophrenia to hybrid BCIs combining two different modalities with complementary strengths to enhance the performance. Among these multimodal approaches, experiments concurrently recording electrophysiological during optical imaging and optogenetic stimulation has become a powerful approach to (i) combine the temporal resolution advantage of electrophysiology with the high spatial resolution and cell-type specificity of optical methods, (ii) bridge the knowledge gap between basic neuroscience research relying on optical methods employing genetic modifications and clinical research mainly using electrical recordings, (iii) expand spatial reach of neural recordings, and (iv) identify cell types through optotagging during electrophysiological recordings of neuronal spikes. To enable crosstalk and artifact free integration of electrical and optical modalities, transparent graphene electrodes have been proposed. Graphene offers characteristics desirable for multimodal neural interfaces including transparency, artifact-free recording capability, flexibility, low noise, biocompatibility, and chronic reliability. Furthermore, active neural interfaces based on graphene have been shown to offer exceptional bandwidth by recording ultra-slow neural dynamics without voltage drift. It has been demonstrated elsewhere with accelerated aging tests that transparent graphene electrodes are expected to function reliably up to 2.6 years in vivo. Transparent graphene electrodes have been employed in multiple chronic studies to record neural activity from the mouse visual cortex and hippocampus for up to 11 weeks. Other materials have also been investigated for fabricating transparent electrodes, such as indium-tin-oxide (ITO), carbon nanotube meshes (CNTs), metal nanowires, meshes or grids, and PEDOT:PSS. FIG. 1 shows a table comparing electrode area, channel count, electrode pitch, normalized impedance, areal coverage, optical transmittance, and chronic recording reliability between the current state-of-the-art in transparent array technologies and an example embodiment of an ultra-high density transparent graphene (shaded box in FIG. 1) array based on the disclosed technology. As shown in FIG. 1, many of the current technologies have limitations such as low channel counts (maximum of 16), large electrode opening sizes (50pm or larger), and limited coverage (maximum of 3.6 mm2), restricting the spatiotemporal resolution for neural recordings. Several other constraints also limit the use of other materials as multimodal chronic interfaces. The brittle nature of ITO, for instance, makes it susceptible to crack formation and mechanical degradation. CNTs and nanowires have shown cytotoxicity in many studies raising concerns on biocompatibility. Metal nanowires and meshes can still absorb light, leading to light-induced artifacts in electrical recordings due to photovoltaic and photothermal effects. PEDOT:PSS can exhibit chronic reliability issues due to delamination. FIG. 2 shows an example plot of optical transmittance as a function of normalized impedance (impedance x electrode area) for all the electrode technologies listed in the table shown in FIG. 1. Among all other alternatives, ultra-high density graphene electrodes based on the disclosed technology provide the smallest electrode size, highest channel count and density, largest coverage, highest optical transmittance, and lowest normalized impedance.

[0089] Reducing electrode dimensions to single-cell size is desirable to detect high-frequency activity including MUA and single unit (SUA) activities with high signal to noise ratio. Increasing the array density and channel count is necessary to capture neural dynamics with high spatial resolution across large areas. Two important challenges must be addressed to realize high-density transparent graphene arrays with ultra-small electrodes. First, in order to keep the FOV clear, microwires of the arrays need to be completely transparent, particularly for high-density arrays. This requires patterning thin and long graphene wires. However, scaling the graphene wires results in increased wire resistance leading to signal attenuation and increases the susceptibility to structural defects from growth or fabrication causing open circuit failures. Second, scaling down the graphene electrode dimensions drastically increases the impedance due to the quantum capacitance, an intrinsic property of graphene due to its unique band structure.

[0090] Disclosed herein are example embodiments which overcome these challenges and include, among other things, optically transparent, high-density, microelectrode arrays with ultrasmall graphene electrodes which can be implemented in multimodal experiments. As will be discussed in the description that follows, the sheet resistance of graphene wires can be reduced 7- fold by adopting double layer graphene and interlayer nitric acid doping, in accordance with implementations of the disclosed technology, to realize high aspect ratio graphene wires with high yield. In one example embodiment, high-density graphene arrays with up to 256-channels are fabricated without any metal wires in the FOV to prevent any shadows that can block the imagingFOV and cause light-induced artifacts. In some implementations of the disclosed embodiments, PtNPs are employed in the arrays to overcome quantum capacitance limitations and lower the impedance of small graphene electrodes. Such techniques can be employed to achieve low impedance electrodes (e.g., -250 kQ impedance for electrodes with 20 pm diameter).

[0091] In an example use case, transparent, high-density electrodes based on the disclosed technology are implanted over the visual cortex of mice and two-photon calcium imaging at different depths is performed simultaneously. Example results from these experiments show that the surface potentials at high frequencies are highly correlated with the average calcium activities of L2 / 3 neurons. Using the multimodal dataset acquired from these experiments, it is demonstrated that recurrent neural networks (RNNs) can be trained to predict the average calcium activities at LI and L2 / 3 from surface recordings. Moreover, it is demonstrated that a representative latent space can be extracted from the neural population’s calcium response and the RNNs can be trained to decode the latent variables. The decoded latent variables can be projected back to the original space to predict single-cell activities of L2 / 3 neurons. These results demonstrate that the average (LI and L2 / 3) and single-cell (L2 / 3) calcium activities can be predicted from the surface potentials recorded by graphene electrodes based on the disclosed technology.L Elimination of defects and reduction of resistivity to enable large area high-density transparent arrays

[0092] Disclosed herein are techniques related to the fabrication of transparent graphene arrays in accordance with implementations of the disclosed technology.

[0093] Complete transparency of the graphene arrays is crucial for multimodal experiments with a completely clear FOV. Previous designs of transparent graphene arrays using monolayer graphene used gold wires surrounding the recording electrode area, which limited the FOV and increased the potential for light-induced artifacts. FIGS. 3A-C show a conventional 64-channel gold array with 20 pm wire widths (FIG. 3A) and a conventional 64-channel gold array with 6 pm wire widths (FIG. 3B) in comparison to a fully transparent 64-channel graphene array, without surrounding gold wires, based on the disclosed technology (FIG. 3C). Although gold microwires offer low resistivity and can be miniaturized to achieve partial transparency for microelectrode arrays, they suffer from light-induced artifacts during two-photon imaging. The severity of optical blocking or shadows and light-induced artifacts depends on the experimental parameters and the type of optical modality used. To further investigate these effects during two-photon imaging,characterization experiments using both the gold and graphene electrodes can be performed. FIG. 3D shows example images (100 pm scale bar) of shadows created by opaque gold wires in the FOV during two-photon imaging at 50 pm (left) and 250 pm (right) depth under the electrodes, resulting in information loss by masking a subset of neurons. FIG. 3E shows examples of signals recorded by the gold electrodes shown in FIG. 3D and the graphene electrodes shown in FIG. 3C during two-photon imaging with a focal plane located 50 pm below the electrode. As shown in FIG. 3E, significant light-induced artifacts are generated by the gold electrode, while no artifacts are present in the signals recorded by the graphene electrodes. FIG. 3F shows an example of the power spectral density of signals recorded by the graphene and gold electrodes during an example two-photon Z-scan from 50 pm to 150 pm depth under the electrodes (with five frames at each level and 2 pm steps). Unlike graphene, the signals recorded by the gold electrode contain all the harmonics of the two-photon scanning frequency (~30 Hz), which heavily distort neural recordings and make it quite impossible to interpret the electrical recordings in multimodal experiments. Conversely, microwires of the high-density, graphene arrays presently disclosed can maintain complete transparency across the entire recording area and eliminate the light-induced artifacts (FIG. 3C).

[0094] To build high-density graphene arrays with an extended fully transparent recording area, it is necessary that the width of the graphene wires be reduced and the electrode density be increased without causing a substantial increase in wire resistance. Unlike conventional metal microwires with finite thicknesses, graphene has relatively high sheet resistance due to its singleplane 2D atomic structure and grain boundaries. Therefore, reducing the width and increasing the length of graphene wires can significantly increase the wire resistance and may lead to attenuation of the recorded signals. Furthermore, thin and long graphene wires are susceptible to defects in the growth and fabrication processes. These defects increase the probability of having open circuits in the graphene wires and reduce the yield of the graphene microelectrode array.

[0095] Implementations of the disclosed embodiments can address the aforementioned challenges by using id-DLG to build flexible and transparent arrays with long, low resistance graphene wires and ultra-small microelectrodes. FIG. 4A shows an example embodiment of a high- density, transparent graphene array including 64 channels and an example image (100 pm scale bar) of a magnified portion of the array with graphene wires shown in the image using dashed lines. The high-density, transparent graphene array shown in FIG. 4A includes id-DLG layersdoped with nitric acid (HNO3). To form the id-DLG layers, the first graphene layer is transferred using an electrochemical delamination transfer method and doped by dipping it in a 50% HNO3 solution. Then, the second graphene layer is transferred using the same method as the first graphene layer to cap the interlayer dopants. FIG. 4B shows an example schematic of HNO3 id- DLG. Trapping dopants between two graphene layers is important for achieving stable doping and the subsequent decrease in resistivity of graphene layers. Details of the fabrication steps are explained in further detail in Section 6 below. FIG. 4E shows an optical image (1 mm scale bar) of another example embodiment of a high-density graphene array (256 channels) that can be achieved in accordance with implementations of the disclosed technology.

[0096] The disclosed id-DLG approach is effective in eliminating the defects formed in growth or fabrication of graphene. FIG. 4C shows example two-photon microscopy images (10 pm scale bar) of graphene microwires made of single layer graphene (shown top in FIG. 4C) and id-DLG (shown bottom in FIG. 4C). SLG often exhibits defects that emerge during fabrication, which can lead to open circuits when the graphene is patterned to form microscale wires. These defects pose a constraint when trying to scale the graphene wires to build high-density and large area microelectrode arrays. These defects are randomly distributed over the graphene, and it is improbable to have overlapping defects across the top and bottom layers. Therefore, the use of double layer graphene in disclosed embodiments allows for continuous conductivity in extended microscale graphene wires, resulting in high yields. Additional exemplary two-photon microscopy images (20 pm scale bar) of SLG and DLG wire pinholes are shown in FIGS. 5A-F. Specifically, FIGS. 5A-C show example two-photon microscopy images of SLG wires with 20 pm, 30 pm, and 40 pm widths, respectively. FIGS. 5D-F show example two-photon microscopy images of DLG wires with 20 pm, 30 pm, and 40 pm widths, respectively.

[0097] However, as the width of these long graphene wires is reduced, their resistance increases. FIG. 4D shows example graphene wire resistances for SLG, DLG, and id-DLG wires as a function of wire length, where the error bars indicate the standard deviation (n=4). HNO3 is a well-known p-type dopant for graphene that induces a shift in the Fermi level by facilitating the transfer of surface charges between HNO3 and carbon, which enhances the conductivity. id-DLG formed with HNO3 doping effectively reduces the graphene sheet resistance from 1908 Q / sq (SLG) to 276 Q / sq (id-DLG) as shown in the example resistance vs. length plot of FIG. 4D, which enables the graphene wires to be shrunk without any attenuation in recorded signals. It is alsoobserved that double layer graphene without interlayer dopants reduces the sheet resistance to 606 Q / sq (DLG).

[0098] By addressing the graphene defect and sheet resistance issues described above, the disclosed id-DLG approach enables the fabrication of high-density arrays with an extended and transparent FOV for artifact-free multimodal experiments. For instance, FIGS. 4A and 4E show, respectively, example embodiments of 64 and 256-channel arrays which can be fabricated with 20 pm openings and 350 pm center-to-center pitch. The 64-channel array (FIG. 4A) includes a total clear area of 3.1 x 2.8 mm2for imaging and the 256-channel array (FIG. 4E) includes 6.4 x 6.1 mm2available for imaging. Moreover, the disclosed techniques enable high-density, transparent arrays to be fabricated in different configurations to meet specific needs. For example, the transparent arrays can be fabricated to include various opening and pitch sizes (e.g., 50 pm center- to-center) and can be tailored to meet specific requirements of in-vivo experiments. For example, FIGS. 6A, C, D, and E show various embodiments of graphene arrays which can be achieved in accordance with implementations of the disclosed technology. FIG. 6A shows a 256-channel array connected to a recording interface board and FIG. 6B shows an impedance histogram of the 256- channel graphene array. FIG. 6C shows an example microscopy image of a graphene array with 350 pm center-to-center pitch and various opening sizes (20 pm, 30 pm, 50 pm, and 100 pm in diameter). FIG. 6D shows another example microscopy image of a 32-channel graphene array with center-to-center pitch of 30 pm. FIG. 6E shows yet another example microscopy image of an ultra- dense, 64-channel graphene array with center-to-center pitch of 50 pm. The scale bars in FIGS. 6C-E represent 100 pm.2, Overcoming quantum capacitance to build ultra-small electrodes

[0099] Scaling down the electrode dimensions is important for recording high-frequency activity and building high-density arrays. However, ultra-small graphene electrodes exhibit large impedance due to quantum capacitance of graphene, which is a result of low density of states near the Dirac point. FIG. 7A shows an example plot of capacitance versus voltage for SLG, DLG, id- DLG, and a Helmholtz electrical double layer (EDL). The quantum capacitance is dominant in the open-circuit potential range of graphene (-100 mV to 100 mV). As shown in FIG. 7A, employing multilayer graphene and introducing dopants increases the quantum capacitance, however overall capacitance is still dominated by the quantum capacitance since it is larger than the EDL capacitance (see Section 6 for additional details).

[0100] To reduce impedance, electrodes based on the disclosed technology may include electrochemically deposited PtNPs which may circumvent the limits of quantum capacitance effect by creating a low impedance parallel conductance path. PtNPs modify the capacitive el ectrode / electrolyte interface dominated by quantum capacitance of graphene through an increase in the effective surface area and by enabling electrochemical reactions via PtNPs. FIG. 7B shows exemplary microscope (left) and scanning electron microscopy (SEM; right) images (5 pm scale bar) of an id-DLG electrode before and after PtNP deposition. FIG. 7C shows the impedance distribution of 64 channels at 1 kHz of an example array before and after PtNP deposition. The average impedances of the electrodes are 5.4±1.1 MQ and 250±56 kQ (mean±standard deviation), before and after PtNP deposition, respectively. As shown in FIG. 7C, a reduction in average impedance by a factor of 21, from 5.4 M to 250 kQ, is observed.

[0101] To quantitatively analyze the electrochemical impedance of electrodes, an equivalent circuit model for id-DLG with and without PtNPs can be constructed. FIG. 7D shows a diagram of an example equivalent circuit model for an id-DLG electrode with and without PtNPs. In FIG. 7D, where Rsis the solution resistance, Roris the graphene wire resistance, Co is the quantum capacitance, CPEGT and CPEpt are the constant phase elements representing EDL of id-DLG and PtNP / id-DLG electrodes, respectively, WB is the bounded Warburg element explaining diffusion processes, Rct is the charge-transfer resistance that simulates Faradaic reactions, and WE and CE stand for working electrode and counter electrode, respectively. In an example analysis, the conventional Randles model is modified to capture the quantum capacitance effect, resistance of graphene wires, and pseudo-capacitance of PtNP. Unlike other circuit models for PtNP / SLG electrodes, there is no need for a parallel branch to explain the electrochemical reaction at the electrolyte / electrode interface as the graphene electrode openings are completely covered by PtNPs and the interface is converted from electrolyte / id-DLG to electrolyte / PtNP. Therefore, the quantum capacitance component is removed from the equivalent circuit model of FIG. 7D for the electrode / electrolyte interface and Cpand Rct are added to represent the pseudo-capacitance of PtNP. FIG. 7E shows measured EIS data of an example PtNP / id-DLG electrode and modeled results for the PtNP / id-DLG electrode obtained from the fitted equivalent circuit model (FIG. 7D). FIG. 7F shows examples of transmission versus wavelength of different stacks in an example array which includes Parylene-C (PC) and graphene (Gr). FIG. 8 shows a listing of the extracted parameters for PtNP / id-DLG and id-DLG models in accordance with measured EIS data (FIG.9A) obtained for an example id-DLG electrode. Additional details related to electrode characterization and platinum nanoparticle deposition are provided in Section 6 below.

[0102] FIG. 9B shows impedances of example PtNP / id-DLG electrodes fabricated with various PtNP deposition times measured at 1 kHz. It is observed from FIG. 9B that the impedance of the electrodes decreases with increased deposition time. Since the impedance of the electrodes saturated after 150 s of PtNP deposition with a value around 200 kQ, the deposition time may be set to 150 s. In addition, FIG. 9C shows example SEM images of PtNP / id-DLG electrodes with various PtNP deposition times. As shown in FIG. 9C, the PtNP coverage and particle grain size increased with the deposition time (scale bars in FIG. 9C are 3 pm in the top row and 1 pm in the bottom row). Although the transparency of electrodes covered with PtNPs are reduced, they only cover 0.23% of the total area of the array, therefore the PtNP / id-DLG arrays maintain high transparency. FIG. 7F shows examples of transmission versus wavelength of different stacks in an example array which includes Parylene-C (PC) and graphene (Gr).

[0103] FIG. 9D shows example cyclic voltammetry (CV) results before and after 150 s of PtNP deposition, where electrodes with PtNP are represented in back and electrodes without PtNP are represented in red. Current peaks in CV measurements can provide information on electrochemical processes that occur at the interface. In FIG. 9D, the absence of redox peaks in the CV curve of id-DLG indicates that the electrolyte / id-DLG interface is fully capacitive. On the other hand, PtNP-deposited id-DLG shows surface oxide reduction peaks around -300 mV and hydrogen absorption peaks around -500 mV to -800 mV, showing that PtNPs are contributing to the charge transfer process at the electrode / electrolyte interface. Furthermore, the CV curves demonstrate a 7.5-fold increase in the charge storage capacity (CSC) following PtNP deposition, raising it from 4.08 mC / cm2(id-DLG) to 30.72 mC / cm2(PtNP / id-DLG). This enhancement can be attributed to the large pseudo-capacitance of the PtNP interface and the increased surface roughness, which results in a larger effective surface area.

[0104] Although the length and resistance of graphene wires may vary across the array, their impact on the CV and EIS results is minimal. This is illustrated in FIG. 9E which shows the ratio of the graphene wire resistance (Ra) to the electrode impedance (ZEiectrode-Ra) for channels with different wire lengths. The gray dots in FIG. 9E represent the ratios for individual channels, while the red dots and bars indicate the mean and standard deviation, respectively, for groups of eight channels with similar wire length. As shown in FIG. 9E, the resistances of graphene wires proveto be substantially lower than the impedances of the electrodes (excluding the graphene wire resistances), regardless of the length of the graphene wires. Additionally, given the high input impedance of typical amplifiers (~10 MQ), there will not be any significant voltage drop on the graphene wires and attenuation in recorded signals. Hence, the length and resistance of the graphene wires have negligible effect on the electrode characterization results and electrophysiological recordings. Overall, these results demonstrate the successful integration of id-DLG and PtNP to realize high-yield fully transparent graphene arrays with ultra-small electrodes and low impedance for multimodal experiments with uncompromised signal quality.3, Example in-vivo multimodal experiments with transgenic mice

[0105] In an example use case, transparent PtNP / id-DLG arrays based on the disclosed technology were employed in multi-modal experiments to record electrophysiological signals from the cortical surface while conducting calcium imaging with two-photon microscopy from the ipsilateral visual cortex of transgenic mice expressing GCaMP6s in most cortical excitatory neurons (CaMK2-tTA: :tetO-GCaMP6s; see Section 6). The primary visual cortex (VI) was specifically targeted as the focal region, given the diverse sensitivity to various visual stimuli attributes exhibited by VI neural populations. Notably, VI neurons demonstrate a wide range of direction and orientation selectivity when stimulated by drifting gratings. Consequently, the extensive and heterogeneous datasets obtained from these experiments serves as valuable resources for conducting various types of multimodal data analyses. FIG. 10A shows an example schematic of the multimodal experimental setup including drifting gratings which were used as visual stimulation (see Section 6). Two-photon imaging was performed at two different depths, 50 pm and 225 pm, corresponding to LI and L2 / 3, respectively. The FOV was 960 pm x 960 pm, which spans over nine channels. FIG. 10B shows example images of exposed cortex area covered by the array with the FOV depicted by the black square (left) and time-averaged two-photon images of LI (middle) and L2 / 3 (right). PtNP / id-DLG electrodes are shown by yellow circles. The scale bars are 700 pm for the left panel and 150 pm for the middle and right panels. While the imaging was performed in LI or L2 / 3, the neural activities using the 64 channels of the array that spans an area of 2.5 mm x 2.5 mm were simultaneously recorded. FIG. 10C shows representative cortical potentials recorded by the 64 channels during one trial of the visual stimulus. The dotted line represents that duration of the visual stimulus. To ensure that electrophysiology recordings were consistent over the cortex, the noise level of recorded signals was examined by calculatingthe standard deviation of bandpass filtered (0.5-4 kHz) signals for all channels. FIG. 11 A shows example noise levels of all the channels with different graphene wire lengths. The gray dots in FIG. HA represent the noise level for individual channels, while the red dots and lines represent the mean and standard deviation of each group of eight channels with same graphene wire length. FIG. 1 IB shows an example heatmap of the noise levels of all the channels overlaid with a microscope image of the 64-channel array. FIGS. 11A-B illustrate the noise level as a function of wire length for all the electrodes, indicating that the noise level is uniform across the array, regardless of the wire lengths. These results substantiate that the length and resistance of the graphene wires do not affect the signal quality.

[0106] FIG. 11C shows an example of pyramidal cells around (blue regions of interest) and underneath (red regions of interest) the electrode (green circle). FIG. 1 ID shows the fluorescence changes AF / F signals of the pyramidal cells in FIG. 11C, demonstrating that the PtNP / id-DLG electrode does not obstruct the FOV and affect the two-photon signal quality. The scale bars in FIGS. 11C and 1 ID indicate 20 pm and 5 z-score, respectively. The high optical transparency of the implanted array allowed for easy detection of excitatory neurons and their compartments and the recording of calcium signals with single-cell resolution. The imaging quality was not compromised by the transparent graphene array and the ultra-small PtNP electrodes did not obstruct the FOV (FIGS. 11C-D). Following motion correction and detection of neural regions of interest, fluorescence signals were extracted and fluorescence changes AF / F were calculated using the Suite2p software (see Section 6). Representative fluorescence activities of ten neurons are depicted in FIG. 10E and are plotted in FIG. 10F (5 z-score scale bar). Specifically, FIG. 10E shows the ten neurons highlighted from the red box in FIG. 10B and FIG. 10F shows the normalized AF / F signals of the ten neurons. The direction and duration of the drifting gratings are represented in FIG. 10F using the black arrows and gray bars, respectively. FIG. 10D shows the trial averaged population activity (relative to the 2-second baseline before stimulus onset) of neurons detected in L2 / 3, where black dashed lines indicate the onset and offset of the visual stimulus. FIG. 10D illustrates that the imaged cells could be categorized into three groups based on their specific responses to the stimulus; activated, suppressed, and non-modulated (see Section 6). Activated cells exhibit an increase in their activity while suppressed cells show decreased activity during stimulus presentation. Non-modulated cells do not show significant changes in their activity during visual stimulation.

[0107] Unlike L2 / 3, LI is occupied mainly by intermingled neuropils, including dendrites and axons extended from deeper layers. Therefore, the observed fluorescence represents dendritic and axonal activity. As there are almost no detectable cell bodies at this depth, the activity of LI is defined as the average (pixel-level) fluorescence changes in the FOV (excluding the blood vessels, see Section 6). FIG. 10G (5 z-score scale bar) shows a representative example of the pixel-level average calcium activity (AF / F signal) of LI in response to drifting gratings presented in eight different orientations. The direction and duration of the drifting gratings are represented in FIG. 10F using the black arrows and gray bars, respectively.

[0108] In the example use case presently described, the flexible array enabled the recording of surface potentials from 64 channels that spanned over a large area (2.5 X 2.5 mm) of the cortex including regions such as primary visual cortex (VI), primary somatosensory cortex (SI), posterior parietal cortex (PPC), and retrosplenial cortex (RSC). FIG. 12A shows an example schematic of the cortical regions covered by the 64 channels of the array. With such broad spatial coverage, the flexible arrays enabled examination of the propagation of visual stimulation responses. FIG. 12B shows examples of the peak-to-peak amplitude (left) and delay map (right) of the visual evoked responses. The horizontal and vertical scale bars in FIG. 12B are 250 ms and 100 pV, respectively. The visual stimulation responses were initiated from the top-right parts of the array that are located over the visual cortex and spread to the other channels while the peak amplitudes are detected over VI. The top three rows of the array, which were placed over VI and RSC, appeared to have the strongest biphasic responses. The power of visual evoked responses were analyzed at different frequency bands. FIG. 12C shows example spatial maps of the evoked powers (relative to the baseline) at different frequency bands across the array. It was found that the high-frequency bands (y, and MUA) were more localized compared to low-frequency bands (5, 0), which propagated to RSC, PPC, and even SI (see Section 6). This result is consistent with previous works that showed the spatial reach of signal is limited at higher frequencies. The small electrodes (20 pm) with low impedance allowed MUA to be recorded from the cortical surface with high fidelity. To compute event-triggered MUA averages, the MUA event times were identified (see Section 6) for a target channel and the average MUA waveform was calculated for all 64 channels, spanning a time window of 1 ms before to 1 ms after each event in the target channel. FIG. 12D shows representative event-triggered MUA waveforms on different channels, where the horizontal and vertical scale bars are 2 ms and 20 pV, respectively, and average waveforms for select channels,distinguished by different colors in FIG. 12D, indicate that detected MUA events are quite localized over the cortex. These short-duration spikes recorded from the surface were classified as MUA since their auto correlograms, shown in FIG. 13 A, did not show any refractory period. To investigate the origins of MUA spikes detected from the surface, the correlation between the cellular signals from calcium imaging and the MUA power for each channel was examined. To calculate the MUA power, the signal was bandpass-filtered between 500 Hz to 4 kHz and the squared values were smoothed with a Gaussian kernel (see Section 6). First, the peaks of the cell- averaged calcium signal were extracted and then the time-average of MUA power around the onset times of those peaks for all 64 channels was taken (see Section 6). FIG. 12E shows the example correlation between the cell-averaged calcium peaks and MUA power around the peak onset for all 64 channels, where the channels in the FOV show the highest correlation values and the yellow box shows the channel with maximum correlation (r=0.71). Black dashed boxes and black circles in the colormaps in FIGS. 12B, 12C, and 12E indicate the FOV and the electrodes’ locations, respectively. As shown in FIG. 12E, the high correlation between the cellular calcium peaks and the MUA for the channels within the FOV suggests that the spiking activity of L2 / 3 excitatory neurons underneath these channels is an important contributor to the MUA signals detected on the surface. FIG. 12F shows a representative example of the cell-averaged calcium signal AF / F and MUA power of the channel with maximum correlation (i.e., yellow box in FIG. 12E), depicted in FIG. 12F using a 2 z-score scale bar for calcium and a 0.5 dB scale bar for MUA power. The correspondence between the two signals is evident from the sharp deflections in the MUA power followed by peaks in the calcium signal. FIG. 12G shows an example scatter plot of cell-averaged calcium peaks and corresponding MUA powers for the channel with maximum correlation (yellow box in FIG. 12E), demonstrating the correlation between calcium peaks and MUA power extracted from the whole recording of the same channel. Similar correlation values were found between MUA power and cell-averaged calcium signal in other experiments with 16 channel PtNP / id-DLG arrays. FIGS. 13B-E shows the correspondence between cell-averaged calcium signals and MUA power from the two other experiments with 16-channels arrays. FIG. 13B shows a representative example of cell-averaged AF / F and MUA power of the channel with maximum correlation for mouse 2, using a 2 z-score scale bar for calcium and 1 dB scale bar for MUA power. FIG. 13C shows an example scatter plot of calcium peaks and MUA powers for the channel with maximum correlation for mouse 2. FIG. 13D shows a representative example of cell-averaged AF / F andMUA power of the channel with maximum correlation for mouse 3, using a 2 z-score scale bar for calcium and 0.5 dB scale bar for MUA power. FIG. 13E shows an example scatter plot of calcium peaks and MUA powers for the channel with maximum correlation for mouse 3.4, Predicting neural activity in LI and L2 / 3 from surface recordings

[0109] Disclosed herein are techniques to predict deep layer brain activity from surface potential recordings in accordance with implementations of the disclosed technology.

[0110] Given the demonstrated correlation between the MUA power recorded from the surface and the cellular calcium signals imaged at 225 pm depth in the example in-vivo multimodal experiments, it is relevant to ask whether it is possible to predict the brain activity at deeper layers by only harnessing high-resolution electrical recordings from the cortical surface. An example embodiment of a neural network model to predict such deep layer brain activity using surface recordings is presently disclosed. In an example implementation, the neural network model was applied to data acquired from the in-vivo multimodal experiments as will be discussed in the description that follows.

[0111] In an example embodiment, the neural network model includes a linear hidden layer, a single-layer bidirectional LSTM (BiLSTM) network, and a linear readout layer. FIG. 14A shows an example schematic of the neural network model which can be used to decode calcium activity from recorded surface potentials. Signal powers at different frequency bands (ten channels are shown in FIG. 14A as an example) around time t were used as inputs to the model to decode the calcium activity at time t. The neural networks were trained to learn the nonlinear relationships between cellular calcium activities and surface potentials in the in-vivo multimodal experiments. It is important to emphasize that simultaneous recordings enabled by the high transparency of the disclosed graphene microelectrode arrays are critical to acquiring the multimodal datasets needed for training the neural networks. The power of signals at different frequency bands (8: 1-4 Hz, 9: 4-7 Hz, a: 8-15 Hz, 0: 15-30 Hz, y: 31-59 Hz, H-y: 61-200 Hz, MUA: 0.5-4 kHz) were fed as inputs to the network to predict the pixel-level averaged calcium fluorescence change of LI and L2 / 3 and the cell-averaged activity of L2 / 3. Five-fold cross-validation was performed by splitting the 40-minute recording sessions into eight-minute-long segments. Representative examples of decoded and ground truth activities for LI and L2 / 3 are shown in FIG. 14B, which shows the decoded (orange) versus ground truth (black) AF / F of LI (pixel -averaged) and L2 / 3 (cell-averaged). Calcium activity predicted from the surface potentials shows good agreement with the ground truth calcium fluorescence change for both layers. To evaluate the contributions spatially provided by different channels, the decoding was performed using subsets of channels starting from those closest to the FOV. The order of channels used for decoding calcium signal is shown in FIG. 15 A, where the red area shows the imaging FOV and the blue area indicates the closest 11 channels around the FOV. FIG. 14C shows the decoding performance for LI and L2 / 3 (cell and pixel-averaged) using all seven frequency bands but different numbers of channels. FIG. 14D shows the decoding performance for LI and L2 / 3 (cell and pixel -averaged) using different frequency bands of the 20 channels closest to the FOV, where bars and black lines indicate the mean and the standard error of the mean (s.e.m.), respectively. As shown in FIG. 14C, the decoding performance increased with the inclusion of more channels, which indicates that different channels provide complementary information. However, the decoding performance was saturated when ~20 channels were used, suggesting that additional channels provide redundant information beyond this point.

[0112] The contribution of different frequency bands to the decoding performance was investigated by carrying out decoding using low (8, 9, a, and 0) and high (y, H-y, MU A) frequency components from 20 channels closest to the FOV (9 channels in the FOV and 11 channels around it, see FIG. 15 A). The results show that the best decoding performance is achieved when MUA and y, H-y were included, suggesting that high-frequency components carry a vast amount of information on the neural activity including the cellular spiking in the FOV (FIG. 14D). As demonstrated in FIG. 12C, the low-frequency bands were also modulated by the visual stimulation, so the model could still use these bands as informative features to decode the calcium activity. FIG. 15B shows the correlation between cell-averaged calcium peaks and surface potential powers around the peak onsets at different frequency bands for the 9 channels in the FOV (red area in FIG. 15 A). The gray dots in FIG. 15B show the correlation values for the 9 channels and the blue bars show the average correlation values of 9 channels in the FOV. The correlation between the peaks of average cellular calcium activity and power at different frequency bands for those channels over and around the FOV (see FIG. 15B and Section 6) is significantly larger for high-frequency bands (H-y and MUA). Therefore, excluding the low-frequency components does not have a substantial effect on the decoding performance of cell-averaged calcium activity. This indicates that low- frequency components do not provide additional information when combined with high-frequencybands for decoding cellular calcium activity at depth.5, Predicting cellular calcium activity from surface recordings

[0113] In accordance with implementations of the disclosed technology, electrical recordings from cortical surfaces can be used to train networks and predict the average calcium fluorescence change of neurons in L2 / 3 as was demonstrated in example in-vivo multimodal experiments previously described. However, the average calcium signal mostly represents the dominant and synchronous dynamics in the neural network. One question that can be asked is whether predicting calcium fluorescence of single cells from deeper layers is possible by only using high-resolution recordings of cortical potentials enabled by the disclosed technology.

[0114] Disclosed herein is an example embodiment of a neural network model to predict such single-cell activity at deeper layers using recordings of cortical potentials. In the description that follows, the neural network model is described in the context of the in-vivo multimodal experiments as a non-limiting example implementation of the neural network model.

[0115] Developing a network similar to FIG. 14A to predict the activity of all 136 neurons would require increasing the complexity of the network which is not efficient due to the covariances in the neural activity. Previous studies have shown that the neural activity of neurons could be defined by low-dimensional manifolds that capture most of the variance. Therefore, a better approach is to predict the low-dimensional neural manifolds and project them back to the single-cell space. Gaussian Process Factor Analysis (GPFA) is a generative model that unifies dimensionality reduction and smoothing in one framework to extract latent representations that describe the shared variability of high-dimensional data.

[0116] FIG. 16B shows an example schematic diagram of the model used for single-cell decoding of calcium signals in the in-vivo multimodal experiments. To investigate the feasibility of predicting the single-cell activities of L2 / 3 neurons from the surface potentials, GPFA was used to find a low-dimensional latent space that is very representative of the high-dimensional calcium fluorescence signal. FIG. 16A shows the variance of the high-dimensional calcium signal by each latent variable. Eight distinct latent variables were identified that explain most of the variance of the high-dimensional data shown in FIG. 16A. Next, eight networks were trained (using the same architecture used for the average calcium fluorescence decoding described in Section 6) to predict each of these latent variables separately using the surface recordings of 20 channels closest to the FOV. FIG. 16C shows the decoding results of the eight latent variables over five folds. Toreconstruct the single-cell calcium fluorescence, the decoded latent variables were projected to high dimensional space using the GPFA parameters. FIG. 17A shows an example schematic of the single-cell decoding model. The eight latent variables (LI to L8) extracted using GPFA were used to train BiLSTM models (similar to FIG. 14A). Inferred latent variables were projected to highdimensional space to achieve single-cell AF / F signals. As shown in FIG. 17A, the single-cell coding model includes dimensionality reduction using GPFA, prediction using RNNs, and projection to high dimensional space (See Section 6 and FIG. 16C for more details). Representative examples of decoded and ground truth single-cell activities are shown in FIG. 17B. Specifically, FIG. 17B shows representative examples for decoded (orange) versus ground truth (black) AF / F of five best-decoded cells, where the scale bar is 3 z-score. FIG. 17B demonstrates that the singlecell decoding model presently disclosed can infer the calcium activity of several neurons at depth using electrical recordings from the cortical surface. FIG. 17C shows the decoding performance (100 pm scale bar) of all 136 cells with their location in the FOV, where black circles are the 9 channels inside the FOV. It is noteworthy to mention that maximum correlation is partially limited by the amount of information extracted using the GPFA model, as seen in the reconstructed calcium signals using true latent variables (FIGS. 18A-B). In FIG. 18A, the blue bars show the decoding results (averaged over 5 folds) for all 136 cells using the inferred (predicted) latent variables, the black lines indicate the s.e.m., and the orange line shows the maximum correlation that could be achieved when true latent variables are used for reconstructing single-cell activities. FIG. 18B (3 z-score scale bar) shows representative examples for reconstructed single-cell activities (same cells shown in FIG. 17B) using true latent variables (orange) vs ground truth AF / F (black). In some implementations, prediction error can be further reduced, and correlation values of the decoding model can be further increased by optimizing the dimensionality reduction methods. The decoding performances for modulated (suppressed or activated) and non-modulated cells was also compared. FIG. 18C shows decoding results for the modulated (n=40) versus nonmodulated (n=96) cells. As shown in FIG. 18C, the decoding performance is significantly better for cells that are responsive to the visual stimulus (see Section 6).

[0117] Next, it was investigated whether the high accuracy of the decoding model can be attributed to the low variance of drifting gratings and the population coupling of neurons in response to the visual stimulus. It has been shown that spontaneous activity in the visual cortex is complex and potentially higher dimensional than evoked responses, with population couplings thatresemble those observed in response to complex visual stimuli such as natural images. Therefore, to further assess the performance of the decoding model, the decoding strategy was applied to a separate dataset acquired from sessions without any visual stimulus (spontaneous activity). The single-cell calcium decoding was repeated and the calcium activity of single neurons in the spontaneous sessions was successfully inferred. These results are shown in FIGS. 18D-E. FIG. 18D shows the decoding results (100 pm scale bar) for all 114 cells in the spontaneous session presented with their locations outlined in the FOV, where the 9 channels inside the FOV are marked with black circles. FIG. 18E (3 z-score scale bar) shows the representative examples for decoded (orange) versus ground truth (black) AF / F of the five best-decoded cells in the spontaneous session.

[0118] To investigate the effect of population coupling on single-cell calcium activity inference, the population couplings for both the evoked and spontaneous sessions were computed (see Section 6). It was found that the population couplings are quite diverse in both sessions and several neurons have calcium activities that are invariant to the stimuli and have similar levels of population coupling in the evoked and spontaneous recordings. FIGS. 19A-B show a comparison of population coupling across cells (n=l 14) in the spontaneous and evoked sessions. In FIG. 19A, the comparison is performed using Equation (3). In FIG. 19B, the comparison is performed using Equation (4). In FIGS. 19A-B, the red (blue) circles indicate highly coupled cells in the spontaneous (evoked) session with low population coupling in the evoked (spontaneous) session. As shown using blue and red dots in FIGS. 19A-B, it was found that some of the neurons are highly coupled to the population activity in the spontaneous sessions but not in the evoked sessions and vice versa. These findings suggest a complex and heterogenous relationship between VI population activity and visual stimuli. The examination of the decoding results in relation to population coupling reveals an unintuitive relationship between decoding performance and population coupling. For example, FIGS. 19C-D show the decoding results as a function of population coupling for the spontaneous (FIG. 19C) and evoked (FIG. 19D) sessions using Equation (4). The green boxes in FIGS. 19C-D highlight highly coupled cells with poor decoding results, while yellow boxes highlight cells with high decoding performance but low population couplings. It was found that highly coupled cells exhibited poor decoding results (green boxes in FIGS. 19C-D). On the other hand, some of the top decoded cells showed low levels of population coupling (yellow boxes in FIGS. 19C-D). Therefore, focused was placed on cells with highdecoding performance to investigate whether these cells tend to have high population coupling, as one might expect that population coupled activity drives the bulk of cortical surface potential modulation. The population coupling for different numbers of top cells was calculated and compared with decoding results. It was found that the decoding accuracy for the top inferred cells was consistently better in the evoked sessions, despite having lower population coupling compared to the spontaneous sessions. These results are illustrated in FIGS. 19E-H. FIG. 19E shows the decoding results of the top 10 decoded cells in the evoked and spontaneous sessions. FIG. 19F shows the population couplings of the top 10 decoded cells in the evoked and spontaneous sessions. FIG. 19G shows the decoding results of the top N decoded cells (N=5 to 30) in the evoked and spontaneous sessions. FIG. 19H shows the population coupling of the top N decoded cells (N=5 to 30) in the evoked and spontaneous sessions. Solid lines and shaded regions in FIGS. 19G- H indicate the mean and s.e.m., respectively, and population couplings were calculated among the top N decoded cells. These findings suggest that while population coupling might potentially contribute to the inference of calcium activity of individual cells, it cannot be the sole contributing factor that determines the decoding performance.

[0119] Taking everything into account, the decoding results indicate that the surface potentials recorded by transparent PtNP / id-DLG arrays based on the disclosed technology carry information about the neuronal activities in both superficial and deep layers of the brain and can be used to infer neural population dynamics not only at the average but even at single-cell level by projecting higher dimensional neural activity to a latent space.

[0120] Various example embodiments of transparent, high-density graphene arrays with ultrasmall electrodes can be fabricated in accordance with implementations of the disclosed technology. As previously described, the graphene arrays can be used to study neural dynamics at different cortical layers with complementary spatiotemporal resolution provided by optical imaging and electrophysiological recording. Complete transparency of the graphene arrays enables multimodal experiments to be performed. In one example use case, electrical recordings from surface were obtained using the graphene arrays and combined with two-photon imaging from depth to investigate the neural dynamics in the visual cortex of awake mice presented with drifting gratings as visual stimuli. By using double layer graphene, interlayer doping, and PtNP deposition, high- aspect-ratio graphene wires and ultra-small electrodes with low impedances can be achieved and drastically reduce artifacts induced by two-photon imaging. Compared to other transparentmicroelectrode array technologies, the disclosed embodiments provide high channel counts, small electrode sizes, large coverage, and low normalized impedances, while maintaining the transparency of an extended FOV which is essential for crosstalk-free multimodal experiments (see FIGS. 1-2). Moreover, it has been demonstrated elsewhere that transparent graphene electrodes are suitable for long-term chronic experiments.

[0121] In the example in-vivo multimodal experiments described in the present patent document, a multimodal dataset was analyzed by examining visually evoked potentials (VEPs), which are widely studied in neuroscience as a signature of the visual system and utilized by researchers to assess and validate the efficacy of new neural recording technologies. Findings of the analysis revealed that the VEPs recorded by the transparent graphene electrodes based on the disclosed technology align with those previously reported in the literature. Additionally, the multimodal datasets were explored and it was found that the trial averaged signal powers at different frequency bands (8, 0, y, and MU A) demonstrated different spatial propagation patterns. Consistent with previous studies on the propagation of signals, it was also found that the responses are more localized at higher frequency bands. It was also observed that the peaks of cell-averaged calcium activity of L2 / 3 neurons are highly correlated with the increases in the MUA power of those channels in and around the FOV. Such correspondence indicates that the high-frequency activities recorded by ultra-small graphene electrodes on the surface convey information about the spiking activity of pyramidal cells at depth. This finding is in line with previous electrophysiology studies that demonstrated a correlation between signals in deep and superficial layers by recording LFPs at different depth using various types of penetrating and surface electrodes. The means for the propagation of spiking activities to the cortical surface could be the axons projected to the superficial layers, volume conduction, and other connections in the neural network. These results indicate that surface potentials convey information about the neuronal activity at depth and proper features extracted from these signals can be used to predict the cellular calcium activities at depth.

[0122] To that end, the example in-vivo multimodal experiments first focused on the average calcium signals in LI and L2 / 3. Due to the absence of neuronal bodies in LI, pixel-averaged AF / F was used as a representative output signal and both the cell- and pixel-averaged AF / F signals were used for L2 / 3. RNNs were developed and fed the powers of 7 different frequency bands (8, 0, a, P, y, Hy, and MUA) as predictive features to decode the average calcium activities in LI and L2 / 3. The inferred calcium signals resembled the true signals with minimal error which demonstratesthe performance of the disclosed decoding model. To investigate the spatial and frequency contribution of these features, the decoding was repeated with different combinations of channels and frequency bands. It was demonstrated that the inclusion of more channels improves the decoding performance due to the non-redundant information provided by each channel. However, adding the channels that are farther away from the FOV does not improve the decoding performance. This indicates that while the channels farther away may contain information about neuronal activities in their vicinity and local neural activities from other regions of the brain, such as the RSC, SI, and PPC, they do not provide any additional information about the activity of imaged neurons in the target FOV in VI. Nevertheless, the disclosed technical approach can be expanded to cover the entire area of the transparent array by performing sequential two-photon imaging across different FOVs (960 pm x 960 pm, which covers 9 channels). It was also demonstrated that excluding the low-frequency (5, 0, a, 0) signals does not deteriorate the decoding performance which suggests that these bands do not provide additional information when combined with high-frequency bands (y, Hy, and MUA).

[0123] To further improve the spatial resolution of the decoding network, GPFA was used to extract a low-dimensional neural manifold and RNNs were trained to separately decode the extracted latent variables. By projecting the decoded latent variables, single-cell calcium activities of all imaged neurons in L2 / 3 could be reconstructed. The activity of several neurons was also predicted with high correlation (r>0.5) which indicates that surface recordings convey information on the spiking activity of neurons at depth. While coupling of the population activity could be a potential factor that contributes to decoding the calcium activity of individual cells in L2 / 3, analysis performed in the in-vivo multimodal experiments suggests that it is not the only determining factor for decoding accuracy. It is also noteworthy to mention that the computational techniques employed for calcium inference at both population and single-cell resolutions are potentially generalizable to other multimodal experiments, provided that the complementary modalities exhibit adequate spatiotemporal resolutions to capture high-frequency activities at the cortical surface with no crosstalk between modalities.

[0124] These results demonstrate that transparent graphene arrays based on the disclosed technology could potentially be integrated with other techniques to facilitate multimodal experiments with unprecedented spatiotemporal resolutions. For instance, optical techniques could be utilized to manipulate / monitor the neural circuits and uncover the complex dynamics of surfacepotentials. Moreover, non-invasive recordings of neural activity at depth through cross modality inference have the potential to extend the lifetime of neural implants and improve the longevity of brain-computer interface (BCI) technologies, which could pave the way for medical translation. Techniques disclosed herein can also enhance the interpretation of electrophysiology studies by reducing the damage to brain tissue and the consequent alteration of neuronal activities, which is a limitation in studies with laminar probes. The disclosed embodiments can be implemented to localize the sources of distinct features in surface recordings, thus holding implications for enhancing and expanding the capabilities of existing BCI technologies in tackling complex motor and behavioral tasks. Furthermore, such multimodal experiments provide an opportunity to investigate the generation and propagation of neural oscillations, which are crucial for various cognitive processes. Ultimately, non-invasive recordings of neural activity at depth have the potential to open new possibilities for developing minimally invasive neural prosthetics or targeted treatments for various neurological disorders.6, Example methods6a. Example fabrication process of id-DLG arrays

[0125] Disclosed herein are techniques related to the fabrication of flexible, transparent graphene arrays in accordance with implementations of the disclosed technology.

[0126] In one example fabrication process, a transparent and flexible substrate for the graphene array is formed by depositing a 14 pm-thick layer of Parylene-C on a 4-inch silicon wafer coated with 100 nm of PMGI SF3 as a sacrificial layer. Next, 5 nm of Chromium and 100 nm of gold are deposited on the Parylene-C substrate via DC sputtering. The Parylene-C substrate is then patterned with photolithography and wet etching to form metal wires and contact pads. The first graphene layer is transferred using an electrochemical delamination process. To decrease the wire resistance, the first graphene layer is immersed into a 50% nitric acid (HNO3) solution for 10 minutes. After cleaning the HNCE-doped graphene with acetone and IP A, the second graphene layer is transferred using the same process as the first layer. To pattern double layer graphene, bilayer photoresist (e.g., PMGVAZ1512) is used and the graphene is etched with oxygen plasma, followed by acetone / isopropyl alcohol (IP A) cleaning. To protect the double layer graphene during the next steps, a 25 nm silicon dioxide (SiCh) etch-stop layer is sputtered onto the patterned graphene. Then, a 2 pm-thick Parylene-C encapsulation layer is deposited and patterned withoxygen plasma to define electrode openings. To remove the protective SiCL layer and get access to the double layer graphene, a 6: 1 buffered oxide etchant (BOE) can be utilized. Finally, the arrays are detached from the wafer by immersing the wafer in acetone and applying slight physical force to the edges of the wafer.6b. Example electrode characterization and platinum nanoparticle deposition

[0127] All the electrochemical characterizations discussed in Section 1 were conducted with a Gamry 600 plus with immersing in lx phosphate buffered saline. Both EIS and CV data were measured under a three-electrode configuration using Ag / AgCl as a reference electrode and Pt as a counter electrode. To avoid electromagnetic noise, all the measurements were conducted inside of Faraday cage. PtNP deposition was conducted using a two-electrode configuration (Gamry 600 plus). The id-DLG electrode was connected to the working electrode while a Pt wire was connected to the counter or auxiliary electrode. While both electrodes were immersed in the FfePtCk (0.05 M) and K2HPO4 (0.01 M) solution, the current of 50 nA was flown from the id-DLG electrode to counter electrode for selected time periods under ambient conditions.6c. Example electrical double layer capacitance and quantum capacitance calculations

[0128] To extract the capacitances, values of CPEGT and CQ are first obtained by fitting the EIS measurement data to the circuit model of id-DLG electrode. Then, the CPE parameters (capacitance parameter, K, and phase change element exponent, a) are extracted and used in Equation (1) to calculate the Cdl, where Rsis the solution resistance.

[0129] Equation (2) and the measured open circuit voltage are used to calculate the impurity concentration of SLG, DLG, and id-DLG.

[0130] Here, vFis Fermi velocity, h is Planck’s constant, and V is open circuit voltage. To plot the capacitances in FIG. 7A, Equation (2) was used and the open circuit voltage was swept from -0.4 to 0.4 V (Cdiis not a function of V, so its value is constant).6d. Animal procedures in the example in-vivo multimodal experiments

[0131] All procedures in the example in-vivo multimodal experiments described above were performed in accordance with protocols approved by the University of California, San Diego Institutional Animal Care and Use Committee and guidelines of the National Institute of Health. Three animals were used in the study. Adult mice (cross between CaMKIIa-tTA (JAX 003010) and tetO-GCaMP6s (JAX 024742), 2 months old) were anesthetized with isoflurane (3% for induction and 1% for maintenance). Both eyes were protected by Vaseline and a circular piece of scalp was removed. After cleaning the underlying bone using a razor blade, a custom-built head plate was implanted to the exposed skull («1 mm posterior to lambda) with cyanoacrylate glue and cemented with dental acrylic (Lang Dental). Two stainless steel screw (F000CE156, J. I. Morris) were implanted over olfactory bulb as reference and ground. A square craniotomy was made over the left hemisphere (-3.5 x 4 mm, centered -1.75 mm lateral and 2 mm posterior to bregma), and dura of the craniotomized area was carefully removed with a hooked needle. The transparent PtNPs / id-DLG electrode array was first attached to a glass window with UV glue and connected to the amplifier board. Then the assembled interface was gently placed onto the exposed cortex with the electrode array facing to the cortical surface. The glass window was gradually press down through a micromanipulator (Sutter Instrument) until the whole electrode array was tightly attached to the cortical surface. Although our target area of the brain was the Primary visual cortex (VI), other neighboring cortical areas were covered by the array including primary somatosensory cortex (SI), posterior parietal cortex (PPC), and retrosplenial cortex (RSC). Vetbond (3M) was applied to fill the gap between the skull and the glass window, and the glass window was further secured with cyanoacrylate glue and dental acrylic. A cocktail of dexamethasone (2 mg / kg body weight), buprenorphine (0.1 mg / kg body weight), and baytril (10 mg / kg body weight) was given at the end of surgery. The animal was fully recovered from anesthesia before recording.6e. Visual stimulation in the example in-vivo multimodal experiments

[0132] Square-wave drifting grating stimuli (100% contrast, 0.04 cycles / degree, 3 cycles / sec, covering entire contralateral receptive field) were presented on an LCD monitor (30 x 38 cm) positioned 15 cm away from the right eye using Psychtoolbox (http: / / psychtoolbox.org / ). Each of 8 orientations (45° apart) were presented for 2 or 2.5 sec on each trial in pseudorandom order, with 8-second inter-stimulus-interval. Each orientation was presented at least 30 times in a session. Moreover, experiments were conducted to record spontaneous activities without any stimuli.6f Two-photon imaging and analysis of imaging data in the example in-vivo multimodal experiments

[0133] Two-photon imaging was conducted for a head-fixed awake mouse through a 16 * 0.8 NA objective (Nikon) mounted on a commercial two-photon microscope (B-scope, Thorlabs) and using a 925 nm laser (Ti :sapphire laser, Newport). Images were acquired at -29 Hz and a resolution of 512 x 512 pixels, covering 960 x 960 pm of the VI area (FIG. 10B). The laser power was -15 mW for imaging LI (~50 pm deep) and ~40 mW for imaging L2 / 3 (-225 pm deep). Acquired images were motion corrected offline. For quantification of calcium signals from LI, pixels in blood vessels and 10 pixels close to frame edges were excluded. The fluorescence time course (F) was calculated as the ground average of remaining pixels in each frame. At each time point, the baseline (Fo) was estimated by the 10thpercentile of the fluorescence distribution. For quantification of calcium signals from L2 / 3 cell bodies, regions of interest were first identified by Suite2P package and then visually inspected to remove non-somatic ones. Next, fluorescence time course of each cellular region of interest and its surrounding neuropil region of interest was extracted using Suite2P package. Then, the fluorescence signal of a cell body was estimated with Fceiibody = FceiiRO! - 0.7*FneuroPiiRoi. AF / Fo was computed as (Fceiibody - Fo) / Fo, where Fo is the 8thpercentile of the intensity distribution during the recording session.

[0134] To analyze the stimulus response of imaged cells, the baseline activity (2 seconds before the stimulus onset) was subtracted from the trial averaged fluorescence signal for each cell body and normalized with the baseline activity. To categorize the cells, the trial averaged fluorescence signal for each cell body was sorted (descending) based on their average of normalized stimulus response (between 0.3 to 3 s after the stimulus onset). The first and last 20 cells were considered as being responsive to the visual stimulation.

[0135] To calculate the population coupling (c;) for the L2 / 3 neurons in the spontaneous and evoked sessions, we employed the following Equations (3) and (4):In Equations (3) and (4),z(t) is the calcium fluorescence change (AF / F) of individual cell i at timepoint t, / j is the average AF / F of cell j over time, H I is the standard deviation of AF / F for cell i over time, and N is the number of cells.6g, Electrophysiology data recording in the example in-vivo multimodal experiments

[0136] Electrophysiological recording was conducted with an RHD2000 amplifier board and RHD2000 evaluation system (Intan Technologies). The sampling rate was set to 20 kHz and the DC offset was removed with the recording system’s built-in filtering above 0.1 Hz. Intan data was imported into MATLAB (MathWorks) and analyzed using custom scripts.6f, Electrophysiology data analysis and statistics in the example in-vivo multimodal experiments

[0137] Electrophysiology data analysis was done mainly in MATLAB v2019b and the decoding was done using Python. Electrodes with impedances above 10 MQ were excluded from analysis. To remove common artifacts (imaging and power line), a bank of notch filters was applied to the raw surface recordings (the filters are optimized for each channel separately). The multi-unit activity (MU A) was extracted by applying a 6thorder bandpass filter from 0.5 to 4 kHz followed by common average referencing. The signals were low-pass filtered below 250 Hz using a 4thorder Butterworth filter to achieve the local field potential (LFP). The visually evoked potentials (VEPs) for each trial were extracted and the trial averaged peak-to-peak amplitude and propagation delay of the stimulus responses were visualized using 2D color maps. To further filter the signals into common low-frequency bands (8: 1-4 Hz, 9: 4-7 Hz, a: 8-15 Hz, [3: 15-30 Hz, y: 31-59 Hz, H-y: 61-200 Hz), 6thorder Butterworth bandpass filters were applied with corresponding frequency ranges.

[0138] The powers at different bands (8, 9, a, , y, H-y, MU A) were calculated by taking the square of the bandpass filtered signals and applying a 100 ms Gaussian filter to reduce the noise. The power changes due to the visual stimulus were calculated by trial averaging the powers at different bands and subtracting the baseline activity (2 seconds before the stimulus onset) and the peak of power changes (in a 4 second window after the stimulus onset) were demonstrated using 2D spatial maps to visualize the localization of different bands. To detect these MUA events, the threshold crossing method was applied on bandpass filtered (0.5 to 4 kHz) signals with a threshold set at -4 times the standard deviation. In FIG. 12D, if multiple nearby channels also captured the neural events, they were assigned the same color as the target channel.

[0139] To analyze the MUA and average cellular calcium correlation, the peaks of normalized cell-averaged AF / F were determined (findpeaks, minimum peak height set to 0.75) and the MUA power of each channel was averaged in a 2-second window [-1.5s, 0.5s] around the calcium peak onset. The Pearson correlation values were calculated for each channel between the calcium peaks and the averaged MUA powers. The same procedure was followed to calculate the correlation of average cellular calcium activity with other frequency bands (8, 0, a, , y, H-y).

[0140] To decode the cell averaged calcium activity from surface potentials, a neural network model with a sequential stack of a linear hidden layer, one bidirectional LSTM (BiLSTM) layer and a linear readout layer was implemented. The linear hidden layer was followed by batch normalization, ReLU activation, and dropout (p=0.3). The BiLSTM layer was followed by batch normalization. Surface potential powers at different frequency bands (8, 0, a, 0, y, H-y, MUA) were down sampled to match the sampling rates of the calcium signal (29 Hz) and clipped with a threshold of 95thpercentile to suppress the potential artifacts. These power signals were then used as inputs to the neural network model. To decode the neural activity at time step t, the power segments between [t - 1.5 s, t + 1.5 s] were used (90 time-steps in total). The 1st linear layer had 25 neurons and the BiLSTM had 15 hidden neurons. The last layer outputs the predicted cell- averaged calcium signal.

[0141] Adam was used to train and optimize the parameters of the model with learning rate = 6 x 10-5, betal = 0.9, beta2 = 0.999, epsilon = 1 x 10-8. The batch size was set to 128 and the training converged within ~20 epochs. The mean squared error (MSE) was used as the loss function. Five-fold cross-validation was performed by splitting the 40 minutes recording sessions into five segments, each lasting for 8 minutes. The Pearson correlation between the decoded and ground truth data was used to evaluate the model performance, and the correlation values were averaged over five folds to get a single correlation value.6f, Low dimensional latent space of population activity

[0142] GPFA models observations as a Gaussian model that is related to the latent variable through the following Equation (5):

[0143] Here, x.trepresents the latent variable at timepoint t, d is the signal mean, C is thefactor loading matrix, and R represents the covariance matrix. The ith latent variable x is modeled as a Gaussian Process (GP) with a covariance matrix K that correlates latent variables across time points: x^N^ K ). (6)

[0144] Using the training data of calcium signal Y, a GPFA model was trained to learn the parameters and infer the trajectory of the latent variable x

[0145] The latent variables x and the inferred latent variable x from the BiLSTM model previously described are used to project into single-cell calcium activity space (FIG. 16C),E[Y] = CX + d (8) E [?] = CX + d (9) where Y is the projected calcium signal using the originally inferred latent variables and Y is the projected calcium signal using the latent variables predicted by the BiLSTM model. The projected calcium signals were then compared to the true calcium signals (FIG. 18).

[0146] FIG. 20 shows a flow diagram illustrating an example method 2000 for measuring activity in a brain at multiple depths. At operation 2010, the method 2000 comprises obtaining, via a first modality and using a transparent flexible electrode array comprising inter-layer doped double layer graphene (id-DLG), a first dataset comprising first signals from the surface or a first depth within the brain. At operation 2020, the method 2000 comprises obtaining, via a second modality that is different from the first modality and using the transparent flexible electrode array, a second dataset comprising second signals from one or more second depths within the brain that are deeper than the first depth, wherein the second dataset and the first dataset are obtained simultaneously. At operation 2030, the method 2000 comprises determining activity at the one or more second depths based on cross modality inferencing results between the first dataset and the second dataset that are obtained using a trained neural network having been trained to learn a nonlinear relationship between the first signals and the second signals.

[0147] FIG. 21 shows a flow diagram illustrating another example method 2100 for analyzing brain activity data. At operation 2110, the method 2100 comprises obtaining a multimodal dataset acquired from a transparent electrode array comprising doped graphene. In some implementations, the multimodal dataset includes a first dataset comprising surface potential recordings of a brainand a second dataset comprising deep layer information of the brain. In some implementations, the first dataset and the second dataset are acquired from the transparent electrode array simultaneously. At operation 2120, the method 2100 comprises determining activity in one or more deep layers of the brain based on cross modality inferencing results obtained from a trained artificial intelligence (Al) network.

[0148] FIG. 22 shows a flow diagram illustrating operations that can be performed in the example method 2100 of FIG. 21. The operations include: determining, using a generative Al model, a set of latent variables associated with variance in the second dataset in a low-dimensional space, predicting, by at least one trained neural network in the trained Al network and for each variable in the set of latent variables, an inferred set of latent variables using at least a portion of the first dataset, and projecting, based on parameters of the generative Al model, each variable in the inferred set of variables to another dimensional space that is higher than the low-dimensional space and associated with one or more deep layers of the brain.

[0149] Embodiments of the disclosed technology support inter alia the following technical solutions that solve the technical problem of measuring or analyzing brain activity.

[0150] 1. A flexible device for measuring brain activity, comprising: a flexible substrate; a transparent array of electrodes disposed on the flexible substrate and comprising an inter-layer doped double layer graphene (id-DLG), wherein the transparent array of electrodes further comprises: a plurality of transparent wires; a plurality of electrode openings configured to obtain brain surface signals therethrough; and a transparent recording area having a field of view comprising at least a portion of the plurality of electrode openings and at least a portion of the plurality of transparent wires such that additional brain signals can be obtained through the field of view as the brain surface signals are simultaneously obtained; a plurality of signal channels disposed on the flexible substrate and configured to facilitate signal communications to and from the transparent array of electrodes; and electrically conductive pads in communication with the plurality of signal channels and configured to interface with data acquisition circuitry to provide the brain surface signals thereto.

[0151] 2. The flexible device of solution 1, wherein the brain surface signals are obtained via a first modality and the additional brain signals are obtained via a second modality different from the first modality.

[0152] 3. The flexible device of solution 2, wherein the first modality is an electrical modalityand the second modality is an optical modality.

[0153] 4. The flexible device of solution 1, wherein the brain surface signals are obtained from a surface layer of a brain, wherein the additional brain signals comprise data describing activity in layers of the brain which are located deeper than the surface layer.

[0154] 5 The flexible device of solution 1, wherein the id-DLG is doped with nitric acid(HN03).

[0155] 6. The flexible device of solution 2, wherein the transparent array of electrodes is dimensioned such that the transparent recording area provides a first spatiotemporal resolution for recording the brain surface signals through a first portion of the field of view via the first modality and a second spatiotemporal resolution for recording the additional brain signals through a second portion of the field of view via the second modality, wherein the first spatiotemporal resolution is different from the second spatiotemporal resolution.

[0156] 7. The flexible device of solution 1, wherein the transparent array further comprises platinum nanoparticles (PtNPs) covering at least some of the plurality of electrode openings, wherein the PtNPs lower impedance of the flexible device by modifying quantum capacitance of graphene in the id-DLG.

[0157] 8. The flexible device of solution 1, wherein the transparent array of electrodes is optically transparent to light of wavelengths between 450 to 800 nm.

[0158] 9 A method for measuring activity in a brain at multiple depths, comprising: obtaining, via a first modality and using a transparent flexible electrode array comprising inter-layer doped double layer graphene (id-DLG), a first dataset comprising first signals from the surface or a first depth within the brain; obtaining, via a second modality that is different from the first modality and using the transparent flexible electrode array, a second dataset comprising second signals from one or more second depths within the brain that are deeper than the first depth, wherein the second dataset and the first dataset are obtained simultaneously; and determining activity at the one or more second depths based on cross modality inferencing results between the first dataset and the second dataset that are obtained using a trained neural network having been trained to learn a nonlinear relationship between the first signals and the second signals.

[0159] 10. The method of solution 9, wherein the first modality is an electrical modality andthe second modality is an optical modality.

[0160] 11. The method of solution 9, wherein the first modality is an electrophysiological modality and the second modality is an optical imaging modality.

[0161] 12. The method of solution 9, wherein the first modality is an electroencephalogram modality and the second modality is a magnetic resonance imaging modality.

[0162] 13. The method of solution 9, wherein the trained neural network includes a linear hidden layer, a single-layer bidirectional LSTM (BiLSTM) network, and a linear readout layer.

[0163] 14. The method of solution 11, wherein the first dataset includes surface layer signals at different frequencies that are used as input to the trained neural network to obtain the cross modality inferencing results.

[0164] 15. The method of solution 11, wherein the cross modality inferencing results are used to obtain one or more reconstructed images of the brain, wherein the one or more reconstructed images describe activity in the one or more second depths at a particular time.

[0165] 16. The method of solution 9, wherein the first dataset and the second dataset are obtained using an identical field of view of the transparent flexible electrode array.

[0166] 17. The method of solution 9, wherein the activity at the one or more second depths describes single-cell level activity of the brain.

[0167] 18. The method of solution 9, further comprising: identifying one or more features in the first dataset that are associated with one or more localized regions of the brain; and determining, based on the one or more features and the one or more localized regions, global activity within the brain, wherein the nonlinear relationship is based at least in part on the one or more features.

[0168] 19. A method for analyzing brain activity data, comprising: obtaining a multimodal dataset acquired from a transparent electrode array comprising doped graphene, wherein the multimodal dataset includes a first dataset comprising surface potential recordings of a brain and a second dataset comprising deep layer information of the brain, wherein the first dataset and the second dataset are acquired from the transparent electrode array simultaneously; and determining activity in one or more deep layers of the brain based on cross modality inferencing results obtained from a trained artificial intelligence (Al) network, wherein the trained Al network is configured toobtain the cross modality inferencing results based at least in-part on: determining, using a generative Al model, a set of latent variables associated with variance in the second dataset in a low-dimensional space, predicting, by at least one trained neural network in the trained Al network and for each variable in the set of latent variables, an inferred set of latent variables using at least a portion of the first dataset, and projecting, based on parameters of the generative Al model, each variable in the inferred set of variables to another dimensional space that is higher than the lowdimensional space and associated with one or more deep layers of the brain.

[0169] 20. The method of solution 19, wherein the first dataset is acquired using a first modality and the second dataset is acquired using is second modality different from the first modality.

[0170] 21. The method of solution 20, wherein the first modality is an electrical modality and the second modality is an optical modality.

[0171] 22. The method of solution 19, wherein the doped graphene comprises interlayer-doped double layer graphene (id-DLG) doped with nitric acid (HNO3).

[0172] 23. The method of solution 19, wherein the transparent electrode array is flexible.

[0173] 24. The method of solution 19, wherein the trained Al network includes a linear hidden layer, a single-layer bidirectional LSTM (BiLSTM) network, and a linear readout layer.

[0174] 25. The method of solution 19, wherein the generative Al model is based on GaussianProcess Factor Analysis.

[0175] 26. The method of solution 19, wherein the activity in the one or more deep layers describes single-cell level activity in the brain.

[0176] 27. The method of solution 19, further comprising: obtaining one or more reconstructed images of the brain that describes the activity in the one or more deep layers at a particular time, wherein the one or more reconstructed images based on the cross modality inferencing results.

[0177] 28. A device, comprising: a processor and a memory with instructions stored thereon, wherein the instructions upon execution by the processor cause the processor to perform the operations recited in any one of solutions 1-27.

[0178] Various operations disclosed herein can be implemented using a processor / controller is configured to include, or be couple to, a memory that stores processor executable code that causes the processor / controller carry out various computations and processing of information. The processor / controller can further generate and transmit / receive suitable information to / from thevarious system components, as well as suitable input / output (IO) capabilities (e.g., wired or wireless) to transmit and receive commands and / or data.

[0179] Various information and data processing operations described herein may be implemented in one embodiment by a computer program product, embodied in a computer- readable medium, including computer-executable instructions, such as program code, executed by computers in networked environments. A computer-readable medium may include removable and non-removable storage devices including, but not limited to, Read Only Memory (ROM), Random Access Memory (RAM), compact discs (CDs), digital versatile discs (DVD), etc. Therefore, the computer-readable media that is described in the present application comprises non-transitory storage media. Generally, program modules may include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Computer-executable instructions, associated data structures, and program modules represent examples of program code for executing steps of the methods disclosed herein. The particular sequence of such executable instructions or associated data structures represents examples of corresponding acts for implementing the functions described in such steps or processes.[00180J Only a few implementations and examples are described and other implementations, enhancements and variations can be made based on what is described and illustrated in this patent document.

Claims

What is claimed:

1. A flexible device for measuring brain activity, comprising: a flexible substrate; a transparent array of electrodes disposed on the flexible substrate and comprising an interlayer doped double layer graphene (id-DLG), wherein the transparent array of electrodes further comprises: a plurality of transparent wires; a plurality of electrode openings configured to obtain brain surface signals therethrough; and a transparent recording area having a field of view comprising at least a portion of the plurality of electrode openings and at least a portion of the plurality of transparent wires such that additional brain signals can be obtained through the field of view as the brain surface signals are simultaneously obtained; a plurality of signal channels disposed on the flexible substrate and configured to facilitate signal communications to and from the transparent array of electrodes; and electrically conductive pads in communication with the plurality of signal channels and configured to interface with data acquisition circuitry to provide the brain surface signals thereto.

2. The flexible device of claim 1, wherein the brain surface signals are obtained via a first modality and the additional brain signals are obtained via a second modality different from the first modality.

3. The flexible device of claim 2, wherein the first modality is an electrical modality and the second modality is an optical modality.

4. The flexible device of claim 1 , wherein the brain surface signals are obtained from a surface layer of a brain, wherein the additional brain signals comprise data describing activity in layers of the brain which are located deeper than the surface layer.

5. The flexible device of claim 1, wherein the id-DLG is doped with nitric acid (HNO3).

6. The flexible device of claim 2, wherein the transparent array of electrodes is dimensioned such that the transparent recording area provides a first spatiotemporal resolution for recording the brain surface signals through a first portion of the field of view via the first modality and a second spatiotemporal resolution for recording the additional brain signals through a second portion of the field of view via the second modality, wherein the first spatiotemporal resolution is different from the second spatiotemporal resolution.

7. The flexible device of claim 1, wherein the transparent array further comprises platinum nanoparticles (PtNPs) covering at least some of the plurality of electrode openings, wherein the PtNPs lower impedance of the flexible device by modifying quantum capacitance of graphene in the id-DLG.

8. The flexible device of claim 1, wherein the transparent array of electrodes is optically transparent to light of wavelengths between 450 to 800 nm.

9. A method for measuring activity in a brain at multiple depths, comprising: obtaining, via a first modality and using a transparent flexible electrode array comprising inter-layer doped double layer graphene (id-DLG), a first dataset comprising first signals from the surface or a first depth within the brain; obtaining, via a second modality that is different from the first modality and using the transparent flexible electrode array, a second dataset comprising second signals from one or more second depths within the brain that are deeper than the first depth, wherein the second dataset and the first dataset are obtained simultaneously; anddetermining activity at the one or more second depths based on cross modality inferencing results between the first dataset and the second dataset that are obtained using a trained neural network having been trained to learn a nonlinear relationship between the first signals and the second signals.

10. The method of claim 9, wherein the first modality is an electrical modality and the second modality is an optical modality.

11. The method of claim 9, wherein the first modality is an electrophysiological modality and the second modality is an optical imaging modality.

12. The method of claim 9, wherein the first modality is an electroencephalogram modality and the second modality is a magnetic resonance imaging modality.

13. The method of claim 9, wherein the trained neural network includes a linear hidden layer, a single-layer bidirectional LSTM (BiLSTM) network, and a linear readout layer.

14. The method of claim 11, wherein the first dataset includes surface layer signals at different frequencies that are used as input to the trained neural network to obtain the cross modality inferencing results.

15. The method of claim 11, wherein the cross modality inferencing results are used to obtain one or more reconstructed images of the brain, wherein the one or more reconstructed images describe activity in the one or more second depths at a particular time.

16. The method of claim 9, wherein the first dataset and the second dataset are obtained using an identical field of view of the transparent flexible electrode array.

17. The method of claim 9, wherein the activity at the one or more second depths describes single-cell level activity of the brain.

18. The method of claim 9, further comprising:identifying one or more features in the first dataset that are associated with one or more localized regions of the brain; and determining, based on the one or more features and the one or more localized regions, global activity within the brain, wherein the nonlinear relationship is based at least in part on the one or more features.

19. A method for analyzing brain activity data, comprising: obtaining a multimodal dataset acquired from a transparent electrode array comprising doped graphene, wherein the multimodal dataset includes a first dataset comprising surface potential recordings of a brain and a second dataset comprising deep layer information of the brain, wherein the first dataset and the second dataset are acquired from the transparent electrode array simultaneously; and determining activity in one or more deep layers of the brain based on cross modality inferencing results obtained from a trained artificial intelligence (Al) network, wherein the trained Al network is configured to obtain the cross modality inferencing results based at least in-part on: determining, using a generative Al model, a set of latent variables associated with variance in the second dataset in a low-dimensional space, predicting, by at least one trained neural network in the trained Al network and for each variable in the set of latent variables, an inferred set of latent variables using at least a portion of the first dataset, and projecting, based on parameters of the generative Al model, each variable in the inferred set of variables to another dimensional space that is higher than the lowdimensional space and associated with one or more deep layers of the brain.

20. The method of claim 19, wherein the first dataset is acquired using a first modality and the second dataset is acquired using is second modality different from the first modality.

21. The method of claim 20, wherein the first modality is an electrical modality and the second modality is an optical modality.

22. The method of claim 19, wherein the doped graphene comprises interlayer-doped double layer graphene (id-DLG) doped with nitric acid (HNO3).

23. The method of claim 19, wherein the transparent electrode array is flexible.

24. The method of claim 19, wherein the trained Al network includes a linear hidden layer, a single-layer bidirectional LSTM (BiLSTM) network, and a linear readout layer.

25. The method of claim 19, wherein the generative Al model is based on Gaussian Process Factor Analysis.

26. The method of claim 19, wherein the activity in the one or more deep layers describes single-cell level activity in the brain.

27. The method of claim 19, further comprising: obtaining one or more reconstructed images of the brain that describes the activity in the one or more deep layers at a particular time, wherein the one or more reconstructed images are based on the cross modality inferencing results.

28. A device, comprising: a processor and a memory with instructions stored thereon, wherein the instructions upon execution by the processor cause the processor to perform the operations recited in any one of claims 1-27.

Images