Systems and methods for using neural objective functions in closed-loop optimization

Abstract

Systems and methods are disclosed that represent various advances in neuroscience. For example, systems and methods are disclosed for optimizing the processing of sensory information, refining diagnostic procedures for neurological disorders, and introducing novel methods for controlling brain network activity. In certain aspects, the collective concepts presented in this disclosure provide concrete and tangible applications in neuroscience and neuroengineering. Furthermore, these concepts represent improvements in computer technology by introducing innovative applications at the intersection of neuroscience and computing.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Application No. 63 / 479,911, filed January 13, 2023, which is incorporated herein by reference in its entirety.

[0002] The present disclosure relates generally to the fields of neuroscience and neuroengineering, and more particularly to systems and methods that utilize novel methodologies that integrate computational neuroscience, signal processing, and machine learning. [Background technology]

[0003] Sensory systems play a critical role in facilitating the flow of information to nonsensory cortical regions. However, direct optimization of downstream sensory pathways can be hindered by sparse or weak neural pathway activity. Furthermore, diagnostic tools for complex neurological disorders such as visual neglect and autism may lack sensitivity and fail to address the individual nature of these conditions. Furthermore, discovering effective sensory inputs that influence brain network activity can be hindered by impractical and / or inefficient search of a vast input space within a limited time frame.

[0004]

[0003] Accordingly, the present disclosure is directed to innovative approaches in neuroscience for optimizing sensory information, diagnosing neurological disorders, and influencing brain network activity. Collectively, these techniques offer transformative advances with potential applications in research and clinical intervention. The description of the background art provided herein is intended to broadly present the context of the disclosure. Unless otherwise indicated herein, the material described in this section is not prior art, is not admitted to be prior art, or is a suggestion of prior art, by inclusion in this section to the claims in this application. Summary of the Invention [Means for solving the problem]

[0005] According to certain aspects of the present disclosure, systems and methods are described for improving the processing of sensory information, the results of which may be utilized in a variety of downstream applications.

[0006] In one aspect, a computer-implemented method for performing iterative adjustments to sensory stimuli in a closed-loop optimization system is provided, comprising: receiving, at a computing device, brain activity data associated with stimuli presented to a subject; pre-processing the brain activity data using a processor associated with the computing device; performing, using the processor, a source estimation technique on the pre-processed brain activity data to identify one or more voxel locations from which electrical activity included in the brain activity data is estimated to originate; and, following performance of the source estimation technique, grouping together a first subset of the one or more voxel locations based on the pre-processed brain activity data, the first subset being associated with a sensory region of interest. the second subset of one or more voxel locations based on the local neural activity identified within the first subset of one or more voxel locations, the second subset being associated with a non-sensory region of interest; determining, using a processor, a first objective function based on the local neural activity identified within the first subset of one or more voxel locations and a second objective function based on the local neural activity identified within the second subset of one or more voxel locations; determining, using a processor, whether the second objective function is greater than or equal to a threshold minimum; in response to determining that the second objective function is greater than or equal to the threshold minimum, averaging with the first objective function an amount by which the second objective function exceeds the threshold minimum to generate an average objective function; and utilizing the average objective function to refine one or more parameters of the stimulation in a subsequent iteration of the closed-loop optimization system.

[0007] In another aspect, a system for performing iterative adjustments to sensory stimuli is provided, comprising one or more processors and one or more computer-readable media having instructions executable by the one or more processors stored thereon to perform operations including receiving brain activity data associated with stimuli presented to a subject, preprocessing the brain activity data, performing a source estimation technique on the preprocessed brain activity data to identify one or more voxel locations from which electrical activity included in the brain activity data is estimated to originate, and subsequent to the source estimation technique, grouping together a first subset of the one or more voxel locations based on the preprocessed brain activity data, the first subset being associated with a sensory region of interest, and subsequent to the source estimation technique, grouping together a second subset of the one or more voxel locations based on the preprocessed brain activity data. and one or more computer-readable media for performing the following steps: grouping the sets together, wherein a second subset is associated with a non-sensory region of interest; determining a first objective function based on the local neural activity identified within a first subset of one or more voxel locations and determining a second objective function based on the local neural activity identified within a second subset of one or more voxel locations; determining whether the second objective function is greater than or equal to a threshold minimum; responsive to determining that the second objective function is greater than or equal to the threshold minimum, averaging with the first objective function an amount by which the second objective function exceeds the threshold minimum to generate an average objective function; and utilizing the average objective function to refine one or more parameters of the stimulation in a subsequent iteration of a closed-loop optimization function of the system.

[0008] In yet another aspect, a non-transitory computer-readable medium storing computer-executable instructions is provided that, when executed by a system, cause the system to receive, at a computing device associated with the system, brain activity data associated with stimuli presented to a subject, pre-process the brain activity data using a processor associated with the computing device, perform a source estimation technique on the pre-processed brain activity data using the processor to identify one or more voxel locations from which electrical activity included in the brain activity data is estimated to originate, and, following performance of the source estimation technique, group together a first subset of the one or more voxel locations based on the pre-processed brain activity data, the first subset being associated with a sensory region of interest, and, following performance of the source estimation technique, group together the pre-processed brain activity data. The method may cause the device to perform operations including grouping together a second subset of one or more voxel locations based on the activity data, the second subset being associated with a non-sensory region of interest; determining, using a processor, a first objective function based on the local neural activity identified within the first subset of one or more voxel locations and a second objective function based on the local neural activity identified within the second subset of one or more voxel locations; determining, using the processor, whether the second objective function is greater than or equal to a threshold minimum; and, in response to determining that the second objective function is greater than or equal to the threshold minimum, averaging with the first objective function an amount by which the second objective function exceeds the threshold minimum to generate an average objective function; and using the average objective function to refine one or more parameters of the stimulation in a subsequent iteration of the closed-loop optimization system.

[0009] Additional objects and advantages of the disclosed embodiments will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the disclosed embodiments. The objects and advantages of the disclosed embodiments will be realized and attained by means of the elements and combinations particularly pointed out in the appended claims.

[0010] The foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosed embodiments, as claimed.

[0011] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate several embodiments and, together with the description, serve to explain the principles of the disclosure. [Brief explanation of the drawings]

[0012] [Figure 1A] 1 illustrates an exemplary computer system for performing the methods described herein.

[0013] [Figure 1B] 1 illustrates an exemplary software platform for performing the methods described herein.

[0014] [Figure 2-1] 1 illustrates an exemplary workflow for performing closed-loop optimization to enhance non-sensory brain activity, in accordance with one or more embodiments of the present disclosure. [Figure 2-2] 1 illustrates an exemplary workflow for performing closed-loop optimization to enhance non-sensory brain activity, in accordance with one or more embodiments of the present disclosure.

[0015] [Figure 3] 1 illustrates an exemplary workflow for diagnosing neurological disorders with visual system function-based diagnosis, according to one or more embodiments of the present disclosure.

[0016] [Figure 4] 1 illustrates an example workflow of a first process involved in utilizing spontaneous brain activity to constrain sensory input used to control brain activity, in accordance with one or more embodiments of the present disclosure.

[0017] [Figure 5]10 illustrates an example workflow of a second process involved in utilizing spontaneous brain activity to constrain sensory input used to control brain activity, in accordance with one or more embodiments of the present disclosure.

[0018] [Figure 6] 1 illustrates an example workflow for utilizing short time analysis windows and hyperscanning for neural SNR maximization for closed-loop optimization, in accordance with one or more embodiments of the present disclosure.

[0019] [Figure 7] 10 illustrates another example workflow for utilizing short time analysis windows and hyperscanning for neural SNR maximization for closed-loop optimization, in accordance with one or more embodiments of the present disclosure.

[0020] [Figure 8] 10 illustrates another example workflow for utilizing short time analysis windows and hyperscanning for neural SNR maximization for closed-loop optimization, in accordance with one or more embodiments of the present disclosure.

[0021] [Figure 9] 1 illustrates an exemplary computing system in accordance with one or more embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0022] The terms used below are to be interpreted in their broadest reasonable manner, even when used in conjunction with the detailed description of certain specific examples of the present disclosure. Indeed, although certain terms may be emphasized below, any terms intended to be interpreted in a limiting manner are expressly and specifically defined as such in this Detailed Description section. Both the foregoing general description and the following detailed description are exemplary and explanatory only and are not limiting of the claimed features.

[0023] In this disclosure, the term "based on" means "based at least in part on." Unless context dictates otherwise, the singular forms "a," "an," and "the" include plural referents. The term "exemplary" is used to mean "example," not "ideal." The terms "comprises," "comprising," "includes," "including," or other variations thereof are intended to cover non-exclusive inclusions, and thus a process, method, or product that includes a list of elements does not necessarily include only those elements, but may also include other elements not expressly listed or elements not inherent in such process, method, article, or apparatus. Relative terms such as "about," "approximately," "substantially," and "generally" are used to indicate a possible variation of ±10% from the stated or understood value. Furthermore, the term "between," when used when describing a range of values, is intended to include the minimum and maximum values ​​set forth herein. The use of the term "or" in the claims and specification is used to mean "and / or" unless expressly indicated to refer only to alternatives or where the alternatives are mutually exclusive, but this disclosure supports a definition that refers only to alternatives and "and / or." As used herein, "another" may mean at least a second or more.

[0024] As used herein, the term "user" generally encompasses any individual or entity, such as a researcher and / or caregiver (e.g., a physician), who may desire information, problem resolution, or any other type of interaction with a provider of the systems and methods described herein (e.g., via an application interface present on an electronic device, etc.). The terms "electronic application" or "application" may be used interchangeably with other terms, such as "program," and generally encompass software configured to interact, modify, overwrite, supplement, or operate in conjunction with other software.

[0025] Vision, the ability to see and understand the world around us, is critical to the quality of life and independence of most humans. In healthy visual processing, light activates photoreceptor cells in the retina of the eye, which then communicate with an extensive visual processing network that encompasses more than half of the brain's cortex. This network can then innervate other areas of the brain. Unfortunately, disease or damage to either the eye, the optic nerve, or nonsensory areas of the brain can significantly impair visual or other neurological functions. Such disorders are a global problem affecting all demographics but are particularly prevalent among the elderly. Relevant examples include age-related macular degeneration (AMD), diabetic retinopathy, glaucoma, myopia, cortical visual impairment, visuospatial neglect, stroke, Parkinsonism, epilepsy, multiple sclerosis, hearing loss, balance disorders, tumor-induced functional impairments, paralysis, autism, visual neglect, and psychiatric disorders such as depression and schizophrenia.

[0026] The techniques described herein assess and treat such sensory and non-sensory disorders through sensory stimulation. However, the ability to treat such disorders, as well as research areas investigating new interventions and treatment pathways, require reliable and valid measurements of visual processing. It is important for clinicians to understand how patients' sensory processing abilities change as disease progresses or damage recovers, and it is important for researchers to objectively measure how various interventions can affect these changes in sensory function. The neuroscientific principles governing visual and other sensory perceptual processing are dynamic and incompletely understood. In fact, imaging approaches to measure the extent and location of eye or brain damage often correlate poorly with reported visual function. To account for this, gold-standard clinical approaches typically rely on behavioral data and self-reports. For example, common measures of visual function include the use of Snellen eye charts (e.g., charts consisting of rows of progressively smaller letters to assess the smallest print size a patient can reliably report), reading speed assessments, Amsler grids (e.g., patients view a grid pattern and report any distortions or missing areas), and automated perimetry (e.g., patients report whether they can see or cannot see visual stimuli presented in various parts of the visual field). Such tests typically rely on thresholding (e.g., patients are asked whether they can see a stimulus) and therefore may not be sensitive to subtle changes in visual function. Furthermore, behavioral assessments require patients to be able to make behavioral responses, which may not be possible for some patient populations (e.g., infants, children, traumatic brain injury and stroke patients, and patients with intellectual disabilities).

[0027] To address the limitations of these behavioral measurements, neuroimaging tests may be used to assess neurological function. These may involve recording neural responses to visual or other sensory stimuli using electroencephalography (EEG) and / or magnetoencephalography (MEG). EEG is a relatively inexpensive and widely available neuroimaging technique that uses electrodes placed on the scalp to measure electrical activity generated by neurons in the brain's cortex. One set of visual system techniques, sometimes referred to as visual evoked potential (VEP) measurements, measures neural responses to sweeps of simple pattern-reversed grating stimuli across various spatial frequencies. These techniques utilize steady-state visual evoked potentials (SSVEPs) to measure the threshold-maximum granularity of visual information a patient can perceive. SSVEPs are oscillatory neural responses to flashing visual stimuli. When visual stimuli flash at a set frequency, visual cortical neurons involved in processing the stimuli respond at the same frequency, and this response can be detected in the EEG signal. Converting EEG signals from the time domain to the frequency domain using methods such as fast Fourier transform (FFT) or wavelet decomposition allows for objective analysis of the intensity of flicker frequencies in the signal (called SSVEP amplitude). In SSVEP protocols for assessing visual acuity, these SSVEP amplitudes can be used to assess the threshold spatial frequency at which SSVEPs become undetectable. Spatial frequency is the rate at which information changes in space; therefore, the more precise and detailed the visual information (i.e., very small checks in a textured grating), the higher the spatial frequency. In these techniques, SSVEPs can be elicited by inverting the white and black elements of a textured grating at a set rate. Therefore, when the spatial frequency becomes too high to be resolved, the pattern may appear as a uniform gray and no longer elicit an SSVEP. A disadvantage of this method is that, like many behavioral tests, it relies on thresholds and does not allow for the investigation of subtle changes in visual function or response profiles.

[0028] Treating nonsensory brain regions with sensory stimulation can also be challenging because the effects of sensory stimulation on these regions can be difficult to detect. Distinguishing weak signals of such downstream effects can be difficult due to the large amount of noise picked up by detection devices (e.g., EEG or MEG). Determining effective treatments is complicated by the large space of possible inputs (e.g., visual stimulus images), only a small fraction of which may have a therapeutic effect. Additionally, certain diseases may have unique neurological signatures that complicate both determining effective sensory inputs and determining pathology and progression.

[0029] Thus, the present disclosure contemplates pioneering advances in neuroscience, proposing innovative techniques for optimizing the processing of sensory information, refining diagnostic approaches for neurological disorders, and introducing novel methods for controlling brain network activity. Collectively, these innovations offer transformative applications with potential impact on research, clinical diagnosis, and therapeutic intervention in the field of neurotechnology.

[0030] In certain aspects, the collective concepts presented in this disclosure provide concrete and tangible applications in neuroscience and neural engineering. Furthermore, these concepts represent improvements in computer technology by introducing innovative applications at the intersection of neuroscience and computing. For example, closed-loop optimization techniques applied to sensory systems (e.g., techniques in which parameters are iteratively optimized, e.g., by algorithms, until optimal parameters are identified) leverage advanced algorithms and real-time observations using EEG / MEG devices, demonstrating a novel integration of computational methodologies for enhancing sensory information processing. High-fidelity functional mapping of the visual system using EEG and / or MEG demonstrates the application of advanced signal processing techniques and provides a more nuanced understanding of neurological disorders. Furthermore, methods for discovering and constraining sensory inputs to control brain network activity may involve machine learning algorithms and computational modeling, demonstrating a clear improvement in computational tools used in brain research. Furthermore, the concepts described herein involve complex computational processes that rely on the capabilities of external computing systems and dedicated equipment and therefore cannot be implemented within the human mind. While the human mind plays a role in designing, interpreting, and utilizing these methodologies, actually implementing these advanced techniques exceeds the intrinsic computational capabilities of the human brain.

[0031] The subject matter of the present disclosure will now be described more fully hereinafter with reference to the accompanying drawings, which form a part hereof, and which show, by way of illustration, specific exemplary embodiments. Any embodiment or implementation described herein as "exemplary" should not be construed as, for example, preferred or advantageous over other embodiments or implementations, but rather is intended to reflect or indicate that the embodiment(s) are "exemplary" embodiment(s). The subject matter may be embodied in a variety of different forms, and thus, it is intended that the protected or claimed subject matter not be construed as limited to any exemplary embodiments described herein, which exemplary embodiments are provided for illustrative purposes only. Likewise, the scope of the claimed or protected subject matter is intended to be quite broad. Among other things, for example, the subject matter may be embodied as a method, device, component, or system. Thus, embodiments may take the form of, for example, hardware, software, firmware, or any combination thereof. The following detailed description, therefore, is not intended to be construed in a limiting sense.

[0032] Throughout this specification and claims, terms may have subtle meanings that are suggested or implied in context beyond their explicitly stated meaning. Similarly, the phrases "in one embodiment" or "in some embodiments," or "in one aspect" or "in some aspects," as used herein, do not necessarily refer to the same embodiment or aspect, and the phrases "in another embodiment" or "in another aspect," as used herein, do not necessarily refer to different embodiments or aspects. For example, the claimed subject matter is intended to include combinations of the example embodiments, in whole or in part.

[0033] 1A illustrates an exemplary system capable of implementing the methods described herein. The exemplary system 100 includes a data collection component 10, a database 20, and a device data intelligence component 30 operatively connected to one another via a network 40. Alternatively, or in addition, one or more of the components may be connected to another component locally, for example, via a wired connection, without relying on a network connection.

[0034] As disclosed herein, data collection component 10 can include a device or machine capable of measuring electrical activity in the brain. In some embodiments, data collection component 10 can be an EEG machine including or configured to support one or more electrodes, amplifiers, filters, analog-to-digital converters, etc. for conducting EEG testing. In some aspects, data collection component 10 can be a database that receives EEG test data from one or more other sources. In other aspects, data collection component 10 can be any other brain recording device or modality capable of transmitting information regarding neural activity.

[0035] The data acquired by the data collection component 10 may be transferred to the database 20 via a network 40 or a local or network connection. In some embodiments, the collected data may be analyzed by the data intelligence component 30 via the network 40 or a local or network connection. Figure 1B shows exemplary functional modules that may be implemented to perform the tasks of the data intelligence component 30.

[0036] 1B illustrates an exemplary computer system 110 for the techniques discussed herein, e.g., using neural objective functions for closed-loop optimization. The exemplary system 110 accomplishes the techniques described herein by implementing, on one or more computer devices, a user input / output (I / O) module 120, a memory or database 130, a data processing module 140, a data analysis module 150, a classification module 160, a network communication module 170, and any other functional modules (e.g., error correction or compensation modules, data compression modules, etc.) that may be required to perform a particular task. These modules may correspond to the modules in FIG. 1A. For example, database 130 may correspond to database 20, modules 140, 150, 160, and 170 may correspond to data intelligence 30, and aspects of the input of module 120 may correspond to data collection 10. As disclosed herein, user I / O module 120 may further include input sub-modules such as a keyboard, MEG, EEG, eye tracking data, etc., and output sub-modules such as a display (e.g., a printer, television, smartphone, monitor, virtual reality (VR) device, and / or touchpad). In some embodiments, all functions may be performed by one computer system. In some embodiments, functions are performed by multiple computer systems. Various modules (e.g., modules for data processing, analysis, classification, communication, etc.) may be one or more processes running in a distributed computing environment. For example, in some embodiments, one or more components of computer system 110 may be network-accessible via a cloud infrastructure. For example, database 130 used to store data may be stored on one or more remote cloud servers. In this regard, the database may be one or more large storage buckets (e.g., cloud-based storage buckets such as simple storage service "S3" buckets) from which data may be retrieved on demand.As another example, data processing, analysis, and classification may be performed in a cloud-based environment using services such as a cloud-based data processing platform, serverless computing, or a cloud-based machine learning platform.

[0037] It is also disclosed herein that a particular task can be performed by implementing one or more functional modules. Specifically, each of the listed modules itself may consequently include multiple sub-modules that implement one or more techniques described herein. For example, data processing module 140 may include a sub-module for data quality assessment (e.g., a sub-module for performing iterative refinement and validation), a sub-module for normalizing assigned weights to ensure that the weights contribute proportionally to the overall response, a sub-module for performing interpolation or extrapolation, etc.

[0038] In some embodiments, a user can use I / O module 120 to manipulate either data available on the local device or data obtainable over a network connection from a remote service device or another user device. For example, I / O module 120 may allow a user to perform data analysis through a graphical user interface (GUI), e.g., via a keyboard, mouse, or touchpad. In some embodiments, a user may manipulate data through voice control. In some embodiments, user authentication may be required before a user is granted access to requested data. In some embodiments, user I / O module 120 may be used to manage various functional modules. For example, a user can request input data through user I / O module 120 while an existing data processing session is in progress. The user can do so by selecting menu options or individually typing commands without interrupting the existing process. In another example, a user can utilize user I / O module 120 to set various thresholds, configure sample matching settings, and / or provide other instructions to computer system 110 that direct how electrical signals in the brain are captured and / or monitored. As disclosed herein, a user may use any type of input to direct and control data processing and analysis via I / O module 120 .

[0039] In some embodiments, system 110 further comprises memory and / or database 130. In some embodiments, database 130 comprises a local database that can be accessed via user I / O module 120. In some embodiments, database 130 comprises a remote database that can be accessed by user I / O module 120 via a network connection. In some embodiments, database 130 is a local database that stores data obtained from another device (e.g., a user device or a server). In some embodiments, memory or database 130 can store data obtained in real time from an internet search. In some embodiments, database 130 can send data to or receive data from one or more other functional modules, including, but not limited to, a data collection module (not shown), a data processing module 140, a data analysis module 150, a classification module 160, a network communication module 170, etc. In some embodiments, some or all of the real sample data and / or synthetic sample data can be stored in database 130.

[0040] In some embodiments, database 130 may be a database local to other functional modules. In some embodiments, database 130 may be a remote database that may be accessed by other functional modules via a wired or wireless network connection (e.g., via network communication module 170). In some embodiments, database 130 may include a local portion and a remote portion.

[0041] In some embodiments, system 110 includes a data processing module 140. Data processing module 140 may receive real-time data from I / O module 120 or database 130. In some embodiments, data processing module 140 may perform standard data processing algorithms, such as one or more of noise reduction, signal enhancement, normalization, interpolation and / or extrapolation, etc. In some embodiments, data processing module 140 may be configured to process received and / or collected neural activity data associated with one or more subjects. In various embodiments, data processing module 140 may further create a training dataset on which one or more machine learning models (e.g., models for classification, clustering, scoring, etc.) may be trained.

[0042] In some embodiments, system 110 includes a data analysis module 150. In some embodiments, data analysis module 150 includes identifying brain activity patterns associated with particular medical conditions, as described in connection with data processing module 140.

[0043] In some embodiments, system 110 includes classification module 160, which may embody a “machine learning model” or a “trained classifier.” As used herein, a “machine learning model” or a “trained classifier” generally encompasses instructions, data, and / or a model configured to receive an input and apply one or more weights, biases, classifications, or analyses to the input to generate an output. The output may include, for example, a classification of the input, an analysis based on the input, a design, process, prediction, or recommendation associated with the input, or any other suitable type of output. Machine learning models are generally trained using training data, e.g., examples of empirical data and / or input data, that are supplied to the model to establish, adjust, or modify one or more aspects of the model, such as weights, biases, criteria for forming classifications or clusters, etc. Aspects of a machine learning model may operate on inputs linearly, in parallel, via a network (e.g., a neural network), or via any suitable configuration.

[0044] Implementing a machine learning model may include deploying one or more machine learning techniques, such as k-nearest neighbors, linear regression, logistic regression, random forests, gradient boosting machines (GBMs), deep learning, deep neural networks (e.g., recurrent neural networks (RNNs), convolutional neural networks (CNNs)), and / or other suitable machine learning techniques for solving problems in the field of natural language processing (NLP). Supervised, semi-supervised, and / or unsupervised training may be employed. For example, supervised learning may include providing training data and labels corresponding to the training data, e.g., as ground truth. Unsupervised techniques may include clustering or classification, etc. K-means clustering or K-nearest neighbors may also be used, which may be supervised or unsupervised. A combination of K-nearest neighbors and unsupervised cluster techniques may also be used. Any suitable type of training may be used, e.g., stochastic, gradient boosting, random seed, recurrent, epoch-based, or batch-based.

[0045] In an exemplary use case, a machine learning model is trained to analyze test data from a subject whose specific neural activity related to a medical condition may be unknown, and then identify portions or characteristics of the subject's brain that may be the cause of the medical condition or that may be the result of the medical condition. In some embodiments, the one or more parameters may include a score (e.g., a binomial probability score that may be calculated based on logistic regression analysis). As disclosed herein, the binomial probability score may correspond to the likelihood that a subject has a specific medical condition, the likelihood that a portion of the subject's brain is active or inactive, the likelihood that a particular stimulus will affect a desired portion of the brain, etc. For example, a score above a predefined threshold may indicate that a particular stimulus effectively stimulated a non-sensory region of the brain.

[0046] As disclosed herein, the network communication module 170 may be used to facilitate communication between a user device, one or more databases, and any other suitable system or device via a wired or wireless network connection. Any communication protocol / device may be used, including, but not limited to, a modem, an Ethernet connection, a network card (wireless or wired), an infrared communication device, a wireless communication device, and / or a chipset (such as a Bluetooth device, an 802.11 device, a WiFi device, a WiMax device, or a cellular communication facility), near field communication (NFC), Zigbee communication, radio frequency (RF) or radio frequency identification (RFID) communication, a PLC protocol, or 3G / 4G / 5G / LTE-based communication. For example, a user device having a user interface platform for processing / analyzing tumor fraction data may communicate with another user device having the same platform, a regular user device (e.g., a regular smartphone) that does not have the same platform, a remote server, a physical device on a remote IoT local network, a wearable device, a user device communicatively connected to a remote server, etc.

[0047] The techniques disclosed herein can be used in combination with the techniques described in U.S. Pat. No. 10,736,526 and U.S. application Ser. No. 18 / 044,054, which are incorporated by reference herein in their entireties.

[0048] The functional modules described herein are provided as examples. It will be understood that different functional modules can be combined to create different utilities. It will also be understood that additional functional modules or sub-modules may be created to implement a particular utility.

[0049] Neural objective functions for closed-loop optimization of sensory system throughput Neuroscientific research has long sought ways to harness the potential of sensory stimulation to influence brain activity beyond traditional sensory domains, with the goal of uncovering broader cognitive enhancements or therapeutic approaches. The challenge lies in the inherent limitations of traditional methods, which primarily focus on stimulating sensory domains and often struggle to effectively engage nonsensory domains due to erratic, weak, or undetectable responses. More specifically, when attempting direct stimulation in nonsensory domains, the sparse nature of neural responses poses a significant obstacle, as the lack of immediate and informative feedback during the optimization process makes it difficult to identify the effectiveness of stimulation and / or make iterative adjustments toward a desired goal. The limitations of these traditional approaches become apparent when attempting to apply closed-loop feedback to domains where neural activity cannot be immediately or reliably expressed.

[0050] The concepts described herein provide improvements in optimizing sensory stimulation to enhance non-sensory brain activity. By focusing on closed-loop optimization within sensory domains, the system can overcome limitations encountered with conventional approaches. More specifically, the described concepts enable a stepwise and informed approach that ensures stimulation is systematically optimized with informative feedback from sensory domains, for example, through the use of neural objective functions based on sensory cortical function metrics and data throughput metrics, which can guide the process until the desired effect extends to non-sensory domains. For example, closed-loop systems incorporating EEG data and advanced optimization techniques (e.g., Bayesian optimization) have shown significant improvements over conventional methods in stimulating non-sensory brain regions for cognitive enhancement.

[0051] 2, an exemplary workflow 200 for performing closed-loop optimization to enhance non-sensory brain activity is provided. Aspects of the exemplary workflow 200 may be performed in accordance with some or all of the components described in FIGS. 1A and 1B.

[0052] In step 205, the optimization process may begin with the selection of one or more stimulus parameters that describe the characteristics of the sensory stimulus to be presented to the participant. In certain embodiments, stimulus parameters refer to various attributes and characteristics that define the sensory stimulus. These parameters may include, but are not limited to, visual attributes (e.g., color, contrast, motion, duration, etc.), auditory features (e.g., frequency, intensity, duration, etc.), other sensor modalities, and / or any combination of the foregoing. In certain embodiments, the selection may be driven by the overall objective of the closed-loop optimization process. In this context, the objective may be to enhance specific non-sensory brain activity. Thus, predetermined stimulus parameters are selected based on their potential to affect neural responses in both sensory and non-sensory domains.

[0053] In some embodiments, the optimization system may iteratively explore a range of stimulation parameters using techniques such as Bayesian optimization. This exploration may involve selecting a set of parameters, presenting the corresponding stimuli, recording the EEG response, and using the collective data to inform the next set of parameters, as described further herein. Initial parameters may be selected based on patient demographics such as age or gender, previously known generally effective parameters, predetermined parameters that take into account the patient's symptoms and / or medical condition, and / or the patient's initial EEG measurements or brain images. The closed-loop nature of the system allows for adaptive adjustment of stimulation parameters based on real-time feedback. For example, if certain parameters are found to be more effective in eliciting a response, the optimization process can dynamically shift toward exploring similar parameter combinations. Furthermore, rather than being fixed, stimulation parameters may be iteratively refined through multiple optimization cycles, as described further herein.

[0054] In step 210, the selected stimulus parameters may be converted into sensory stimuli using a stimulus generator. This conversion process ensures that the characteristics and features defined by the parameters are accurately represented in the generated stimuli. In certain embodiments, the nature of the sensory stimuli depends on the sensory modality under consideration. For example, if a study focuses on visual stimuli, the generated stimuli may be visual patterns or sequences. Similarly, as another example, in the case of auditory stimuli, the generated stimuli may include specific tones or sound sequences. In certain embodiments, the temporal and spatial aspects of the stimuli may be important. For example, the duration, timing, and / or spatial characteristics (if applicable) may be determined based on the experimental requirements and the sensory system being targeted.

[0055] In some embodiments, a stimulus generator may be used to generate sensory stimuli according to defined parameters. The stimulus generator may be, for example, a computer program or one or more dedicated stimulus generation hardware. The stimulus generator may ensure precision and consistency in presenting stimuli across different iterations of the optimization process. In some embodiments, the generated stimuli may be presented for a predetermined period of time to allow sufficient time for the participant's brain to respond. The stimuli may be repeated or presented in a specific order to capture variability in neural responses. In some situations, participants may be instructed to interact with or respond to the presented stimuli (e.g., in studies exploring cognitive or motor responses in addition to neural activity). In some embodiments, if the optimization algorithm indicates a need for ongoing adjustments, e.g., based on EEG data, the system may dynamically modify the stimulation parameters for the next iteration, as further described herein.

[0056] In step 215, the transformed sensory stimuli may be presented to the subject, and EEG data may be recorded. Alternative techniques may use other brain imaging modalities, such as MEG. EEG is a non-invasive neuroimaging technique that records the brain's electrical activity through electrodes placed on the scalp. It captures the synchronous firing of neurons to provide a real-time measurement of neural activity. Electrodes may be strategically placed on the subject's scalp according to standardized or customized configurations based on the experimental design. In some embodiments, the selection of electrode locations is often determined by the specific brain region under investigation. The EEG system may record electrical potentials generated by the brain over time. The recorded signal represents the sum of the postsynaptic potentials of neurons in the vicinity of each electrode. EEG offers high temporal resolution, allowing for accurate tracking of rapid changes in neural activity, making it particularly well-suited for capturing dynamic responses to sensory stimuli.

[0057] In some embodiments, EEG data recording can be performed in a continuous manner throughout the presentation of a sensory stimulus. This ensures that the entire duration of the stimulus and any post-stimulus effects are captured for analysis. Continuous monitoring of the EEG signal can be important for identifying artifacts that occur during recording. These artifacts can arise from eye movements, muscle activity, external sources, etc., and their presence can affect the accuracy of the recorded neural signals.

[0058] In step 220, data recorded during an EEG session may initially be in raw form (e.g., representing the brain's electrical activity as a function of time), and various techniques can be used to "clean" and preprocess this raw data to improve the quality of the recorded signal, remove artifacts, and prepare the data for subsequent analysis. For example, one or more filtering techniques can be applied to the raw EEG data to isolate specific frequency bands of interest (e.g., by eliminating frequencies that are too low or too high). As another example, various techniques (e.g., independent component analysis (ICA)) can be used to remove artifacts from the EEG data. In yet another example, a baseline correction process can be used to align the EEG data to a common baseline, helping to eliminate baseline drift and ensuring that subsequent analysis focuses on changes in neural activity relative to a stable reference point. Additionally or alternatively, one or more other types of filtering and / or normalization techniques may be used.

[0059] In step 225, a source estimation process may be performed to determine the anatomical locations or brain regions contributing to the recorded EEG signal. This step may help determine the neural basis of the observed response and tailor stimulation to optimally impact the target brain region (e.g., a non-sensory region). Source estimation may be performed utilizing one or more models and / or algorithms, such as Low Resolution Electromagnetic Tomography Analysis (LORETA). In some embodiments, source estimation often results in the identification of voxel locations corresponding to potential neural sources. These voxels are three-dimensional points in brain space where neural activity is determined to occur. In some embodiments, the selected source estimation algorithm, e.g., LORETA, may define voxel activity as a function of EEG channel activity. For example, each voxel may have a different definition function. Finally, instead of per-channel activity, "per-voxel" activity may be obtained.

[0060] In step 230, the measured brain activity may be classified into distinct groups or clusters based on their spatial and / or functional properties. In some embodiments, sensory processing in the brain involves hierarchical stages, particularly in areas specialized for processing specific sensory modalities, such as vision. For example, in the visual system, information travels through different cortical regions, each involved in extracting and processing specific features. Sensory regions such as V1, V2, V4, and MT (temporal lobe region) are identified based on their known roles in the hierarchical processing of sensory information. Each region is associated with a distinct function in processing visual stimuli. As alluded to above, a "voxel" refers to the three-dimensional unit to which brain activity is correlated and measured. In neuroimaging, such as with EEG source estimation (performed in step 225), the brain is divided into these small volumetric units to correlate neural activity patterns.

[0061] In some embodiments, the spatial coordinates of voxels within identified sensory regions are considered. Voxels within a particular region may be grouped based on their proximity to a particular stage in the sensory processing hierarchy and their functional relevance. Grouping may also take into account spatiotemporal connectivity patterns between voxels. Grouped voxels not only share spatial proximity but also exhibit synchronized or functionally related neural activity, reflecting their involvement in the same sensory processing stage. Once voxel groups corresponding to different sensory regions are established, an objective function can be calculated based on local activity across each group, as further described herein. Machine learning or deep learning models can be used to optimize stimulation according to these objective functions.

[0062] In step 235, brain activity measured in brain regions not primarily associated with sensory processing, i.e., non-sensory regions, may be organized and classified into distinct groups or clusters based on spatial and functional properties. In the context of the present application, non-sensory regions correspond to brain regions that are not primarily responsible for processing sensory input, but may have downstream or higher-order functions. Examples include the frontal cortex, centrotemporal region, motor cortex, posterior parietal lobe, etc.

[0063] The spatial coordinates of voxels within identified non-sensory regions may be grouped together based on the spatial proximity and functional relevance of the voxels within that particular region. In some embodiments, the grouping may also take into account the spatiotemporal connectivity patterns between voxels. This ensures that grouped voxels not only share spatial proximity but also exhibit functionally related neural activity, reflecting their involvement in the same non-sensory processing region. The grouping of voxels in non-sensory regions complements the previous grouping in sensory regions. The goal is to understand and optimize neural activity in these non-sensory brain regions with the overall goal of stimulating these regions through sensory input.

[0064] In step 240, an objective function may be calculated based on local neural activity within grouped voxel regions associated with sensory processing. In some embodiments, the objective function provides a quantitative measure of neural response in these sensory regions, thereby guiding the optimization process. In some embodiments, the objective calculation may begin by measuring local neural activity within each voxel group. This may include evaluating the strength, pattern, or characteristics of neural response in specific sensory regions. In some embodiments, an example metric used in the objective calculation may be the average power of neural activity within each voxel group in a non-sensory region. This may involve quantifying the amplitude or strength of neural oscillations in a given frequency band, thereby providing a measure of overall activity. In some embodiments, another example metric that may be used may be the gradient of power across sensory regions. This metric evaluates how neural activity output changes when moving from one sensory region to another, thereby capturing the dynamics and progression of neural responses in hierarchical sensory processing pathways. Other metrics not explicitly listed and / or described herein, such as the degree of phase locking across different regions, may also be described and / or utilized.

[0065] In step 245, an objective function may be calculated based on local neural activity within grouped voxel regions associated with non-sensory brain regions. The objective function provides a quantitative measure of neural response in these non-sensory regions to further guide the optimization process. Similar to step 240, the objective calculation may begin by measuring local neural activity within each voxel group associated with a non-sensory region, which may include evaluating the strength, pattern, or characteristics of neural response in the non-sensory region. As previously described, metrics such as mean power of neural activity, power slope, and / or one or more other metrics may be utilized. Additional metrics may also be used, such as identifying peaks in the frequency spectrum or detecting phase-locked oscillations in frequency bands across two different non-sensory regions (e.g., phase-locking refers to a fixed time relationship between two waveforms, particularly in the case of sinusoidal components of a waveform, which have a fixed phase relationship between the two waveforms).

[0066] At step 250, a decision-making process can be used to determine whether the calculated objectives for non-sensory regions reach a predefined minimum threshold. Prior to investigating this decision point, objective functions are calculated based on local neural activity within grouped voxel regions associated with non-sensory regions of the brain. These objectives represent quantitative measures of neural response in non-sensory regions. In some embodiments, a predefined minimum value can be set for the objectives of non-sensory regions. This minimum value can serve as a threshold that the calculated objectives must meet and / or exceed in order for the optimization process to consider the non-sensory objectives satisfactory.

[0067] In some aspects, a comparison of the calculated objectives in non-sensory regions with a predefined minimum value may be performed. In response to determining in step 250 that the calculated objectives in non-sensory regions have not reached the predefined minimum value, the optimization process may then conclude that the desired level of neural activity in these regions has not yet been achieved. In such a situation, embodiments may no longer consider the non-sensory objectives for further analysis in the current optimization iteration in step 255. The optimization process continues using only the sensory objectives for the next iteration. This decision acknowledges that the non-sensory regions may not yet exhibit the desired neural response. Conversely, in step 250, it may be determined that the calculated objectives in non-sensory regions have reached the predefined minimum value, and in response to determining that the calculated objectives in non-sensory regions have reached or exceeded the predefined minimum value, the optimization process may conclude that a satisfactory level of neural activity has been achieved in these regions. In such situations, embodiments may utilize the amount of non-sensory objective exceeding the minimum (i.e., the surplus achieved above a predefined minimum) and average it with the sensory objective calculated in the same iteration at step 260. The averaged objective serves as guidance for adjusting stimulation parameters in the next iteration of the optimization process. The goal may be to refine the stimulation to further enhance overall neural activity in both sensory and / or non-sensory regions. More specifically, the optimization process provides the advantage of considering both sensory and non-sensory contributions, which may potentially result in more refined and effective adjustment of stimulation parameters over successive iterations.

[0068] In step 265, an optimal stimulus may be selected, and the optimization process may terminate. In some embodiments, the optimization process may include multiple iterations in which stimulation parameters are adjusted based on calculated objectives related to neural responses in both sensory and non-sensory regions. The specific number of iterations may be manually defined or may be dynamically determined, e.g., by the system. Progress toward the calculated objectives is continuously monitored throughout this closed-loop optimization process. In some embodiments, the optimization process may include criteria for checking for objective plateaus. At this point, the system may be configured to determine whether the objectives have reached a plateau or no longer show significant improvement over a specified number of consecutive iterations. If no improvement is detected, the system may conclude that further adjustment of the stimulation parameters is unlikely to yield substantial benefit. Such a determination may trigger the optimization process to stop, and the stimulus and / or stimulation parameters that best achieved the objectives throughout the iterations may then be selected. This stimulation may be deemed optimal because it evoked the most favorable neural response according to the defined objectives.

[0069] Methods for diagnosing neurological disorders using visual system function-based diagnosis Certain neurological disorders, even those that interact with the sensory system, can present particular challenges in diagnosis or treatment. Disorders such as visual neglect or autism often result from stroke or developmental factors and can manifest as complex, heterogeneous conditions requiring an accurate and nuanced diagnostic approach. Traditional diagnostic tests for these disorders are basic and provide limited insight into the complexity of an individual's condition. For example, standard diagnostic tools for visual neglect often include bedside tests such as line bisection, line erasure, and drawing tasks. While these tests are rapid and commonly used, they are static, rely on heuristically designed stimuli, and rely primarily on behavioral responses. This reliance on behavioral observation can introduce errors and limit diagnostic accuracy. Furthermore, these traditional methods are limited in scope and sensitivity, potentially missing milder disorders or previously undescribed variants of the disorder. Traditional diagnostic approaches for autism and other neurological disorders affecting visual processing suffer from similar limitations.

[0070] Shortcomings of traditional diagnostic approaches generally stem from their reliance on static stimuli and behavioral responses. These methods often lack the ability to explore a broad stimulus space, including dynamic stimuli, and comprehensively measure neural responses. This results in poor diagnostic accuracy and an incomplete understanding of the unique characteristics of each individual's disorder.

[0071] The concepts described herein improve upon traditional diagnostics for neurological disorders affecting the visual system. The disclosed methods can identify the complexity of an individual's visual system function by introducing high-fidelity functional mapping through the creation of specialized maps using EEG and / or MEG. Such maps can be relatively high-dimensional (e.g., greater than 100 dimensions) and can be projected into low-dimensional (e.g., less than 10 dimensions) metrics, which can then be associated with known diagnoses. The resulting classification, clustering, and scoring models improve diagnostic sensitivity and provide a dynamic and comprehensive understanding of visual system-related disorders.

[0072] 3, an exemplary workflow 300 for diagnosing neurological disorders with visual system function-based diagnosis is provided. Aspects of the exemplary workflow 300 may be performed according to some or all of the components described in FIGS. 1A and 1B.

[0073] In step 305, a specialized map may be constructed for each individual subject. In some embodiments, this construction process may begin with the acquisition of neural data via EEG and / or MEG techniques. These non-invasive techniques capture electrical and magnetic activity occurring in the brain, respectively. As previously discussed above, EEG records electrical activity using electrodes placed on the scalp, while MEG measures magnetic fields generated by neural activity. Each technique may determine neural data generated by the occipital lobe, the region responsible for visual processing. The collected neural data may then be processed to create a detailed and comprehensive functional map, which may effectively represent how an individual's visual system encodes dynamic information. Specifically, the constructed map may represent coordinated dynamic neural responses across various subregions of the occipital lobe or combinations of other regions, highlighting processing circuits that enhance or reduce activity during associated visual processing. For simplicity, this constructed map is referred to herein as a "Gemini" map. It is important to note that the name Gemini is for designation purposes only and is not intended to imply any functional limitations or other inferences associated with this constructed map. In certain aspects, the construction of a Gemini map, or "stimulus map," can be facilitated as described below.

[0074] In one aspect, the disclosed embodiment provides a method for providing spatiotemporal sensory input to one or more participants to generate a brain stimulation map. The method includes sampling a spatiotemporal sensory code generation model with a first encoded vector to generate a first spatiotemporal sensory code in the form of a first video sequence. The method further includes outputting the first video sequence to provide the first spatiotemporal sensory input to the one or more participants. The method further includes receiving one or more neural response measurements for the one or more participants, the one or more neural response measurements being performed while the first spatiotemporal sensory input is presented to each of the one or more participants. The method further includes determining a result function based at least in part on the one or more neural response measurements for the one or more participants. The method further includes generating a second encoded vector based at least in part on the first encoded vector and the result function. The method further includes iteratively repeating the sampling, outputting, receiving, and determining with the second encoded vector and any successive encoded vectors until a defined stopping criterion set for the result function is met. Once a set of stopping criteria defined for the outcome function is met, the resulting spatiotemporal sensory code is stored and forms part of the brain's stimulation map.

[0075] Embodiments may include one or more of the following features, either separately or in any feasible combination.

[0076] The spatiotemporal sensory code may include one or more of the following: visual sensory input, auditory sensory input, and somatosensory input. The generative model may include procedural graphics using input parameters including one or more of spatial frequency, temporal frequency, spatial location, spatial extent, and translation-based motion vectors. The spatiotemporal sensory code generative model may include a generative adversarial network or a deep diffusion model, and the first encoding vector may point to a location within the latent generative space.

[0077] The spatiotemporal sensory codes in the form of video sequences have a defined time length and are partially overlapping in time. The first video sequence begins at time T i The method may further comprise applying a frame-by-frame window function to the first video sequence and adding the results to an output frame buffer to generate a frame buffer of N frames starting from T i From T i +N. Successive encoded vectors may be generated based at least in part on the result function and a number of previous encoded vectors.

[0078] The generation of the second encoding vector is performed at time T i +S, where S<=N, and the method further comprises applying a frame-by-frame window function to the second video sequence and adding the result to an output frame buffer to generate a frame T i ~T i and obtaining an output frame buffer containing T+S+N. During the output, while the second video sequence is being generated, i From T i Up to +S frames may be output from the output frame buffer and presented to one or more of the participants.

[0079] Outputting may include displaying the series of spatiotemporal sensory inputs on one or more electronic screens. The one or more neural response measurements may be performed using one or more of the following: EEG, quantitative EEG, MEG, single photon emission computed tomography (SPECT), positron emission tomography (PET), functional magnetic resonance imaging (fMRI), and functional near-infrared spectroscopy (fNIRS). The one or more neural response measurements may be received from a multi-channel buffer containing current and previous multi-channel neural response measurements.

[0080] The method may further include time-aligning the one or more neural response measurements across the one or more participants, extracting one or more features for each measurement time step across the one or more neural response measurements and the one or more participants, and comparing the extracted one or more features to a target to calculate a result function. The set of defined stopping criteria may include one or more of the following: a specified convergence criterion, a specified number of iterations, and a specified time.

[0081] In storing the resulting spatiotemporal sensory code to form part of a brain stimulation map, the feature representation of the one or more neural response measurements can be associated with a location in high-dimensional space. The resulting spatiotemporal sensory code can be associated with a neural state at a specific brain location. The resulting spatiotemporal sensory code can be associated with a whole-brain neural state. A whole-brain neural state can be defined in terms of multivariate cross-coherence across spectral bands, and the resulting spatiotemporal sensory code can be adapted to maximize cross-coherence across one or more pairs of nodes in the brain map.

[0082] In another aspect, the disclosed embodiments provide a system for providing spatiotemporal sensory input to one or more participants to generate a brain stimulation map, the system including at least one processor and at least one non-transitory processor-readable medium storing processor-executable instructions that, when executed by the at least one processor, cause the at least one processor to perform the above-described method.

[0083] In step 310, the Gemini map may be compared to one or more reference maps derived from healthy individuals, which may represent typical neural activity within the visual system during visual processing. The maps may also include those derived from individuals with known medical conditions. In some embodiments, this comparison may be performed in a multidimensional map space, taking into account various dimensions that capture the complexity of neural responses in the occipital lobe. This multidimensional approach allows for the examination of subtle differences, as opposed to simple comparisons based on a single metric. In some embodiments, a difference calculation method is employed in which the Gemini map constructed in step 305 is subtracted from the synthesized healthy map, and the magnitude of the difference may be subjected to a threshold that may depend on the variance of random spontaneous activity at that map location and / or the variance of responses to identical stimulus conditions. This calculation may result in a map that highlights the magnitude and sign of specific differences (e.g., areas of divergence or convergence) between the individual's neural activity and the synthesized healthy map.

[0084] In step 315, the map or observed differences may be projected into a set of low-dimensional metrics. In certain aspects, these metrics may serve as a condensed representation of various aspects of neural activity divergence or convergence and are designed to summarize specific features related to visual system function, thereby providing a more interpretable and manageable set of parameters for further analysis. In step 315, we describe two exemplary methods of how this projection process may be facilitated.

[0085] A first exemplary method of projecting differences or similarities, e.g., "design projection," may be achieved through expert input. Specifically, by capturing important aspects of neural processing, such as processing speed and data throughput, a more focused analysis of a subject's visual system abnormalities may be achieved.

[0086] The first substep captures temporal aspects of neural processing, particularly those related to the speed of information processing within the visual system. Specific portions of the Gemini map that reflect response latency and / or oscillation frequency can be selected by an expert. By applying a first weighted average to these selected portions, the resulting metric represents the speed at which the subject's visual system processes information.

[0087] In a second substep, the efficiency and information flow within the visual system can be evaluated. In this regard, a second weighted average metric can be designed by an expert by evaluating a portion of the Gemini map that reflects cross-spectral power across known processing hierarchies, including the visual system and cognitive processing. This metric provides insight into data throughput, or the ability of the visual system to transmit and process information.

[0088] A second exemplary method of projecting a map or difference, e.g., "learned projection," represents a data-driven approach in which a projection is learned based on different strategies for evaluating low-dimensional metrics of subjects. This method can be particularly useful when the available subject database is sufficiently large (e.g., more than 100 subjects). In some embodiments, subjects may undergo behavioral tests to assess the speed of visual processing. The goal is to optimize the map projection across these subjects to predict the speed of processing scores. This optimization process involves adjusting the map projection so that the projected map, or the projected difference observed between the Gemini map and the healthy map, is consistent with the behavioral metric. In some embodiments, constrained optimization techniques can be applied to ensure that the projection complies with certain criteria.

[0089] Additionally or alternatively, the Gemini Map can be provided to a machine learning model, such as a random forest or multi-layer perceptual model (MLP). The model can be trained on the Gemini Map features or map variances to learn the relationship between the constructed map features and low-dimensional metrics. This trained model can then be used to predict a new individual's score based on their own constructed map.

[0090] In step 320, the high-dimensional map and low-dimensional metrics can be correlated to the subject's known diagnosis. These correlations can not only help confirm or refine existing diagnoses, but also serve as a basis for extending diagnostic capabilities to new individuals with potentially different or lesser-known conditions. In one aspect, the high-dimensional distance map created by comparing the subject's constructed Gemini map to a healthy composite map, as described in step 310, provides a spatial representation of deviations or similarities in neural activity. This map, combined with the low-dimensional metrics derived in step 315, serves as a comprehensive characterization of the subject's visual system function.

[0091] Next, in step 325, one or more machine learning models can be trained, for example using one or more supervised learning approaches, to classify, cluster, and score new individuals based on their association with a dataset of individuals with known diagnoses, as described further below.

[0092] In some embodiments, a dataset containing information about individuals with known diagnoses may be used to train a classification model. The dataset may include individuals with various neurological disorders affecting the visual system, with each neurological disorder associated with the individual's high-dimensional distance map and low-dimensional metrics. In some embodiments, feature extraction may be performed when both the constructed Gemini map and the low-dimensional metrics are utilized in the classification process. In this regard, the Gemini map may be downsampled to have, for example, 10,000 or fewer data points, and the map may be averaged across each axis (e.g., for a 3D map, the map may be averaged across the x-, y-, and z-axes). In some embodiments, classification models such as random forest models may be used for this task. These models are well-suited for scenarios with a small number of training samples (e.g., 100-1000), making them suitable for neurological disorders where limited data may be available. In some embodiments, the classification model may be trained on the dataset and learn to recognize patterns and associations between input features and known diagnoses. The model may then be validated using another dataset to ensure its generalizability.

[0093] In some aspects, once the model has been trained and validated, it may be applied to new individuals. The Gemini map and each individual's low-dimensional metrics may be input into the model, which can predict the most likely diagnostic category based on patterns learned from the training dataset. The output of the classification process provides a diagnostic category for the individual, which indicates the neurological disorder likely affecting the individual's visual system. This information can help clinicians and medical professionals make informed decisions regarding treatment plans, interventions, and ongoing care.

[0094] In some embodiments, individuals who share similar score profiles can be clustered together into groups.Clustering can help identify patterns and similarities in visual system function that may not fit into predefined diagnostic categories, and may reveal new or undefined diagnoses.For this purpose, standard clustering algorithms such as K-means clustering can be used.

[0095] In some embodiments, individuals may be scored based on the severity of their diagnosed neurological disorder. Similar to the classification and clustering steps described above, features can be extracted from the constructed map and the low-dimensional metric. Downsampled maps and averaged metrics across specific axes may be used as features. Unlike classification, which assigns individuals to discrete categories, scoring may use a regression model. For example, a regression model, such as a linear regression or other regression algorithm adapted to the task, may be trained to predict a continuous severity score based on the extracted features. Once trained, the regression model can be applied to new individuals. The features extracted from the individual's constructed map and the metrics from the individual-specific low-dimensional metric may be input into the model, which may predict the individual's severity score. This score quantifies the severity of the individual's diagnosed neurological disorder.

[0096] Using spontaneous brain activity to constrain sensory inputs used to control brain activity EEG and MEG are techniques that can provide valuable insights into the dynamic interplay of neural processes in the brain, providing a window into the complex world of cognitive function and perception. However, significant challenges remain in identifying and manipulating specific sensory inputs that can effectively control brain network activity. In nonlinear systems and networks, the search for such inputs becomes particularly challenging, often requiring an exhaustive search of a vast space of possible inputs.

[0097] Traditional approaches to this problem involve conducting experiments involving human participants, exposing them to various sensory stimuli, and monitoring the resulting brain activity using EEG or MEG. While informative, this process is resource-intensive and lacks efficiency. Furthermore, in linear systems, the superposition property allows responses to individual stimuli to be combined to predict responses to combinations of stimuli. However, these properties may be weak or absent in nonlinear brain networks, complicating the identification of effective stimulus combinations. Furthermore, traditional attempts to find dynamic similarities between brain activity and sensory input often rely on simple metrics such as the autocovariance matrix. These methods may miss subtle patterns and relationships in the data.

[0098] The techniques described herein reduce search time and enhance the organization of the search space. First, the use of dynamic similarity metrics, such as advanced statistical methods and predictor-based approaches, provides a more nuanced understanding of the relationship between sensory input and brain activity. This leads to a more targeted and efficient search process. Second, the described concepts focus on organizing the search space to increase smoothness, recognizing that adjacent points in space are more likely to exhibit similar control properties. By learning projections that place similar dynamics closer to each other and leveraging transformations of brain activity, certain embodiments introduce a structured and systematic way to navigate the input space.

[0099] 4 and 5, exemplary workflows 400 and 500 are provided for using spontaneous brain activity to constrain sensory inputs used to control brain activity. Specifically, exemplary workflows 400 and 500 may be performed according to some or all of the components described in FIGS. 1A and 1B.

[0100] Related terms associated with this section may include the terms explained below.

[0101] A "large space of possible inputs" may correspond to a single sensory input, which may be uniquely characterized by a list of unique values ​​used to characterize or create the input. This represents a point in the space of possible inputs. The list of unique values ​​may be parameters used in an algorithm that utilizes a parametric function (such as a Gaussian or Gabor function). An input may have dynamic properties, such as spanning a period of time, such as a video clip, an audio clip, or a somatosensory spatiotemporal stimulus clip.

[0102] "Brain activity" can be multi-electrode / channel EEG or MEG (or other brain recording modalities with a time component that conveys important information about neural activity). Activity can have a temporal component, typically a time series sampled periodically at a fixed rate, e.g., 1 kHz. Activity can also have a spatial component, either in "signal space," where electrodes represent points in space, or in "source space," where a source estimation model such as low-resolution electromagnetic tomography analysis (LORETA) is applied to map a collection of channel time series to a "voxel" time series, where each voxel is a 3D spatial point in the brain. Spatiotemporal activity can be mapped to designed or learned basis functions, representing a library of potentially complex spatiotemporal components. These can be learned with principal component analysis (PCA), independent component analysis (ICA), or deep representation learning, or (for example) wavelet or sinusoidal Fourier components. The output of the basis function projections can be real- or complex-valued. Activity can be a function of multiple channel activity over a period of time, e.g., a graph describing pairwise cross-correlations between channels in various basis function bands. Subsets of the graphs in a subset of basis functions can be thought of as independent channels of activity. Such channels may be defined or learned to minimize similarity between channels under certain constraints, e.g., similarity is measured only when certain conditions are met, such as when a particular EEG microstate is active or other independently defined states based on a task, behavioral observation, or brain activity metrics. An instance of brain activity can be defined in terms of a period of time. This period may not be fixed. Two periods may have different durations, each resulting in an independent observation of brain activity. The period can be based on when certain conditions are met, such as when a particular EEG microstate is active or other independently defined states based on a task, behavioral observation, or brain activity metrics. The methods herein can be applied to such non-fixed durations.

[0103] "Control of brain activity" may correspond to properties of brain activity that can be described as a function of sensory input. These can be characterized by vectors.

[0104] "Similar properties of control" may correspond to saying that if the control of brain activity can be treated as an N-dimensional vector, two sensory inputs have similar control properties if the distance between the two vectors is closer than the distance between them to most other sensory inputs, or if the proximity between the vectors is small compared to the vector space spanning all sensory inputs.

[0105] "Spontaneous brain activity" may correspond to brain activity recordings of unconstrained viewing / listening / action. It may be obtained from a repository containing such recordings from a large number of healthy people. It should not be obtained from specialized, challenging, or concentrated activities or from people with brain disorders.

[0106] In certain aspects, two independent processes are described herein that may make the search for relevant input stimuli more achievable.

[0107] The first process may involve reducing the average time required to search for sensory inputs that control brain activity by more highly weighting regions of the input space that are much more likely to produce results. If some of the weights are zero, the size of the search space may be strictly reduced; otherwise, the size of the search space is effectively reduced. This can be done in a variety of ways. For example, in one embodiment, the weights may be derived by analyzing spontaneous brain activity and assigning higher weights to portions of the input space that have dynamics that are more similar to those of brain activity.

[0108] Statistical methods for identifying dynamic similarities between a given input space and brain activity are described herein. Dynamics (e.g., time series statistics) can be characterized in various known ways, and similarities between dynamic or stochastic processes can be characterized in various known ways. One simple method is the autocovariance matrix of "brain activity" (see definition—which may be in signal space, source space, and after basis function projection), which can be matched to the autocovariance matrix of each point in the input space using a Euclidean distance metric. (Similarly, the input autocovariance may be multi-channel, where the channels are the spatial location and / or activations of various basis functions after projection, including random permutation projections.) Another technique may compare eigenvalue decompositions. Additionally or alternatively, methods for comparing higher-order statistical properties of nonlinear dynamics may be more effective in some situations. Examples include comparisons between Markov chain models of each process, or comparisons based on information theory.

[0109] Also described herein is a predictor-based method for identifying dynamic similarities between a given input space and brain activity. In this method, weights can be the degree to which one process or the statistics of one process can be predicted from other processes under certain constraints of the predictive architecture. This can be done, for example, by training a small recurrent neural network (RNN) (one of many example model architectures that can predict and generate time series) to predict processes in the input space, measuring the predictive strength of the same RNN to predict spontaneous brain activity, and retraining only the output fully connected layer for a small number of fixed learning iterations.

[0110] In one embodiment, the input space can be learned by the joint objective of dynamic similarity to brain activity and desired sensory input statistics. This concept can be generalized to cases where an input space is learned that is a substantially reduced space for controlling brain activity. In this approach, an RNN (also one example of many model architectures that can generate time series) can start with random weights, and these weights can be iteratively adjusted (e.g., using stochastic gradient descent (SGD)) to maximize the dynamic similarity between the generated process and spontaneous brain activity. By repeatedly starting with various random RNN initializations, the width of the input space can be ensured, and each point in the input space can be characterized by the converged or initialized RNN coefficient values ​​or some function thereof (e.g., a low-dimensional projection by principal component analysis (PCA)). The generator can include projections by several learned or designed basis functions. These may be trained to generate inputs that match desired statistical properties of the input, e.g., spatial frequency distribution, spatial cross-covariance, activation of computational models of visual cortex, etc., and "space" (which may apply to visual, auditory / tonal, and somatosensory spaces) may be extended to space and other projections (spatial-spatial frequency, spatial-spatial basis functions / wavelets, etc.).

[0111] In some embodiments, an input space can be learned based on a dynamics-preserving transformation of brain activity that optimizes desired sensory input statistics. The input space can be learned by learning a transformation of brain activity. In this method, spontaneous brain activity, or the output of a model trained to generate spontaneous brain activity, can be passed through a linear (e.g., random permutation matrix) or nonlinear (e.g., a randomly initialized deep neural network) random static (operation is applied only to the non-temporal axis) transformation that maps the brain activity channel / basis function domain to the input channel (e.g., spatial) / basis function domain. Basis functions on the input domain projection can be designed or learned to match the desired statistical properties of the input, such as spatial frequency distribution, spatial cross-covariance, or activation of a computational model of the visual cortex. "Space" (which can apply to visual, auditory / tonal, and somatosensory spaces) can be extended to spatial and other projections (e.g., spatial-to-spatial frequency, spatial-to-spatial basis functions / wavelets, etc.). The input space points can be characterized by the coefficients of random projections, or the activations of designed or learned basis functions, or some function thereof (e.g., low-dimensional projections via PCA).

[0112] In some embodiments, weights can be updated based on control experiments. Weightings of such sensory input spaces based on spontaneous brain activity can be updated according to the observed degree of control of brain activity established between that point in the input space and the brain activity. Finally, weights can be learned based on the degree of control and multiple observations of input features describing the input space and spontaneous brain activity, for example, using an ML model trained to perform regression using the observed weights as targets.

[0113] In an exemplary process flow related to the foregoing, spontaneous brain activity may be analyzed at step 405. In some embodiments, data from spontaneous brain activity recorded by, for example, EEG or MEG, obtained from one or more data locations may be utilized. The brain activity data may be cleaned and pre-processed to isolate relevant patterns and features.

[0114] The dynamics of brain activity may be characterized at step 410. In certain embodiments, various time series statistical methods may be used to characterize the dynamics of spontaneous brain activity, which may include measures such as autocovariance matrices, eigenvalue decomposition, and / or higher order statistical properties.

[0115] In step 415, weights may be assigned to regions of the input space. In one embodiment, for dynamic similarity weighting, the weights may be derived by analyzing spontaneous brain activity and assigning higher weights to regions in the input space that have dynamics more similar to those of the brain activity. In another embodiment, for Euclidean distance metric weighting, a Euclidean distance metric may be used to match the autocovariance matrix of brain activity to each point in the input space, emphasizing regions with more dynamic similarity.

[0116] In step 420, the search space may be reduced. More specifically, depending on whether some weights are zero or non-zero, the size of the search space may be strictly reduced (if some weights are zero) or effectively reduced (if all weights are non-zero). This reduction streamlines the search process.

[0117] The second process may involve organizing the search space to increase smoothness. In the context of this application, "smoothness" may indicate that the values ​​of adjacent points tend to be close to each other, e.g., the local derivative across the direction between the points is small or within a predetermined threshold. Furthermore, "value" may refer to some measure of the resulting response, such as amplitude, frequency, and / or phase. More specifically, in some aspects, a smooth space may indicate that adjacent points in the space are likely to have similar control properties, and that as you move in a direction between two adjacent points in the space, the change in the control property is more likely to predict the change between the second and third points, which are the next steps in the same direction. This may be completed in a variety of ways, including creating a smooth input space or increasing the smoothness of the input space, for the applications described herein.

[0118] In some aspects, projections can be learned that place input space points closer to each other if their dynamics are more similar (e.g., using a dynamics similarity measure described herein). The projections can maintain or reduce the original spatial dimension. In some aspects, this method can be extended by iteratively updating the projections based on observations of pairwise distances between corresponding brain-controlled properties. The warping itself may have a smoothness constraint that attempts to maintain the distances in the original projection. That is, for points that were originally very close to move farther apart, the control properties must be repeatedly observed with high certainty to be very different.

[0119] In an exemplary process flow related to the foregoing, in step 505, a projection can be learned to increase the smoothness of the input space. In one embodiment, a projection method can be defined that organizes input space points so that those with higher dynamic similarity (e.g., similar brain activity control properties) are closer to each other. In one embodiment, it can be determined whether the projection maintains the original spatial dimension or reduces it. This determination may depend on the particular characteristics of the input space and the desired result.

[0120] At step 510, the projections may be iteratively updated. In some embodiments, observations of pairwise distances between corresponding brain-controlled properties for different input space points may be collected. The projections may be iteratively updated based on these observations to ensure that similar brain-controlled properties remain close in the projection space. In some embodiments, certain smoothness constraints may be implemented to minimize distortion in the input space.

[0121] Neural Signal-to-Noise Ratio (SNR) Maximization for Rapid Closed-Loop Optimization Using Short Analysis Windows and Hyperscanning Reliable interpretation of neural signals related to brain function is crucial for understanding cognitive processes, identifying biomarkers, and developing effective interventions for neurological disorders. However, accurate extraction of neural information faces significant challenges due to the presence of noise and limitations of traditional observational techniques. Traditional methods such as EEG and MEG are fundamental tools for studying neural activity. However, these methods often generate observations that contain both signal and noise, making it difficult to identify relevant neural information. SNR serves as an important metric, quantifying the extent to which the signal of interest is obscured by unwanted noise. In search of meaningful insights, one technique employs long time windows and multiple trials, which results in long experimental periods and potentially leads to oversights in capturing dynamic natural events.

[0122] To achieve a sufficiently large SNR to infer functionality, techniques can use "biomarkers" (such as "alpha frequency intensity") defined over a long time window or average activity over repeated trials of the same reference state. The reference state can be induced by a specific sensory input, or a motor task, or a cognitive task. Often, both a long analysis time window and multiple-trial averaging are used in combination to find several biomarkers with a sufficient SNR to infer functionality or changes in functionality.

[0123] However, such practices can limit the degree of function that can be observed. In behavioral settings that continuously evolve naturally, the brain continuously and rapidly adjusts neural assemblies that dynamically connect different neural populations throughout the brain. Observing the functionality of such network behavior requires short time windows. In such naturalistic settings, the fine temporal structure of neural signals may contain much more information than the amount of information contained in a single oscillatory frequency or the scale of overall activity observed over a longer time window. Another drawback of multi-trial averaging is that the strongest and most detailed aspects of neural signals are often present upon the first presentation of a sensory input, whereas subsequent presentations of the same input weaken or completely eliminate neural activity.

[0124] Additionally, the use of long time windows and multiple-trial averaging slows scientific progress to an impractical level. If a biomarker requires collecting activations over a few seconds and averaging them over dozens of trials, the required time is two to three orders of magnitude slower than if activity patterns can be observed over time windows of tens to hundreds of milliseconds with continuously evolving (non-repeated) input. In closed-loop optimization neurofeedback applications, such practices can prevent convergence or at least significantly delay convergence to a suboptimal state. If such neurofeedback involves achieving effects that generalize to the entire population, the situation becomes even more time- and effort-intensive, orders of magnitude more. In essence, most of the possible benefits of closed-loop neurofeedback systems cannot be achieved in practice.

[0125] The concepts described herein combine hyperscanning, short-time windows, and / or alignment functions to provide a comprehensive solution for maximizing the SNR of neural activity, enabling more accurate and rapid closed-loop optimization in neuroscience research and applications. In the context of this application, hyperscanning may refer to the neuroimaging of multiple people substantially simultaneously, for example, in response to the same synchronous stimulus. More specifically, hyperscanning techniques can be incorporated to integrate neural signals from multiple subjects simultaneously exposed to the same stimulus. This technique utilizes short time windows (e.g., 10-100 ms) to observe neural events, thereby capturing rapid adjustments in neural assemblies. Furthermore, spatial alignment functions can be utilized to optimize the weights of each channel, ensuring optimal integration of neural signals across subjects. The alignment functions effectively establish temporal alignment using per-channel delays or dynamic time warping to compensate for individual differences in brain structure and electrode location.

[0126] 6-8 collectively, exemplary workflows 600, 700, and 800 are provided for utilizing short time analysis windows and hyperscanning for neural SNR maximization for closed-loop optimization. Aspects of exemplary workflows 600, 700, and 800 may be performed in accordance with some or all of the components described in FIGS. 1A and 1B.

[0127] Below is provided a series of steps that can be followed to design a spatial alignment function.

[0128] In step 605, one or more reference subjects may be selected from the pool of N subjects to serve as a baseline against which the neural signals of other subjects can be compared and adjusted. In certain embodiments, the reference subjects may represent individuals whose neural signals will be used as a reference for alignment. In certain embodiments, factors such as age, sex, health or disease state, anatomical similarities, and / or overall neurological characteristics may be considered to minimize bias in the alignment process. In general, an ideal reference subject may be one whose neural responses are consistent and expected to be representative of a particular group, thereby enhancing the effectiveness of the alignment process.

[0129] In step 610, the compound stimuli may be presented to all subjects (excluding the reference subject), and the subjects' EEG may be recorded. In certain embodiments, "complex stimuli" may refer to rich and engaging sensory inputs that are often used to elicit diverse neural responses. This may include naturalistic videos, audiovisual sequences, or other stimuli tailored to the research objectives. In certain embodiments, the selection of stimuli may be dictated by the specific goals of the experiment (e.g., studying cognitive processes, perception, motor tasks, etc.). In certain embodiments, the stimuli may be presented simultaneously to all subjects to create a synchronized experimental environment. This may ensure that all participants are exposed to the same sensory input at the same time. In certain embodiments, the experimental conditions, e.g., lighting, sound, and any other relevant environmental factors, may be controlled to minimize extraneous variables that may affect neural responses. In certain embodiments, EEG recording equipment may be used to capture the brain's electrical activity during the presentation of the stimuli. Once the setup is complete and relevant parameters (e.g., sampling rate, filter settings, etc.) are established, EEG data may be recorded. In certain embodiments, EEG data may be recorded for the entire duration of stimulus presentation. Alternatively, in other embodiments, recording may be performed over a shorter period of time, for example, based on the design of the experiment.

[0130] In step 615, a neighborhood weighting function may be optimized for each channel to ensure that the cross-correlation between the individual target channel and the corresponding channel of the reference target is maximized. In some embodiments, the neighborhood weighting function may assign a weight to each adjacent channel of a given channel. These weights determine the contribution of each adjacent channel to the cross-correlation with the corresponding channel of the reference target. In some embodiments, the goal may be to maximize the height of the cross-correlation peak between the channel and the corresponding channel of the reference target. The weights may be adjusted during the optimization process to improve the accuracy of the alignment. In some embodiments, depending on the selected spatial context, the adjacent channels may include the immediate neighboring channel, the second-closest adjacent channel, or a wider set of channels. In some embodiments, the weights may be normalized so that their sum equals one to ensure that they contribute proportionally to the overall response. This normalization may prevent one channel from dominating the aligned channels. In some embodiments, the optimization process may include constraints that ensure that the weights adhere to certain criteria. For example, constraints may be applied to prevent negative weights or to limit the maximum weights assigned to adjacent channels.

[0131] In the following, two approaches to designing the time alignment function are described: Approach 1 may be directed to a "delay per channel" technique, and Approach 2 may be directed to a "dynamic time warping" technique.

[0132] With respect to the first approach, one or more reference subjects may be selected from the pool of N subjects to serve as a baseline against which the neural signals of other subjects can be compared and adjusted, in step 705. Step 705 may be similar to step 605, the details of which are described above.

[0133] In step 710, the composite stimuli may be presented to all subjects and the subjects' EEG may be recorded. Step 710 may be similar to step 610, the details of which are described above. Additionally or alternatively, the composite stimuli selected may be different to be more optimal for spatial or temporal alignment, and / or the stimulus duration may be different between the two steps.

[0134] In step 715, the optimized weights (eg, weights obtained from the spatial alignment determined in step 615) may be applied to generate new activity for each channel.

[0135] At step 720, a cross-correlation function may be calculated for each subject and each EEG channel. In certain embodiments, the cross-correlation function quantifies the similarity between two signals as a function of the time lag between them. In this context, it measures the similarity between the EEG signal of the subject (excluding the reference) and the corresponding channel of the reference subject. The calculation may be performed separately for each EEG channel, comparing the signal from the channel of interest in the subject with the corresponding channel in the reference subject.

[0136] In some embodiments, the time lag of the peaks in the cross-correlation function indicates a temporal misalignment or delay between the signals. A positive time lag indicates that the signal in the target channel is delayed relative to the reference channel, while a negative time lag indicates an advance. In some embodiments, a threshold value (e.g., 0.4 for peak values ​​ranging from 0 to 1) may be applied to determine whether the identified peaks are significant. Peaks below the threshold may be considered negligible or indicative of noise, and their corresponding delays may be marked as "missing." In some embodiments, the shape of the cross-correlation function may provide further insight into the temporal relationship between the signals. For example, a wider peak may indicate a more gradual change in alignment. Statistical analysis may be applied to assess the significance of the identified delays or to compare the distribution of delays between subjects.

[0137] In step 725, the delays identified in the cross-correlation function may be compensated, particularly for channels whose peaks exceed a threshold. In some embodiments, for each subject (except the reference subject), the identified delays for each EEG channel are compensated to align the signal with the corresponding channel of the reference subject. Compensation may be achieved by applying a delay filter or shifting samples of the EEG signal by an appropriate number of samples corresponding to the identified delay. In some embodiments, various methods may be employed for delay compensation, including finite impulse response (FIR) filters, infinite impulse response (IIR) filters, or other signal processing techniques. For channels whose cross-correlation function peaks did not exceed a threshold (i.e., "missing" delays), interpolation methods are used to estimate these delays. For example, linear interpolation, spline interpolation, or other mathematical techniques may be utilized to predict missing delay values. In some cases, extrapolation may be used to estimate delays for channels with missing information, extending the compensation process beyond the observed data. In some embodiments, the specific characteristics of the EEG signal, such as its frequency content, amplitude variability, and SNR, may influence the selection of interpolation or extrapolation.

[0138] Regarding the second approach, DTW is a powerful technique used for time alignment in the context of hyperscanning. This approach is particularly useful when dealing with temporal variations in signals that cannot be perfectly aligned due to differences in subject reaction times or signal distortion.

[0139] In the context of DTW, in step 805, a stimulus may be selected that is played simultaneously (e.g., for 10 seconds) to all subjects, and the electrical activity of all members in the subject pool may be recorded by EEG.

[0140] In step 810, the optimized weights (eg, weights obtained from the spatial alignment determined in step 615) may be applied to generate new activity for each channel.

[0141] In step 815, the time series data may be adjusted to synchronize neural activity across subjects and EEG channels. In some embodiments, temporal alignment may typically be performed after spatial alignment to ensure that EEG signals from different subjects and channels are synchronized in time. Spatial alignment using DTW implicitly facilitates temporal alignment because the warping path is adapted to align temporal dynamics. In the DTW approach, per-channel delays are determined for each EEG channel of a subject relative to a reference subject. These delays represent the temporal mismatch between channels and reflect individual differences in reaction time or temporal variability. When using per-channel delays, delay compensation may be performed relative to the reference subject using a delay filter or by sample shifting an appropriate number of samples corresponding to the delay. When using DTW, the DTW algorithm may be applied to each channel relative to the reference subject.

[0142] In step 820, relevant features from the EEG signals may be extracted and / or calculated over short time windows. This process allows for the analysis of neural activity patterns in response to stimuli. In certain embodiments, the length of the short time windows may depend on the experimental goals desired to be achieved. For example, the short time windows may range from 10 to 100 milliseconds. In certain embodiments, one or more relevant feature extraction methods may be selected to characterize neural activity within each short time window. The selected feature extraction methods may be applied to calculate relevant features for each short time window within the aligned EEG signals. For example, event-related potentials (ERPs) correspond to measured brain responses that are a direct result of specific sensory, cognitive, or motor events and may be measured by EEG. The features of the short time windows may be features that describe the ERP patterns. This application may result in a set of feature vectors that capture the temporal evolution of neural activity. In certain embodiments, the calculated features may be averaged across subjects to obtain group-level information. Alternatively, trends or consistency across subjects in the calculated features may be examined. This analysis may involve identifying consistent patterns or fluctuations in neural activity across groups.

[0143] The functional modules described herein are provided as examples. It will be understood that different functional modules can be combined to create different utilities. It will also be understood that additional functional modules or sub-modules may be created to implement a particular utility.

[0144] Generally, any process discussed in this disclosure that is understood to be computer-implementable may be executed by one or more processors of a computer system, such as system environment 110, as described above. Processes or process steps performed by one or more processors may also be referred to as operations. One or more processors may be configured to perform such processes by having access to instructions (e.g., software or computer-readable code) that, when executed by the one or more processors, cause the one or more processors to perform a process. The instructions may be stored in a memory of a computer server. The processor may be a central processing unit (CPU), a graphics processing unit (GPU), or any suitable type of processing unit.

[0145] A computer system, such as system environment 110, may include one or more computing devices. When one or more processors of a computer system are implemented as multiple processors, the multiple processors may be included in a single computing device or may be distributed across multiple computing devices. When the system environment includes multiple computing devices, the memory of the computer system may include each memory of each computing device of the multiple computing devices.

[0146] 9 is a simplified functional block diagram of a computer system 900 that may be configured as a computing device for executing the processes described herein, according to an exemplary embodiment of the present disclosure. FIG. 9 is a simplified functional block diagram of a computer that may be configured according to an exemplary embodiment of the present disclosure. In various embodiments, any of the systems herein may be a hardware assembly including, for example, a data communication interface 920 for packet data communication. The platform may also include a central processing unit ("CPU") 902 in the form of one or more processors for executing program instructions. The platform may include an internal communication bus 908 and a storage unit 906 (e.g., ROM, HDD, SDD, etc.) that may store data on a computer-readable medium 922, while the system 900 may receive programming and data (e.g., voice, video, audio, images, or any other data over the electronic network 925) by network communication via an electronic network 925. The system 900 also includes a memory 904 (such as a RAM) that stores instructions 924 for executing the techniques presented herein, although the instructions 924 may also be stored temporarily or permanently in other modules of the system 900 (e.g., the processor 902 and / or the computer-readable medium 922). The system 900 may also include input / output ports 912 and / or a display 910 for connecting input / output devices such as a keyboard, mouse, touchscreen, monitor, display, etc. Various system functions may be implemented in a distributed fashion across multiple similar platforms to distribute the processing load. Alternatively, a server may be implemented by appropriate programming of one computer hardware platform.

[0147] In this disclosure, the term "based on" means "based at least in part on." Unless context dictates otherwise, the singular forms "a," "an," and "the" include plural referents. The term "exemplary" is used to mean "example," not "ideal." The terms "comprises," "comprising," "includes," "including," or other variations thereof are intended to cover non-exclusive inclusions, and thus a process, method, or product that includes a list of elements does not necessarily include only those elements, but may also include other elements not expressly listed or elements not inherent in such process, method, article, or apparatus. Relative terms such as "about," "approximately," "substantially," and "generally" are used to indicate a possible variation of ±10% from the stated or understood value. Furthermore, the term "between," when used when describing a range of values, is intended to include the minimum and maximum values ​​set forth herein. Although the use of the term "or" in the claims and specification is used to mean "and / or," unless expressly indicated to refer to alternatives only if the alternatives are mutually exclusive, the present disclosure supports a definition that refers only to alternatives and "and / or." As used herein, "another" may mean at least a second or more.

[0148] As used herein, the term "user" generally encompasses any individual or entity, such as a researcher and / or caregiver (e.g., a physician), who may desire information, problem resolution, or any other type of interaction with a provider of the systems and methods described herein (e.g., via an application interface present on an electronic device, etc.). The terms "electronic application" or "application" may be used interchangeably with other terms, such as "program," and generally encompass software configured to interact, modify, overwrite, supplement, or operate in conjunction with other software.

[0149] Program aspects of the technology can be considered "products" or "articles of manufacture," typically in the form of executable code and / or associated data carried on or embodied in some type of machine-readable medium. The "storage" class of media includes any or all of the tangible memory of a computer, processor, or the like, or associated modules such as various semiconductor memories, tape drives, disk drives, and the like, which can provide non-transitory storage for software programming at any time. All or portions of the software can be communicated from time to time via the Internet or various other telecommunications networks. Such communication may, for example, enable the software to be loaded from one computer or processor to another, e.g., from an administrative server or host computer of a mobile communications network to a server computing platform, and / or from a server to a mobile device. Accordingly, other types of media capable of carrying software elements include light waves, radio waves, and electromagnetic waves, such as those used across physical interfaces between local devices via wired and optical terrestrial communications networks and via various air links. Physical elements carrying such waves, such as wired or wireless links, optical links, and the like, can also be considered software-bearing media. As used herein, if not limited to non-transitory, tangible "storage" media, terms such as computer or machine "readable medium" refer to any medium that participates in providing instructions to a processor for execution.

[0150] Furthermore, although some embodiments described herein include some features and not others included in other embodiments, combinations of features from different embodiments are within the scope of the present invention and form different embodiments, as will be understood by those skilled in the art. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0151] While specific embodiments have been described, those skilled in the art will recognize that other and further modifications are possible without departing from the spirit of the present invention, and it is intended that all such changes and modifications be claimed as being within the scope of the present invention. For example, functions may be added to or deleted from the block diagrams, and operations may be interchanged between functional blocks. Steps may be added or deleted to the methods described within the scope of the present invention.

[0152] The subject matter disclosed above should be considered illustrative and not limiting, and the appended claims are intended to cover all such modifications, enhancements, and other embodiments that fall within the true spirit and scope of the present disclosure. Accordingly, to the maximum extent permitted by law, the scope of the present disclosure should be determined by the broadest permissible interpretation of the following claims and their equivalents, and should not be limited or constrained by the foregoing detailed description. While various embodiments of the present disclosure have been described, it will be apparent to those skilled in the art that many more embodiments are possible within the scope of the present disclosure. Accordingly, the present disclosure should not be limited except in light of the appended claims and their equivalents.

Claims

1. 1. A computer-implemented method for performing iterative adjustments to sensory stimuli in a closed-loop optimization system, comprising: receiving, at a computing device, brain activity data associated with stimuli presented to the subject; pre-processing the brain activity data using a processor associated with the computing device; using the processor to perform a source estimation technique on the pre-processed brain activity data to identify one or more voxel locations from which electrical activity contained in the brain activity data is estimated to originate; subsequent to performing the source estimation technique, grouping together a first subset of the one or more voxel locations based on the preprocessed brain activity data, the first subset being associated with a sensory region of interest; subsequent to performing the source estimation technique, grouping together a second subset of the one or more voxel locations based on the preprocessed brain activity data, the second subset being associated with a non-sensory region of interest; and using the processor to determine a first objective function based on the local neural activity identified within the first subset of the one or more voxel locations and to determine a second objective function based on the local neural activity identified within the second subset of the one or more voxel locations; using the processor to determine whether the second objective function is greater than or equal to a threshold minimum; responsive to determining that the second objective function is greater than or equal to the threshold minimum, averaging the amount by which the second objective function exceeds the threshold minimum with the first objective function to generate an average objective function; utilizing the average objective function to refine one or more parameters of the stimulus in subsequent iterations of the closed-loop optimization system; and The computer-implemented method includes:

2. The computer-implemented method of claim 1 , wherein the stimulus is a visual stimulus.

3. 2. The computer-implemented method of claim 1, wherein the one or more parameters of the stimulus include one or more of a color associated with the stimulus, a contrast associated with the stimulus, a movement included in the stimulus, and a length of the stimulus.

4. The computer-implemented method of claim 1 , further comprising: ceasing consideration of the second objective function in response to determining that the second objective function is not equal to the threshold minimum.

5. The computer-implemented method of claim 4 , further comprising utilizing only the first objective function in the subsequent iterations.

6. 2. The computer-implemented method of claim 1, further comprising selecting optimal stimulation parameters for the stimulation in response to determining that the first objective function and the second objective function are not increasing over a predetermined number of successive iterations.

7. The computer-implemented method of claim 1 , wherein the source estimation technique is Low Resolution Electromagnetic Tomography Analysis (LORETA).

8. The computer-implemented method of claim 1 , wherein the first subset of the one or more voxel locations are grouped together based on their identified roles in hierarchical sensory information processing.

9. The computer-implemented method of claim 1 , wherein the second subset of the one or more voxel locations is grouped together based on a spatial-functional characteristic.

10. 2. The computer-implemented method of claim 1, wherein the second subset of the one or more voxel locations corresponds to one or more regions selected from the group consisting of the frontal cortex, the centrotemporal region, the motor cortex, and the posterior parietal lobe.

11. 1. A system for performing iterative adjustments to a sensory stimulus, comprising: one or more processors; One or more computer-readable media having stored thereon instructions executable by the one or more processors to perform operations, the operations including: receiving brain activity data associated with stimuli presented to the subject; preprocessing the brain activity data; performing a source estimation technique on the pre-processed brain activity data to identify one or more voxel locations from which electrical activity contained in the brain activity data is estimated to originate; subsequent to performing the source estimation technique, grouping together a first subset of the one or more voxel locations based on the preprocessed brain activity data, the first subset being associated with a sensory region of interest; subsequent to performing the source estimation technique, grouping together a second subset of the one or more voxel locations based on the preprocessed brain activity data, the second subset being associated with a non-sensory region of interest; determining a first objective function based on the local neural activity identified within the first subset of the one or more voxel locations and determining a second objective function based on the local neural activity identified within the second subset of the one or more voxel locations; determining whether the second objective function is greater than or equal to a threshold minimum; responsive to determining that the second objective function is greater than or equal to the threshold minimum, averaging the amount by which the second objective function exceeds the threshold minimum with the first objective function to generate an average objective function; and utilizing the average objective function to refine one or more parameters of the stimulus in subsequent iterations of a closed-loop optimization function of the system; the one or more computer-readable media, The system comprising:

12. The system of claim 11 , wherein the stimulus is a visual stimulus.

13. 12. The system of claim 11, wherein the one or more parameters of the stimulus include one or more of a color associated with the stimulus, a contrast associated with the stimulus, a movement included in the stimulus, and a length of the stimulus.

14. The instructions, by the one or more processors, further include: in response to determining that the second objective function is not equal to the threshold minimum, ceasing consideration of the second objective function; The system of claim 11 , wherein the system is executable to perform an action.

15. The instructions, by the one or more processors, further include: selecting optimal stimulation parameters for the stimulation in response to determining that the first objective function and the second objective function are not increasing over a predetermined number of successive iterations. The system of claim 11 , wherein the system is executable to perform an action.

16. The system of claim 11 , wherein the source estimation technique is Low Resolution Electromagnetic Tomography Analysis (LORETA).

17. The system of claim 11 , wherein the first subset of the one or more voxel locations are grouped together based on their identified roles in hierarchical sensory information processing.

18. The system of claim 11 , wherein the second subset of the one or more voxel locations are grouped together based on a spatial-functional characteristic.

19. 12. The system of claim 11, wherein the second subset of the one or more voxel locations corresponds to one or more regions selected from the group consisting of the frontal cortex, the centrotemporal region, the motor cortex, and the posterior parietal lobe.

20. A non-transitory computer-readable medium storing computer-executable instructions that, when executed by a system, cause the system to: receiving, at a computing device associated with the system, brain activity data associated with stimuli presented to the subject; pre-processing the brain activity data using a processor associated with the computing device; using the processor to perform a source estimation technique on the pre-processed brain activity data to identify one or more voxel locations from which electrical activity contained in the brain activity data is estimated to originate; subsequent to performing the source estimation technique, grouping together a first subset of the one or more voxel locations based on the preprocessed brain activity data, the first subset being associated with a sensory region of interest; subsequent to performing the source estimation technique, grouping together a second subset of the one or more voxel locations based on the preprocessed brain activity data, the second subset being associated with a non-sensory region of interest; and using the processor to determine a first objective function based on the local neural activity identified within the first subset of the one or more voxel locations and to determine a second objective function based on the local neural activity identified within the second subset of the one or more voxel locations; using the processor to determine whether the second objective function is greater than or equal to a threshold minimum; responsive to determining that the second objective function is greater than or equal to the threshold minimum, averaging the amount by which the second objective function exceeds the threshold minimum with the first objective function to generate an average objective function; utilizing the average objective function to refine one or more parameters of the stimulus in subsequent iterations of a closed-loop optimization system; and The non-transitory computer-readable medium causes the computer to perform operations including:

Images