Using artificial intelligence algorithms for data modeling of functional human brain activity via functional magnetic resonance imaging
A hybrid LSTM/GCN model for fMRI data analysis addresses the limitations of static connectivity methods by predicting CNS disorders at their earliest stages, enabling timely intervention through dynamic brain activity modeling.
Patent Information
- Application Number
- PCT/US2025/042553
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-09-25
- Filing Date
- 2025-08-19
- Publication Date
- 2026-02-26
AI Technical Summary
Existing methods for detecting cognitive impairments related to language comprehension or production, such as those seen in patients with CNS diseases, are often ineffective at early stages due to the reliance on static functional connectivity data and lack of dynamic modeling of brain activity.
The use of a hybrid Long Short-Term Memory (LSTM) and Graph Convolutional Network (GCN) model to process functional magnetic resonance imaging (fMRI) data, capturing dynamic functional connectivity changes in the brain, particularly during language processing, to predict potential medical conditions before behavioral symptoms appear.
This approach enables accurate prediction of subclinical CNS disorders, providing a window for early intervention and improving clinical outcomes by modeling 4D whole brain functional network dynamics.
Smart Images

Figure US2025042553_26022026_PF_FP_ABST
Abstract
Description
Levdig 774186HHS E-220-2024-0-PC-011USING ARTIFICIAL INTELLIGENCE ALGORITHMS FOR DATA MODELING OF FUNCTIONAL HUMAN BRAIN ACTIVITY VIA FUNCTIONAL MAGNETIC RESONANCE IMAGINGCROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 685,059, filed August 20, 2024, and U.S. Provisional Application No. 63 / 698,852, filed September 25, 2024. Both of which are herein incorporated by reference in their entirety.STATEMENT OF GOVERNMENT SUPPORT
[0002] This invention was made with Government support under project number ZIA CL090077 by the National Institutes of Health, National Institute on Deafness and Other Communication Disorders. The United States Government has certain rights in the invention.BACKGROUND
[0003] Patients with cognitive impairments often present with language comprehension or production difficulties. Unfortunately, the underlying disease is usually well advanced by the time that a patient presents for clinical evaluation. For these reasons, researchers have been searching for a neurobehavioral proxy using paper / pencil tasks, cerebrospinal fluid (CSF) markers and functional neuroimaging neuroactivation / positron emission tomography (PET) studies. More recently, some investigators have reported static functional connectivity changes using resting state data in patients with mild to moderate disease using parcellations that are restricted to specific areas known to involve language processing.SUMMARY
[0004] In some examples, a method for processing functional magnetic resonance imaging (fMRI) data using one or more machine learning - artificial intelligence (ML - Al) models is provided. The method comprises: obtaining blood-oxygen-level-dependent (BOLD) signals associated with changes in blood oxygenation and blood flow in a brain of a subject; determining the fMRI data for the subject based on the BOLD signals, wherein the fMRI data indicates correlation strengths between regions of the brain for the subject; generating jack-knife datasetsLevdig 774186HHS E-220-2024-0-PC-012 from the fMRI data using a jack-knife correlation process that removes one or more data points associated with the regions of the brain from the fMRI data to generate each of the jack-knife datasets; processing the jack-knife datasets and the fMRI data using the one or more ML - Al models to generate predicted correlation strengths for each region of the brain for a plurality of time steps; and outputting information associated with the predicted correlation strengths that is generated using the one or more ML - Al models.
[0005] In some instances, the fMRI data comprises a plurality of entries and each entry indicates a correlation strength between a first region of the brain for the subject and a second region of the brain for the subject, and wherein generating the jack-knife datasets using the jackknife correlation process that removes the one or more data points from the fMRI data comprises: removing at least one entry from the fMRI data; and generating a jack-knife dataset based on recalculating the remaining entries from the fMRI data for the plurality of time steps.
[0006] In some variations, the method further comprises: comparing the predicted correlation strengths to baseline correlation strengths to identify a deviation; comparing the deviation with one or more thresholds, wherein the one or more thresholds are associated with one or more regions of the brain and one or more corresponding medical conditions; and determining one or more medical conditions for the subject based on comparing the deviation with the one or more thresholds, wherein outputting the information comprises outputting information associated with the one or more medical conditions for the subject.
[0007] In some examples, the method further comprises: generating a biomarker associated with the medical condition using the deviation, and wherein outputting the information comprises outputting information associated with the biomarker.
[0008] In some variations, determining the fMRI data based on the BOLD signals comprises: processing raw fMRI data indicating the BOLD signals to generate the fMRI data based on removing noise and motion artifacts from the raw fMRI data.
[0009] In some instances, the raw fMRI data of the subject is acquired using a magnetic resonance imaging (MRI) system configured to identify the regions of the brain where blood flow indicates activity.
[0010] In some examples, the method further comprises: obtaining training fMRI data associated with a plurality of subjects, wherein the training fMRI data comprises static dataLevdig 774186HHS E-220-2024-0-PC-013 associated with the plurality of subjects; and prior to determining the fMRI data for the subject, training the one or more ML - Al models using the training fMRI data.
[0011] In some variations, the one or more ML - Al models comprise one or more graph convolutional network (GCN) layers, and wherein training the one or more ML - Al models comprises: generating training j ack-knife datasets for a first subject from the plurality of subjects using first static data for the first subject, wherein the first static data indicates correlation strengths between different regions of a brain of the first subject; processing the training jack-knife datasets and the first static data using the one or more GCN layers to generate a plurality of node features for a plurality of nodes, wherein each of the plurality of nodes is associated with a region of the brain of the first subject; and training the one or more ML - Al models based on the plurality of node features.
[0012] In some instances, the one or more ML - Al models further comprise one or more long short term-memory (LSTM) layers and one or more dense layers, and wherein training the one or more ML - Al models further comprises: processing the plurality of node features for the plurality of nodes using the LSTM layers to generate an LSTM layer output; and processing the LSTM layer output using the one or more dense layers to generate an ML - Al output, wherein training the one or more ML - Al models is further based on using the ML - Al output.
[0013] In some examples, training the one or more ML - Al models is further based on using the ML - Al output comprises: comparing the ML - Al output with groundtruth data; and updating parameters of the one or more ML - Al models based on the comparison.
[0014] In some variations, comparing the ML - Al output with the groundtruth data comprises: computing a first loss by comparing correlation strengths from a first brain region indicated by the ML - Al output with correlation strengths from the first brain region indicated by the groundtruth data; and computing a second loss by comparing correlation strengths from a second brain region indicated by the ML - Al output with correlation strengths from the second brain region indicated by the groundtruth data, wherein updating the parameters of the one or more ML - Al models is based on the first computed loss and the second computed loss.
[0015] In some instances, the fMRI data is task-based fMRI data.
[0016] In some examples, a computing platform for processing functional magnetic resonance imaging (fMRI) data using one or more machine learning - artificial intelligence (ML - Al) models is provided. The computing platform comprises one or more processors; and a non-transitoryLevdig 774186HHS E-220-2024-0-PC-014 computer-readable medium having processor-executable instructions stored thereon, wherein the processor-executable instructions, when executed by the one or more processors, facilitate: obtaining blood-oxygen-level-dependent (BOLD) signals associated with changes in blood oxygenation and blood flow in a brain of a subject; determining the fMRI data for the subject based on the BOLD signals, wherein the fMRI data indicates correlation strengths between regions of the brain for the subject; generating jack-knife datasets from the fMRI data using a jack-knife correlation process that removes one or more data points associated with the regions of the brain from the fMRI data to generate each of the jack-knife datasets; processing the jack-knife datasets and the fMRI data using the one or more ML - Al models to generate predicted correlation strengths for each region of the brain for a plurality of time steps; and outputting information associated with the predicted correlation strengths that is generated using the one or more ML - Al models.
[0017] In some variations, the fMRI data comprises a plurality of entries and each entry indicates a correlation strength between a first region of the brain for the subject and a second region of the brain for the subject, and wherein generating the jack-knife datasets using the jackknife correlation process that removes the one or more data points from the fMRI data comprises: removing at least one entry from the fMRI data; and generating a jack-knife dataset based on recalculating the remaining entries from the fMRI data for the plurality of time steps.
[0018] In some instances, the processor-executable instructions, when executed by the one or more processors, further facilitate: comparing the predicted correlation strengths to baseline correlation strengths to identify a deviation; comparing the deviation with one or more thresholds, wherein the one or more thresholds are associated with one or more regions of the brain and one or more corresponding medical conditions; and determining one or more medical conditions for the subject based on comparing the deviation with the one or more thresholds, wherein outputting the information comprises outputting information associated with the one or more medical conditions for the subject.
[0019] In some variations, the processor-executable instructions, when executed by the one or more processors, further facilitate: generating a biomarker associated with the medical condition using the deviation, and wherein outputting the information comprises outputting information associated with the biomarker.Levdig 774186HHS E-220-2024-0-PC-015
[0020] In some instances, the processor-executable instructions, when executed by the one or more processors, further facilitate: obtaining training fMRI data associated with a plurality of subjects, wherein the training fMRI data comprises static data associated with the plurality of subjects; and prior to determining the fMRI data for the subject, training the one or more ML - Al models using the training fMRI data.
[0021] In some examples, the one or more ML - Al models comprise one or more graph convolutional network (GCN) layers, and wherein training the one or more ML - Al models comprises: generating training j ack-knife datasets for a first subject from the plurality of subjects using first static data for the first subj ect, wherein the first static data indicates correlation strengths between different regions of a brain of the first subject; processing the training jack-knife datasets and the first static data using the one or more GNC layers to generate a plurality of node features for a plurality of nodes, wherein each of the plurality of nodes is associated with a region of the brain of the first subject; and training the one or more ML - Al models based on the plurality of node features.
[0022] In some variations, the fMRI data is task-based fMRI data.
[0023] In some examples, a non-transitory computer-readable medium for processing functional magnetic resonance imaging (fMRI) data using one or more machine learning - artificial intelligence (ML - Al) models is provided. The non-transitory computer-readable medium having processor-executable instructions stored thereon, wherein the processor-executable instructions, when executed, facilitate: obtaining blood-oxygen-level-dependent (BOLD) signals associated with changes in blood oxygenation and blood flow in a brain of a subject; determining the fMRI data for the subject based on the BOLD signals, wherein the fMRI data indicates correlation strengths between regions of the brain for the subj ect; generating jack-knife datasets from the fMRI data using a jack-knife correlation process that removes one or more data points associated with the regions of the brain from the fMRI data to generate each of the jack-knife datasets; processing the jack-knife datasets and the fMRI data using the one or more ML - Al models to generate predicted correlation strengths for each region of the brain for a plurality of time steps; and outputting information associated with the predicted correlation strengths that is generated using the one or more ML - Al models.
[0024] All examples and features mentioned above may be combined in any technically possible way.Levdig 774186HHS E-220-2024-0-PC-016BRIEF DESCRIPTION OF THE DRAWINGS
[0025] The present application will be described in even greater detail below based on the exemplary figures. The application is not limited to the examples described below. All features described and / or illustrated herein can be used alone or combined in different combinations in examples of the application. The features and advantages of various examples of the present application will become apparent by reading the following detailed description with reference to the attached drawings which illustrate the following:
[0026] FIG. 1 is a simplified block diagram depicting an exemplary computing environment in accordance with one or more examples of the present application.
[0027] FIG. 2 is a simplified block diagram of one or more devices or systems within the exemplary environment of FIG. 1.
[0028] FIG. 3A is an exemplary process for determining correlation strengths between regions of a brain in accordance with one or more examples of the present application.
[0029] FIG. 3B shows an exemplary ML - Al model 306 that is used to determine correlation strengths between regions of a brain in accordance with one or more examples of the present application.
[0030] FIG. 4 is an example of functional magnetic resonance imaging (fMRI) data before and after a jack-knife process in accordance with one or more examples of the present application.
[0031] FIG. 5A is an exemplary process for determining correlation strengths between regions of a brain in accordance with one or more examples of the present application.
[0032] FIG. 5B is an exemplary process for training the one or more ML-AI models in accordance with one or more examples of the present application.
[0033] FIG. 6 is a line graph depicting true values of correlation strength of a certain brain region over a plurality of time steps compared to predicted values of correlation strength of the certain brain region over the plurality of time steps in accordance with one or more examples of the present application.
[0034] FIG. 7 is another line graph depicting true values of correlation strength of a certain brain region over a plurality of time steps compared to predicted values of correlation strength of the certain brain region over the plurality of time steps in accordance with one or more examples of the present application.Levdig 774186HHS E-220-2024-0-PC-017
[0035] FIG. 8 is a line graph depicting true values of correlation strength of a plurality of brain regions over a plurality of time steps compared to predicted values of correlation strength of the plurality of brain regions over the plurality of time steps in accordance with one or more examples of the present application.DETAILED DESCRIPTION
[0036] Examples of the presented application will now be described more fully hereinafter with reference to the accompanying exemplary figures, in which some, but not all, examples of the application are shown. Indeed, the application may be embodied in any different forms and should not be construed as limited to the examples set forth herein; rather, these examples are provided so that the disclosure will satisfy applicable legal requirements. Where possible, any terms expressed in the singular form herein are meant to also include the plural form and vice versa, unless explicitly stated otherwise. Also, as used herein, the term “a” and / or “an” shall mean “one or more” even though the phrase “one or more” is also used herein. Furthermore, when it is said herein that something is “based on” something else, it may be based on one or more other things as well. In other words, unless expressly indicated otherwise, as used herein “based on” means “based at least in part on” or “based at least partially on”.
[0037] Systems, methods, and computer program products are herein disclosed that provide for predicting correlation strengths between regions of a brain using functional magnetic resonance imaging (fMRI) data thereby predicting potential conditions or illnesses long before behavioral symptoms are present in a subject. In some embodiments, systems and methods are described that use long short-term memory (LSTM) / graph convolutional networks (GCN) (e.g., a hybrid LSTM / GCN) for four dimensional (4D) data modeling of functional human brain activity via fMRI.
[0038] For instance, the analysis of static functional connectivity (sFC) in neuroscience often utilizes graph neural networks (GNNs) to interpret the complex interplay between brain regions. Given the dynamic nature of cognitive processes, especially in language processing, examples of the present disclosure may aim to develop models capable of capturing dynamic functional connectivity changes. For instance, examples of the present disclosure may describe and / or develop a hybrid LSTM / GCN model that accurately captures the underlying dynamic trends and variables in whole brain functional connectivity data.Levdig 774186HHS E-220-2024-0-PC-018
[0039] In some examples, using whole brain analyses may be able to elucidate the involvement of additional brain areas as these areas have been reported in human functional neurorecovery following central nervous system (CNS) injury. Given the dynamic nature of cognitive processes, especially in language processing, a hybrid computational LSTM / GCN model that accurately captures the underlying dynamic trends and variables in whole brain function connectivity data in healthy individuals using task-based fMRI data would provide solutions to conventional systems in this field. This process may be exploited to develop novel neurodiagnostics of subclinical CNS disease.
[0040] In some instances, examples of the present disclosure implement and / or use a hybrid LSTM / GCN model to predict whole brain functional network dynamics to provide clinical treatment planning, and may further use the hybrid LSTM / GCN model for guided neuromodulation with brain-computer interface technologies.
[0041] In some variations, examples of the present disclosure may relate to using task-based functional magnetic resonance imaging (fMRI) dynamics data and for using jack-knifing (jackknife) correlations in brain modeling. In some instances, the LSTM / GCN 4D modeling and using the LSTM / GCN 4D modeling for predictive capabilities of individualized medical treatment planning or use in guided-neuromodulation with brain computer interface (BCI) is described. In some instances, examples of the present disclosure use hybrid LSTM / GCN hybrid models to predict 4D whole brain task based functional network dynamics (e.g., three-dimensional (3D) spatial network organization and individual temporal network correlation dynamics over time) for neurodiagnostics, or for medical intervention planning, and / or for guided neuromodulation using BCI. Conventional systems may use machine learning (ML) to model two-dimensional (2D) or 3D data of static or dynamic resting state functional connectivity network.
[0042] In some instances, examples of the present disclosure may refine the hybrid LSTM / GCN model to include the hierarchical 3D spatial information into the model. By doing so, examples of the present disclosure may aim to enhance the modeling of spatiotemporal dynamics, thereby providing a more robust framework for capturing significant real-time functional connectivity changes in the human brain in 4D that occur during language processing. In some examples, further training of the model on additional longitudinal data may be performed as well.Levdig 774186HHS E-220-2024-0-PC-019
[0043] In some variations, the hybrid LSTM / GCN model may accurately predict healthy human brain network dynamics. Therefore, the hybrid LSTM / GCN model may hold promise for advancing breakthroughs in neurodiagnostic screening of central neurologic disorders at their earliest stages of disease when clinical manifestations are subclinical and still unknown to a patient (but when brain patterns are already beginning to deviate from normative patterns). Once clinically validated, this computational model may be able to offer clinicians a unique window of opportunity to treat the CNS disease at the subclinical stage of the disease and hence prolong quality of life in patient populations. Additionally, and / or alternatively, this model may also serve as the basis for predicting cognitive and language related brain activity outcomes by using it to as the basis to run brain computer simulations that predict brain clinical outcomes prior to performing a medical intervention. The model uses noninvasive fMRI data from the human brain as input. Computer simulations may be executed using fMRI data from humans or generated fMRI data.
[0044] In some instances, several research groups may use resting state fMRI data to classify particular brain pathologies in particular patient populations. The salient difference between conventional approaches and examples of the present disclosure may be that examples of the present disclosure might not rely on resting state fMRI data because specific research data has been found that shows that network dynamics are fundamentally different during language processing, and are not interchangeable with network dynamics when those networks are in a resting state, even when analyzing identical networks. In some examples, examples of the present disclosure may use machine learning - artificial intelligence (ML - Al) techniques / models as a diagnostic screening tool to detect subclinical disease.
[0045] In some examples, the conceptualization and development of LSTM / GCN hybrid computational model for biologic utilization, including computer simulations for the LSTM / GCN computational model, is described herein. In some instances, examples of the present disclosure may utilize TENSORFLOW, scikit-learn (e.g., a free and open-source machine learning library for the Python programming language), NumPy library for PYTHON programming language, Matplotlib plotting library, and / or Pandas software library.
[0046] In some instances, Exploring Dynamic Functional Connectivity in the Brain: A comparative Study of Line Graph Neural Networks, Long Short-Term Memory (LSTM) is described. For instance, the analysis of static functional connectivity (sFC) in neuroscience often utilizes graph neural networks (GNNs) to interpret the complex interplay between brain regions.Levdig 774186HHS E-220-2024-0-PC-0110Preliminary data from the lab has shown promise in generalizing across different types of cognitive tasks when using line graph neural networks (LGNN) that transform edges into nodes to emphasize inter-regional connectivity. Traditional GNN models mainly focus on nodes representing brain regions, which may not adequately capture the dynamics of connectivity changes. Given the dynamic nature of cognitive processes, especially in language processing, examples of the present disclosure aim to develop models capable of capturing dynamic functional connectivity changes. For these reasons, the ability of LGNN methods to model dynamic functional connectivity changes using Jackknife Pearson correlation generated datasets to enhance the understanding of brain network connectivity fluctuations over time were assessed.
[0047] The methods for performing examples of the present disclosure are described below. For instance, fMRI data from 172 participants who performed an auditory language comprehension task from the Human Connectome Project (HCP) were included. The fMRI data underwent image pre- and post-processing using Analysis of Functional NeuroImages (AFNI) software and Free- surfer software (e.g., brain imaging software). The data also underwent motion correction and Bonferroni correction (e.g., a multiple-comparison correction used when several dependent or independent statistical tests are being performed simultaneously) for multiple comparisons. Desikan parcellations were overlaid to generate functional connectivity networks. Dynamic functional connectivity networks were generated using Jack-knife Pearson correlations in which one data time point is systemically left out and changes in whole brain network correlations are recalculated as a function of the single data point that was left out. The performance of Line GNN and Long Short-Term Memory (LSTM) neural networks were assessed to predict the correlation changes, separately and in a hybrid LSTM / GCN model. A subset of subject data was used to train the models and a different subset was used as ground truth.
[0048] The results are described below. An LGNN model effectively captured static functional connectivity patterns. It exhibited suboptimal performance in predicting the dynamic functional connectivity data. However, the initial run of the LSTM model demonstrated stable predictive accuracy across the dataset in the non-training dataset with Mean Squared Error (MSE): 0.0151, Mean Absolute Error (MAE): 0.0990, R-squared: 0.764. The training and validation losses were 0.0163 and 0.0151, respectively, suggesting minimal overfitting.
[0049] A hybrid model using GCN models that generate regions of interest (ROI) and LSTM to calculate the correlations changes over time in each ROI generate network was able toLevdig 774186HHS E-220-2024-0-PC-0111 remarkably predict the actual brain dynamics over time. For instance, the performance for the GCN / LSTM hybrid is even better than using solely the LSTM models, which is shown by the following metrics: MSE: 0.0031, MAE: 0.03120 and R-squared 0.9344.
[0050] Exploring dynamic models is essential for advancing the understanding of brain function. A hybrid LSTM / GCN model is described herein that accurately captures the underlying dynamic trends and variabilities in whole brain functional connectivity data. In some examples, further refining the hybrid LSTM / GCN model may be performed to include the hierarchical 3D spatial information into the model. By doing so, examples of the present disclosure aim to enhance the modeling of spatiotemporal dynamics, thereby providing a more robust framework for capturing significant real-time functional connectivity changes in the human brain in 4D that occur during language processing. In some instances, the model may predict network dynamics and may hold promise for advancing breakthroughs in neurodiagnostic screening of central neurologic disorders at their earliest subclinical stages when clinical manifestations are still unknown to the patient and thereby offering a unique window of treatment opportunity at the subclinical stage that can prolong quality of life in patient populations.
[0051] In some instances, an overall goal of examples of the present disclosure may include predicting functional neuronal recovery following CNS injury. For instance, examples of the present disclosure may predict changes in how different brain regions communicate with each other during auditory language comprehension (e.g., functional connectivity) following CNS injury, which may be used to better understand a patient’ s neurolinguistics profile and / or to predict how a patient may fare following a specific planned intervention. Examples of the present disclosure may develop a computational modeling of language processing that predict functional neuronal recovery following CNS injury and / or predict through the use of computational simulations how a patient may fare a priori to a specific planned intervention.
[0052] Three lines of scientific investigation are described below. First, conduct experiments that identify neurob ehavi oral features that should be included in the modeling. Second, identifying a computational model that may predict the performance of neural network performance in space and time, (a) model training; and b) confirm model performance through cross-validation. Three, run computer simulations of model predictions for real patient data and confirm those predictions with real patient data.Levdig 774186HHS E-220-2024-0-PC-0112
[0053] Regarding the first line, static functional connectivity (FC) approach and dynamic FC approach may be compared. The measurements for the static FC may include average time courses over the fMRI examination and there might not be a time variant. The measurements for the dynamic FC may include temporal fluctuation on FC over the fMRI examination and there may be a time variant. The optimizing the identification of networks may include a sliding window technique (e.g., window-sensitive effect) and / or Jack-Knife Correlations.
[0054] Regarding the second line, spatial features, dynamic features, and dynamic features derived using Jack-Knife Pearson correlations are described. The computational modeling of spatial features may use graph based models. Line GNN models may generalize to other language- related tasks but they cannot be used to model dynamic time series data. The model descriptions are described. Gated recurrent unit (GRU) and LSTM models are both types of recurrent neural network (RNN) layers designed to handle sequential data. They address the vanishing gradient problem in traditional RNNs by introducing gating mechanisms that allow then to capture long term dependencies more effectively. LSTMs are more complex, comprising three gates - input gate, forget gate, output gate. These gates control the flow of information through the cell state, allowing the LSTM to remember or forget over time. GRU has a simplified architecture with two gates - an update gate and a reset gate. The update gate controls how much of the previous hidden state should be retained and the reset gate determines how much of the past information to forget.
[0055] The LSTM network model is described. For instance, RNNs may be capable of learning long term dependencies. It is designed to process sequences of data, remembering relevant information throughout the sequence and using it to make predictions.
[0056] In some variations, examples of the present disclosure may include a hybrid model (e.g., a hybrid GCN and LSTM) to account for both spatial-temporal modeling.
[0057] Regarding the third line, the hybrid model uses LSTM / GCN to accurately model tasked-based network dynamics in normal healthy brain using jack-knife correlations inputs. The model accurately predicts individualized brain network dynamics much more accurately than the other models. This may enable one to detect the slightest deviation from the normative pattern in a patient with subclinical CNS disease.
[0058] In other words, examples of the present disclosure may combine these computational models with emerging cutting-edge BCI (Brain-Computer Interface) technologies, guidedLevdig 774186HHS E-220-2024-0-PC-0113 neuromodulation techniques, and for more mechanistic neurorehabilitative strategy that shortens recovery time.
[0059] In some examples, a hybrid GNN / LSTM model to model dynamic data may be used. For example, the hybrid GNN / LSTM may be used to model brain dynamics. In some instances, the hybrid GNN / LSTM model might not use a sliding window, but may instead use jack-knife correlations. For instance, sliding windows may demonstrate a window length sensitivity effect and this may introduce a certain “arbitrariness” to the data analyses that may be difficult to statistically eliminate. As such, examples of the present disclosure may usejack-knife correlations rather than using sliding window. In some variations, brain data may be 4D in nature (e.g., 3D spatial and a temporal dynamic component). Therefore, modeling in 4D data may be more difficult than modeling in 2D data. The hybrid GNN / LSTM may currently be given the 3D data and it may be able to model the temporal dynamics of each individual network. Additionally, and / or alternatively, the 3D data may be generated from the source data and this may be incorporated into the model.
[0060] In some instances, examples of the present disclosure may use a type of dynamic modeling described herein to not only classify but predict neurodegenerative diseases as its earliest point of deviation from the norm. For instance, linguistic behavioral changes may still be a later point of differentiation than brain dynamics because there may be a lot of redundancy built into the brain’s neural instantiation of behavior. Thus, it might still be well after neuronal dynamic changes have occurred before seeing behavioral changes. In some variations, examples of the present disclosure may use predictive dynamic modeling to develop individualized treatment plans. For instance, examples of the present disclosure may collect prospective data that may assist in confirming how the model described herein may help to guide neuromodulatory rehabilitative approaches. In some instances, examples of the present disclosure may focus on speech perception and using brain dynamics and / or speech production and using dynamic speech output as the model input. For instance, the above two may be intimately tied together and share a lot of overlap, although the model described herein is modeling 4D data which may be more difficult.
[0061] FIG. 1 is a simplified block diagram depicting an exemplary computing environment in accordance with one or more examples of the present application. The environment 100 includes one or more MRI training computing systems (e g., fMRI systems) 102, an inference MRILevdig 774186HHS E-220-2024-0-PC-0114 computing system 104, a network 106, and a computing platform 108 (e.g., a cloud computing platform). The computing platform 108 may include and / or use one or more ML - Al models 110. Although the entities within environment 100 may be described below and / or depicted in the FIGs. as being singular entities, it will be appreciated that the entities and functionalities discussed herein may be implemented by and / or include one or more entities.
[0062] The entities within the environment 100 such as the MRI training computing systems 102, the inference MRI computing system 104, and the computing platform 108 may be in communication with other systems within the environment 100 via the network 106. The network 106 may be a global area network (GAN) such as the Internet, a wide area network (WAN), a local area network (LAN), or any other type of network or combination of networks. The network 106 may provide a wireline, wireless, or a combination of wireline and wireless communication between the entities within the environment 100. Additionally, and / or alternatively, the entities of environment 100 may be communication without using the network 106. For instance, the inference MRI computing system 104 may use one or more wired connections and / or one or more communication protocols such as WI-FI or BLUETOOTH to communicate with the computing platform 108.
[0063] The MRI training computing systems 102 are and / or include one or more computing devices and / or systems that are configured to generate, receive, utilize, and / or otherwise obtain training fMRI data, and provide the training fMRI data to the computing platform 108. For example, the MRI training computing systems 102 are and / or include one or more computing devices, computing platforms, systems, servers, desktops, repositories, databases, laptops, tablets, mobile devices (e.g., smartphone device, or other mobile device), MRI devices, or any other type of computing device that generally comprises one or more communication components, one or more processing components, and one or more memory components.
[0064] In some examples, the MRI training computing systems 102 may be in communication with and / or include an MRI device that captures fMRI data for a plurality of subjects that form the fMRI training data. For example, the MRI training computing systems 102 may include an MRI device that obtains raw fMRI data from a plurality of subjects (e.g., individuals). The MRI training computing systems 102 may then provide the raw fMRI data to the computing platform 108. For instance, after the MRI device obtains the raw fMRI data, another device, such as a computing device and / or mobile device, may transmit the raw fMRI data to the computing platform 108.Levdig 774186HHS E-220-2024-0-PC-0115Additionally, and / or alternatively, the MRI training computing systems 102 may generate training fMRI data from the raw fMRI data, and provide the training fMRI data to the computing platform 108. In some instances, the MRI training computing systems 102 might not include MRI devices. Instead, the MRI training computing systems 102 may be and / or include databases that obtain the raw fMRI data and / or the training fMRI data from the MRI devices, and provide the raw fMRI data and / or the training fMRI data to the computing platform 108.
[0065] In some variations, the MRI training computing systems 102 may be implemented as engines, software functions, and / or applications. In other words, the functionalities of the MRI training computing systems 102 may be implemented as software instructions stored in storage (e.g., memory) and executed by one or more processors.
[0066] The computing platform 108 may implement, utilize, and / or include one or more ML - Al models 110. For instance, the computing platform 108 may obtain data from the MRI training computing systems 102 and / or the inference MRI computing system 104. For instance, the computing platform 108 may obtain the raw fMRI data and / or the training fMRI data from the MRI training computing systems 102. Subsequently, the computing platform 108 may use the raw fMRI data and / or the training fMRI data to train and / or update the ML - Al models 110. For instance, the computing platform 108 may iterate multiple epochs of the ML-AI models 110 using the training fMRI data by communicating with the computing platform 108 via network 106 until the training has completed. In some variations, the computing platform 108 may generate j ackknife datasets using the fMRI training data. The jack-knife datasets may be provided as input to the ML-AI models 110. The jack-knife datasets may be generated from the training fMRI data by removing one or more data points or entries, and recalculating the rest of the dataset based on the missing data point. A jack-knife correlation process may refer to a process of systematically leaving out one observation or data point from a dataset and calculating a parameter estimate over the remaining observations of the dataset and then aggregating the calculations to generate a jackknife dataset. The training and updating the one or more ML - Al models 110 will be described in more detail below. Once the ML - Al models 110 are trained, they may be used during an inference phase, which is described below.
[0067] The computing platform 108 includes one or more computing devices, computing platforms, processors, cloud computing platforms, systems, servers, and / or other apparatuses capable of performing tasks, functions, and / or other actions such as training and / or using the MLLevdig 774186HHS E-220-2024-0-PC-0116- Al models 110. For instance, the computing platform 108 may include a training system that is configured to train the ML - Al models 110. Afterwards, the computing platform 108 may include a database (e.g., memory) that stores the ML - Al models 110. The computing platform 108 may further include a prediction system that is configured to use the trained ML - Al models 110 to generate one or more predictions. In some variations, the computing platform 108 may be implemented as engines, software functions, and / or applications. In other words, functionalities of the computing platform 108 may be implemented as software instructions stored in storage (e.g., memory) and executed by one or more processors.
[0068] In some examples, the computing platform 108 may include one or more processors such as central processing units (CPUs). The processor may include one or more physical cores (e.g., CPU cores) or processing units, which are pieces of logic capable of independent performing the functions of the processor. The physical cores may include multiple different threads of execution that can execute multiple different tasks at one time. For instance, each of the threads of execution may be executing programming code independent of each other. By using multiple threads that executed at simultaneously (e.g., multi -threaded processing), the processor may perform multiple tasks at the same time.
[0069] In some instances, the computing platform 108 may further include one or more MRI devices. As such, the functionalities and capabilities of the training and inference MRI computing systems 102 and 104 may be performed by the computing platform 108. In other words, the computing platform 108 may include MRI devices (e.g., MRI machines) that obtain the raw fMRI data from subjects, generate training fMRI data, train the ML - Al models 110, and use the trained ML - Al models 110 to generate one or more predictions.
[0070] The inference MRI computing system 104 includes one or more computing devices, computing platforms, systems, servers, desktops, laptops, tablets, mobile devices (e.g., smartphone device, or other mobile device), or any other type of computing device that generally comprises one or more communication components, one or more processing components, and one or more memory components. The inference MRI computing system 104 may include and / or be in communication with an MRI machine that scans a subject to obtain and / or generate fMRI data (e.g., raw inference fMRI data and / or the inference fMRI data) for the subject. The inference MRI computing system 104 may provide the obtained and / or generated fMRI data to the computing platform 108. The computing platform 108 may use the trained ML - Al models 110 to generateLevdig 774186HHS E-220-2024-0-PC-0117 one or more predictions (e.g., inference predictions). The computing platform 108 may provide the one or more predictions back to the inference MRI computing system 104.
[0071] In some examples, the inference MRI computing system 104 may include one or more computing devices and / or mobile devices that are capable of receiving information from the computing platform 108. For instance, the one or more computing devices and / or mobile devices may receive, from the computing platform 108, the predictions that are generated from the trained ML - Al models 110. In some instances, the inference MRI computing system 104 may receive and / or use the trained ML - Al models 110. For instance, after training, the computing platform 108 may provide the trained ML - Al models 110 to the inference MRI computing system 104. The inference MRI computing system 104 may receive inference fMRI data from subjects (e.g., patients) and use the inference fMRI data to generate predictions. In other words, after training and receiving the inference fMRI data, the inference MRI computing system 104 may perform the inference, and use the trained ML - Al models 110 to generate predictions.
[0072] The inference MRI computing system 104 is and / or includes one or more computing devices, computing platforms, systems, servers, desktops, repositories, databases, laptops, tablets, mobile devices (e.g., smartphone device, or other mobile device), MRI devices, or any other type of computing device that generally comprises one or more communication components, one or more processing components, and one or more memory components. In some variations, the inference MRI computing system 104 may be implemented as engines, software functions, and / or applications. In other words, the functionalities of the inference MRI computing system 104 may be implemented as software instructions stored in storage (e.g., memory) and executed by one or more processors.
[0073] In some instances, the inference MRI computing system 104 and / or the computing platform 108 may compare the predicted outputs from the ML - Al models 110 (e.g., the predicted correlation strengths) to baseline correlation strengths to identify a deviation in one or more of the regions of the brain of a subject or predicted subject. The deviation may correspond to a value. The value may be compared to one or more threshold values that are maintained and defined by the inference MRI computing system 104 and / or the computing platform 108. The inference MRI computing system 104 and / or the computing platform 108 may maintain a mapping of threshold values to particular regions of the brain and corresponding conditions. For example, certain threshold values may be associated with certain regions of the brain and a deviation that is lessLevdig 774186HHS E-220-2024-0-PC-0118 than, equal to, or surpasses the threshold may be indicative of early onset Alzheimer’s disease in the subject. In some variations, the inference MRI computing system 104 and / or the computing platform 108 may be in communication with one or more databases that maintain and update treatment courses or plans for one or more conditions, illnesses, or diseases. The inference MRI computing system 104 and / or the computing platform 108 may be configured to map a determined condition to treatment options which may be provided to a user device or computer system for presentation or explanation. The inference MRI computing system 104 and / or the computing platform 108 may generate a biomarker that is associated with a particular condition using the deviations for the particular region of the brain at one or more time steps. The biomarkers may also be used to diagnose a subject or recommend a treatment plan for a determined condition. In some instances, the inference MRI computing system 104 and / or the computing platform 108 may be associated with a novel dataset reader that may be configured to predict patterns found in a dataset (e.g., inference data) that is similar, but different from the training dataset (e.g., the training data). For instance, based on predicting the patterns, the inference MRI computing system 104 and / or the computing platform 108 may be used to identify and / or quantify subtle biomarkers associated with a particular condition such as CNS disease.
[0074] It will be appreciated that the exemplary environment depicted in FIG. 1 is merely an example, and that the principles discussed herein may also be applicable to other situations — for example, including other types of institutions, organizations, devices, systems, and network configurations. As will be described herein, the environment 100 may be used by health care enterprise organizations.
[0075] FIG. 2 is a block diagram of an exemplary system and / or device 200 (e.g., one or more MRI training computing systems (e.g., fMRI systems) 102, inference MRI computing system 104, computing platform 108, and one or more ML- Al models 110) within the environment 100. The device / system 200 includes one or more processors 204, such as one or more CPUs, controller, and / or logic, that executes computer executable instructions for performing the functions, processes, and / or methods described herein. In some examples, the computer executable instructions are locally stored and accessed from a non-transitory computer readable medium, such as storage 210, which may be a hard drive or flash drive. Read Only Memory (ROM) 206 includes computer executable instructions for initializing the processor 204, while the random-access memory (RAM) 208 is the main memory for loading and processing instructions executed by theLevdig 774186HHS E-220-2024-0-PC-0119 processor 204. The network interface 212 may connect to a wired network or cellular network and to a local area network or wide area network, such as the network 106. The device / system 200 may also include a bus 202 that connects the processor 204, ROM 206, RAM 208, storage 210, and / or the network interface 212. The components within the device / system 200 may use the bus 202 to communicate with each other. The components within the device / system 200 are merely exemplary and might not be inclusive of every component, server, device, computing platform, and / or computing apparatus within the device / system 200. For example, the one or more MRI training computing systems (e.g., fMRI systems) 102, inference MRI computing system 104, and computing platform 108 may include some of the components within the device / system 200 and may also include additional and / or alternative components. Additionally, and / or alternatively, the device / system 200 may further include components that might not be included within every entity of environment 100.
[0076] FIG. 3A is an exemplary process 300 for determining correlation strengths between regions of a brain in accordance with one or more examples of the present application. The process 300 may be performed by the MRI training computing systems 102, the inference MRI computing system 104, and / or the computing platform 108 of environment 100 shown in FIG. 1. However, it will be recognized that the process 300 may be performed in any suitable environment and that any of the following blocks may be performed in any suitable order. The process 300 may be used both in the inference and training phases. The training phase will be described initially below and then the inference phase will be described.
[0077] The process 300 includes, at block 302, receiving input data. For instance, during the training phase, the computing platform 108 may receive input data (e.g., fMRI data) from the MRI training computing systems 102. In some examples, the input data may include fMRI data of one or more subjects. For instance, the MRI training computing systems 102 may include one or more MRI machines, which may obtain MRI information from the one or more subjects. For example, the MRI machines may perform fMRI, which measures brain activity by detecting changes associated with blood flow, and obtain raw fMRI data. In some instances, the MRI training computing systems 102 may perform one or more data processing algorithms, steps, and / or processes to process the raw fMRI data. For instance, based on processing the raw fMRI data, the MRI training computing systems 102 may generate the fMRI data (e g., the training fMRI data), and may provide the fMRI data to the computing platform 108. For example, the MRI trainingLevdig 774186HHS E-220-2024-0-PC-0120 computing systems 102 may generate fMRI data indicating correlation strengths between different regions of the subject’s brain (e.g., different networks of the subject’s brain). For instance, in some variations, the generated fMRI data may be a two-dimensional (2-D) matrix that includes a plurality of entries. The rows and columns of the 2D matrix may each indicate a different region of the subject’s brain (e.g., the 2D matrix may include 39 rows and 39 columns for the 39 regions of the brain). Each entry within the 2D matrix may indicate a correlation strength between the associated regions of the brain from the row and column. This is shown in FIG. 4.
[0078] For example, FIG. 4 is an example of functional magnetic resonance imaging (fMRI) data before and after a jack-knife process in accordance with one or more examples of the present application. For simplicity, the fMRI data 400 (e.g., the training and / or inference fMRI data) includes five rows 404 and five columns 406. Each row 404 and each column 406 may indicate a region of the brain. For example, the third row (“3”) and the fourth column (“4”) may indicate a correlation strength between a region of the brain associated with the third row (e.g., the third region of the brain) and another region of the brain associated with the fourth row (e.g., the fourth region of the brain).
[0079] In other words, the MRI training computing systems 102 and / or the computing platform 108 may obtain raw fMRI data that indicates a blood-oxygen-level-dependent (BOLD) signal associated with changes in blood oxygenation and blood flow in the brain of a subject. For instance, after an area of neural activity, a BOLD signal may be acquired indicating a spike of deoxygenated hemoglobin in the red blood cells. Specifically, the deoxygenated hemoglobin may be paramagnetic, and the MRI device may be able to acquire the signal change (e.g., signal changes that are occurring relative to a baseline signal of the subject). In other words, the MRI device may acquire the raw fMRI data of the subject indicating the activity changes in the signal because of the spike in deoxygenated hemoglobin due to the neurons that used up the oxygen. The MRI training computing systems 102 and / or the computing platform 108 may process the acquired raw fMRI data such as by using one or more software packages, algorithms, processes, and / or other methods to filter, clean, and / or other enhance the raw fMRI data. For instance, based on performing the filtering / cleaning, the MRI training computing systems 102 and / or the computing platform 108 may remove noise, artifacts (e.g., due to the movement of the subject’s head during the MRI process), and / or other unwanted data. Subsequently, after processing the raw fMRI data, the MRI training computing systems 102 and / or the computing platform 108 may obtain the fMRILevdig 774186HHS E-220-2024-0-PC-0121 data (e.g., the training fMRI data) that is used for process 300, which has the data structure shown in data structure 400 (e.g., the rows and columns indicating different regions of the brain and including entries that indicate correlation strengths between the different regions).
[0080] In some instances and as mentioned above, the MRI training computing systems 102 may obtain the raw fMRI data from a plurality of subjects, generate the fMRI data (e.g., the training fMRI data 400), and provide the training fMRI data to the computing platform 108. In other instances, the computing platform 108 may receive the raw fMRI data from the MRI training computing systems 102 and generate the training fMRI data. In yet other instances, the computing platform 108 may receive the raw and / or training fMRI data from one or more other data source such as a data source associated with the Human Connectome Project (HCP).
[0081] In some examples, the MRI training computing systems 102 and / or the computing platform 108 may perform one or more processes to improve image quality and / or otherwise enhance the data. For instance, as mentioned above, the MRI training computing systems 102 and / or the computing platform 108 may improve the image quality based on removing noise and / or motion artifacts from the raw fMRI data and / or the generated fMRI data.
[0082] Returning back to FIG. 3A, the input data 302 may include the training fMRI data (e.g., data that is within the data structure 400 shown in FIG. 4). In some examples, the fMRI data may be task-based fMRI data.
[0083] At block 304, the computing platform 108 may perform pre-processing on the fMRI data (e.g., the training fMRI data). For instance, in some variations, the computing platform 108 may perform a jackknife correlation process that removes one or more entries from the input data.
[0084] For instance, returning to FIG. 4, the data structure 400 includes five rows 404 and five columns 406 indicating different regions of the brain. The entries of the data structure 400 may indicate correlation strengths associated with the different regions of the brain, which is based on the raw fMRI data. The computing platform 108 may perform a jackknife correlation process that removes one or more entries from the data structure 400, and generates a data structure 408 based on recalculating the remaining entries for the data structure 400 over a number of time steps or time points. For instance, the computing platform 108 may utilize the below equation to generate a jackknife dataset such as the jackknife dataset 408:Levdig 774186HHS E-220-2024-0-PC-0122 where T is the number of time points and xtand ytare the expected values.
[0085] For instance, to generate a jack-knife dataset 408, the computing platform 108 may remove one or more entries from the input data 302 (e.g., the fMRI data structure 400), and then recalculate the other entries based on the removal and using the above equation. For example, based on removing an entry from row 1 of the fMRI data structure 400 and using the above equation, the computing platform 408 may generate the data structure 408. The data structure 408 includes the same number of rows (e.g., five) as the data structure 400, but includes seven columns 410. The rows 404 of the data structure 408 may still indicate the different regions of the brain, but the columns 410 may indicate a number of time points (e g., T in the equation above). As such, based on the data from the data structure 400, removing one or more entries, and using the equation above, the computing platform 108 may generate a jackknife dataset that has the data structure 408.
[0086] In operation, the computing platform 108 may perform the jack-knife process a plurality of instances to generate a plurality of jack-knife datasets. For instance, after generating the first jackknife dataset, the computing platform 108 may generate a second jack-knife dataset based on removing one or more other entries. For instance, for the first j ackknife dataset, the computing platform 108 may remove the entry associated with row 1 and column 2. In some instances, for the first j ackknife dataset, the computing platform 108 may further remove one or more additional entries such as the entry associated with row 1, column 3. Then, for the second jackknife dataset, the computing platform 108 may remove one or more other entries (e.g., one or more entries that are not row 1 and column 2). Following, the computing platform 108 may generate a third jack-knife dataset, and so on. The data structure 408 shows only seven columns or time steps for simplicity, but the jackknife dataset may include any number of columns (e.g., time steps) such as 100 or more time steps. For instance, in one example, based on having correlation strengths associated with 31 regions of the brain (e.g., 31 rows and columns from the data structure 400) and 100 time steps, the computing platform 108 may generate 31 jackknife datasets, and each of the 31 jackknife datasets may include 31 rows (e.g., associated with the 31 regions of the brain) and 100 time steps based on removal of one entry from the original fMRI training data. In some examples, the computing platform 108 may generate a 3D tensor for the jackknife dataset, and the 3D tensor may have the dimensions 31 x 31 x 100 based on the example described above. As such, the 31 jackknife datasets may be included into a 3D tensor. In someLevdig 774186HHS E-220-2024-0-PC-0123 instances, the computing platform 108 may remove two entries and as such, the computing platform 108 may generate 31 jackknife datasets, with each having 31 rows and 50 columns (e.g., due to the removal of two entries from the row).
[0087] In some examples, for calculating and / or generating the jackknife datasets, the computing platform 108 may use weights associated with one or more regions of the brain. For example, depending on the task or application, different regions of the brain may have more importance. Thus, the computing platform 108 may obtain a plurality of weights associated with the regions of the brain (e.g., weights for the 31 regions of the brain). The computing platform 108 may use the plurality of weights and the above equation to calculate the jackknife datasets, and then use the jackknife datasets as input to the ML - Al model 306.
[0088] Returning back to FIG. 3, the input data 302 and the jack-knife datasets from the preprocessing jack-knife correlation process 304 may be provided to the ML - Al model 306. The training and usage of the ML - Al model 306 is described in more detail below with reference to FIG. 3B. For example, the computing platform 108 may provide inputs to the ML - Al model 306. The inputs may include static data associated with the input data 302 (e.g., a static matrix such as a matrix based on the data structure 400 that indicates the correlation strengths associated with different regions of the brain) and dynamic data (e.g., time dependent adjacency matrices such as the jack-knife datasets generated at block 304 using the pre-processing jack-knife correlation process and including a number of time steps). For instance, the time dependent adjacency matrices may include a matrix sequence for a subject where all of the matrices have a time point removed and the rest of the data set is recalculated for that matrix.
[0089] In some instances, during training, the computing platform 108 may perform batch processing. For instance, the computing platform 108 may select a batch size and split the inputs (e.g., the static data and / or the dynamic data) into different batches. The computing platform 108 may provide each batch of data into the ML - Al model 306 to generate one or more outputs and use the generated outputs to perform training for the ML - Al model 306. After processing the entire batch, an epoch may be completed. The training process may perform a plurality of epochs (e.g., 100 epochs) until the ML - Al model 306 is sufficiently trained. In some instances, for each epoch, the static data and the dynamic data that is used to train the ML - Al model 306 may be associated with a single subject or individual. In other instances, for each epoch, the static data and the dynamic data that is used to train the ML - Al model 306 may be associated with moreLevdig 774186HHS E-220-2024-0-PC-0124 than one subject or individual. The training of the ML - Al model 306 will be described in further detail in FIG. 3B.
[0090] FIG. 3B shows an exemplary ML - Al model 306 that is used to determine correlation strengths between regions of a brain in accordance with one or more examples of the present application. For instance, the ML - Al model 306 may include one or more GCN layers 308, LSTM layers 310, and / or dense layers 312. The example of the ML - Al model 306 shown in FIG. 3B is merely one such example, and the ML - Al model 306 may include additional and / or alternative layers and / or Al models / algorithms. For instance, in other examples, the ML - Al model 306 may utilize graph neural network (GNN) layers instead of GCN layers and / or gated recurrent unit (GRU) layers instead of LSTM layers.
[0091] In operation, the GCN layers 308 may receive, as depicted in FIG. 3A, input such as static input (e.g., a static matrix) and / or dynamic input (e.g., the jackknife datasets that include time steps). For instance, the static input (e.g., a static matrix) may be based on the fMRI data of a subject that indicates correlation strengths of each region of a brain of the subject (e.g., the training fMRI data). For instance, the static matrix may be a 2D matrix that represents the static, unchanging relationships between the regions of interests (ROIs) such as the region of the subject’s brain. The static matrix may be flattened across the time points / time steps to align with the dynamic input of the model. For instance, if there are 31 unique ROIs (e.g., 31 regions of the brain or 31 rows 404 / columns 406), the dimensions of the static input may be (“batch_size”, 31, and 31). This is shown and described above except in FIG. 4, only five regions of the brain were used for simplicity.
[0092] The dynamic input may be based on the j ack-knife datasets that are generated from the fMRI data of the subject. For instance, the dynamic input may be a three-dimensional (3D) tensor that has the dimensions (“batch size”, “time steps”, “num ROIs”, “num ROIs”) representing how the connections between ROIs change over time. For instance, for 100 time points and 31 unique ROIs, the shape may be (“batch_size”, 100, 31, 31). This is shown and described above except in FIG. 4, only five regions of the brain and seven time steps were used for simplicity.
[0093] Afterwards, the GCN layers 308 may be configured to process the input such as the static input and / or the dynamic input at each time step. For instance, the adjacency matrix may encode the pairwise relationships between brain regions, and the GCN layers 308 may learn how to update each node’s (ROI’s) features based on the connections it has with other nodes. ForLevdig 774186HHS E-220-2024-0-PC-0125 example, based on the dynamic input and the static input, the GCN layers 308 may generate and / or update node features for each of the nodes. Each node may be associated with an ROI (e.g., a region of the brain, which is indicated by both the static input and the dynamic input). For example, for 31 ROIs, the GCN layers 308 may determine 31 nodes and node features for each of the nodes (e.g., 80 node features).
[0094] In other words, the computing platform 108 may provide, as input to the GCN layers 308, static and dynamic data. The static data may be the training fMRI data (e.g., the 2D matrix indicating correlation strengths between the different regions of the brain). The dynamic data may be the j ackknife datasets (e.g., the 3D tensor that is generated from the static data using the equation above, and the 3D tensor is for a plurality of different time steps). Additionally, and / or alternatively, to achieve the same dimensions between the static and dynamic data, the computing platform 108 may flatten and / or duplicate the static data. For instance, the computing platform 108 may duplicate the static data a number of times based on the time steps (e.g., 100 times due to the 100 time steps for the dynamic data). As a result, the static data is transformed to a 3D tensor, and the 3D tensors are provided as input the GCN layers 308.
[0095] The GCN layers 308 may process the static and dynamic data a number of iterations (e.g., based on the number of time steps) such as by applying one or more graph convolutions. For instance, for the first iteration, the GCN layers 308 may process the first time step of the dynamic data (e.g., a column 410 from the data structure 408 of FIG. 4) for each of the regions. As a result, the GCN layers 308 may output a plurality of node features for each region of the brain. For example, based on having 31 regions, the GCN layers 308 may output a number of node features (e.g., 80 features per node and 2,480 total node features). Next, for the second iteration, the GCN layers 308 may process the second time step of the dynamic data and adjust, update, include, and / or other modify the node features from the first iteration based on the dynamic data for the second time step. Then, the GCN layers 308 may process the second time step of the dynamic data, and adjust / update / modify the node features from the second iteration. The GCN layers 308 may continue and repeat for each of the time steps. Following, after processing each of the time steps, the GCN layers 308 may output the node features for the different regions of the brain and provide the node features to the LSTM layers 310.
[0096] In other words, the output of the GCN layers 308 is a transformed version of the graph at each time step where the GCN layer 308 has processed the adjacency matrix representingLevdig 774186HHS E-220-2024-0-PC-0126 connections between ROTs and updated the node features. The GCN layer 308 takes the matrix for a specific time step and applies a graph convolution, which updates each nodes features based on its neighboring nodes. The output is a matrix where each row corresponds to a node and each column is a feature that the GCN learned for that node based on its connections. For example, if there are 31 ROIs and the GCN outputs 80 features per node, the output would have a shape of (31,80) at each time point. Since there is a sequence of matrices, the GCN layer 308 processes each time point independently, generating a new feature representation for the nodes at each time step. These individual GCN outputs are stacked together to form a 3D tensor with the shape (batch size, time steps, num ROIs x gen output size), (batch size, 100, 31 x 80). This stacked 3D tensor is then passed to the LSTM layer 310, which processes the node embeddings over time to learn how the connections between ROIs evolve.
[0097] To put it another way, the output from the GCN layers 308 is a 3D tensor of node feature embeddings that describe the relationship between ROIs at each time step (e.g., timepoint). Then, in addition to the dynamic data, the static matrix may flattened and repeated across all time steps. The dynamic GCN outputs and the flattened static matrix are concatenated for each time step, forming a combined input for the LSTM layer 310. For example, if the static matrix is flattened and it has 31 ROIs, 31 x 31 = 961 elements, then the dynamic GCN output is 31 x 80 = 2480, then the combined input to the LSTM layer would be 2480 + 961 = 3441 features per time step.
[0098] In some variations, the GCN layers 308 may include a normalization layer that implements a normalization process where an adjacency matrix (e.g., a matrix associated with the 3D tensor for the dynamic data) may be augmented with self-loops that are normalized to ensure that each node incorporates information from itself and its neighbors (message passing process between nodes). In some instances, the GCN layers 308 may include a feature propagation layer that implements feature propagation to propagate information across the graph as well as a linear transformation to ensure that each node in the graph aggregates information from its connected nodes. In some examples, the GCN layers 308 may include a function / layer (e.g., a rectified linear unit (ReLU) activation function / layer) that introduces non-linearity allowing the ML - Al models 306 to capture complex relationships between brain regions.
[0099] In some variations, the GCN layers 308 may stack all the outputs and / or inputs from the GCN layers 308 together. In some examples, the ML - Al models 306 may flatten theLevdig 774186HHS E-220-2024-0-PC-0127 connectivity matrix of the static input (the training fMRI data) and combine it with the dynamic data (e.g., 3D tensors of the jackknife datasets) via concatenation to form the input of the GCN layers 308.
[0100] Afterwards, the generated output from the GCN layers 308 (e.g., the concatenation of the dynamic data and static input) may be provided as input to the LSTM layers 310. The LSTM layers 310 may process the output of the GCN layers 308 over time allowing the ML - Al models 306 to capture temporal dynamics of how the brain region interactions evolve across the time series. For instance, the LSTM layers 310 may process the node features that are output from the GCN layers 308 to determine connections between the nodes (e.g., the regions of the brain) and / or other information associated the nodes. The output from the LSTM layers 310 may then be provided to the dense layers 312.
[0101] The dense layers 312 may process the output from the LSTM layers 310 and generate a final prediction. The final prediction of the dense layers 312 may be provided as the output 314 from FIG. 3 A.
[0102] For example, returning to FIG. 3 A, by using the ML - Al model 306, the computing platform 108 may generate an output 314. In some instances, the output 314 may be a 3D tensor that predicts how the connectivity between pairs of ROIs evolve over time (e.g., a prediction of changes in connectivity between regions over time such as predicted j ackknife correlation data for the subject). For instance, the 3D tensor may have the dimensions (“batch size”, “num connections”, “time steps”). The batch size and the time steps for the predicted jackknife correlation data is described above. The “num connections” may indicate a number of connections between a first node and other nodes. For instance, the output 314 may indicate a first node (e.g., a first region of the brain) and a number of connections between the first node and other nodes (e.g., other regions of the brain). The output 314 may indicate the number of connections for each of the nodes (e.g., a first node may have connections to two other nodes, a second node may have connections to five other nodes, and so on).
[0103] This output 314 may be represented in graphical form, such as the line graphs depicted in FIGs. 6-8, which show a comparison between predicted correlation strengths (e.g., the output 314 from the ML - Al model 306) to actual correlation strengths over a plurality of time steps for a particular region of the brain or across all the regions of the brain for a subject or predicted subject.Levdig 774186HHS E-220-2024-0-PC-0128
[0104] In other words, the inputs to the ML - Al model 306 may include training datasets. The training datasets may include dynamic adjacency matrices (e.g., jack-knife datasets) and also static connectivity matrices (e.g., the fMRI data from the subject). Further, the training datasets may include target or groundtruth correlations (e.g., the actual changes in connectivity that the model 306 is attempting to predict). The ML - Al model 306 may be trained a plurality of epochs (e.g., 100 epochs).
[0105] For each training iteration or step, the computing platform 108 may perform batching on the static and / or dynamic input based on a batch size to generate a plurality of subsets of training data. Then, for each batch, the computing platform 108 may process a subset of the training data (e.g., the batch of adjacency matrices and static data) to determine or generate predictions using the ML - Al model 306 (e.g., the GCN layers 308, the LSTM layers 310, and the dense layers 312). Following, the computing platform 108 may compute a loss (e.g., based on how far off the predictions are from the actual correlations) and apply gradients to update the weights / parameters of the ML - Al model 306 (e.g., the GCN layers 308, the LSTM layers 310, and the dense layers 312) using the calculated loss. Furthermore, for every epoch, a validation step may be performed. The ML - Al model 306 may evaluate itself on the validation set by making predictions and computing the validation loss.
[0106] In other words, for each epoch, a training loop may be performed. The loop may iterate through the training dataset in batches. For each batch, a forward pass may be performed. For instance, the model 306 takes the batch of adjacency matrices and static matrices as input, then makes predictions on the output using the GCN layers 308 and the LSTM layers 310. Then, a loss calculation is performed. The model 306 may calculate the mean squared error (MSE) between the predicted changes in connectivity and the actual values from the training data. Then, a backward pass is performed. For instance, using gradient descent, the model 306 may update its weights / parameters to minimize the loss. Following, an epoch summary is performed. For instance, after processing all batches in an epoch, the training loss and validation loss are computed. In the validation loop, after each epoch, the model processes the validation set. It may compute the validation loss to monitor how well the model is generalizing. The validation loss may be important to avoid overfitting.
[0107] In some examples, for each iteration, the losses may be calculated for each region of the brain. For instance, the ML - Al model 306 may generate an output 314 indicating predictedLevdig 774186HHS E-220-2024-0-PC-0129 data for one or more regions of the brain (e.g., predicted jackknife data). Afterwards, the computing platform 108 may compare the predicted data for the particular region(s) of the brain with groundtruth data. Based on the comparison, the computing platform 108 may determine a computed loss for the region(s) of the brain. As such, in the example above with 31 ROIs, the computing platform 108 may determine 31 computed losses, and compute an overall loss based on the 31 computed losses. In some variations, a scalar weight may be applied to one or more of the computed losses. For instance, some regions of the brain may be more impactful than other regions of the brain. Thus, certain ROIs may be weighed higher than other ROIs. As such, to compute the overall loss, the computing platform 108 may apply weights to the computed losses (e.g., the 31 losses), and compute an overall loss based on the applied weights. Subsequently, the computing platform 108 may apply gradients and update the weights / parameters of the model 306 based on the overall loss.
[0108] As such, the ML - Al model 306 may combine both dynamic and static connectivity information, where the static matrix is concatenated with the dynamic information before being passed into the ML - Al model 306, giving the ML - Al model 306 a better understanding of the underlying structure.
[0109] To put it another way, the computing platform 108 may process input data, including static data (e.g., the 2D matrix indicating correlation strengths between regions of the brain) and dynamic data (e.g., jackknife datasets such as a 3D tensor indicating the correlation strengths between regions of the brain over a plurality of time steps based on removing one or more entries from the static data), using the ML - Al model 306. For instance, the GCN layers 308 may process each time step of the jackknife datasets individually to generate and / or update node features for each of the regions of the brain indicated by the static data. Afterwards, the node features are provided to the LSTM layer 310 to generate an LSTM output. The LSTM output is provided to the dense layers 312 and generates an ML - Al model output.
[0110] The model output may be and / or indicate predicted connectivity correlations between pairs of ROIs, and the evolution of the predicted connectivity correlations over time. For example, the model output may be and / or include predicted jackknife datasets indicating correlation strengths over a plurality of time steps, such as the predicted jackknife datasets shown in FIGs. 6- 8. For instance, the model output may be provided as a predicted 3D tensor that includes a plurality of predicted jackknife datasets (e.g., 2D matrices). Each of the predicted jackknife datasets mayLevdig 774186HHS E-220-2024-0-PC-0130 indicate correlations between different regions of the brain for a particular subject over a number of time steps (e.g., the evolution of the correlation strength for the regions of the brain over multiple time steps). Following, the computing platform 108 may compare the predicted jackknife datasets with groundtruth data to compute losses. The groundtruth data may include and / or indicate actual jackknife datasets that were obtained from fMRI data for one or more other subjects). For example, initially, the MRI training computing system 102 (e.g., an MRI device) may obtain training data for the first subject (e.g., the training fMRI data). Further, the computing platform 108 may obtain fMRI data for one or more additional subjects. The computing platform 108 may use the jackknife process described above to generate jackknife datasets for the additional subjects, and use the generated jackknife datasets as the groundtruth data. Thus, the computing platform 108 may compare the model output with the groundtruth data (e.g., compare the predicted dataset associated with the first subject and groundtruth jackknife datasets associated with one or more additional subjects) to compute one or more loses. After, the computing platform 108 may update the parameters of the ML - Al models 110 using the computed losses. Additionally, and / or alternatively, as mentioned above, the computing platform 108 may calculate losses associated with each region of the brain (e.g., each row of the jackknife dataset). Subsequently, the computing platform 108 may apply a scalar weight to each of the calculated losses, and compute a total loss based on applying the scalar weight.
[0111] After performing a number of epochs (e.g., 100 epochs), the computing platform 108 may determine that the ML- Al model 306 is sufficiently trained. Afterwards, the computing platform 108 may use the trained ML - Al model 306 during an inference phase. For example, the computing platform 108 may receive raw inference fMRI data from the inference MRI computing system 104. The computing platform 108 may generate inference fMRI data indicating correlation strengths between regions of a subject’s brain over a period of time steps based on the raw inference fMRI data. In some examples, the computing platform 108 may receive the inference fMRI data indicating correlation strengths between regions of a subject’s brain over a period of time steps directly from the inference MRI computing system 104. Subsequently, the computing platform 108 may use process 300 to generate an output 314, which may indicate predicted correlations strengths for each region of the brain for a subject or predicted subject for a plurality of time steps. The computing platform 108 may use the output 314 for one or more applications, which are described in further detail below.Levdig 774186HHS E-220-2024-0-PC-0131
[0112] In some examples, the ML - Al model 306 may further accept as input demographic information associated with the subject. For example, the computing platform 108 may receive demographic information associated with the subject such as, but not limited to, gender, age, degree of handedness (e.g., degree that the subject is left or right handed), bi-lingual, has a hearing loss, and / or other information. The computing platform 108 may input the demographic information into the ML - Al model 306 such as inputting the demographic information into the GCN layers 308. The computing platform 108 may use the demographic information to both train the ML - Al model 306 and during the inference phase.
[0113] Referring to FIG. 4, the fMRI data 400 represents the input for the one or more ML- AI models described herein such as a static matrix (e.g. static input) derived from the fMRI data of a subject that includes correlation strengths of each region of a brain (networks). The fMRI data 400 may include simulated training data prior to pre-processing via a jack-knife process. FIG. 4 also depicts an example of a jack-knife dataset 408, such as after removing a time step 410 from the fMRI data 400. The jack-knife dataset 408 may also include correlation strengths of each region of a brain (networks) 404 of the subject at a plurality of time steps 410. The correlation strengths of the jack-knife dataset 408 may represent a recalculation of the correlation strengths after removing one or more entries from data structure 400.
[0114] FIG. 5A is an exemplary process 500 for determining correlation strengths between regions of a brain in accordance with one or more examples of the present application. The process 500 may be performed by a control system or computer such as MRI training computing systems 102, computing platform 108, inference MRI computing system 104, from FIG. 1, the exemplary system and / or device 200 from FIG. 2, and / or other control systems (e.g., mobile computing devices, tablet computers, or other computational and / or control devices, systems, and / or apparatuses). However, it will be recognized that any of the following blocks may be performed in any suitable order, the blocks may be performed by any suitable system, and that the process 500 may be performed in any suitable environment. The descriptions, illustrations, and processes of FIG. 5A are merely exemplary and the process 500 may use other descriptions, illustrations, and processes. The process 500 may represent an inference phase once the ML - Al models are trained using fMRI data of a plurality of subjects across a plurality of time steps as described below with reference to FIGs. 5B-8.Levdig 774186HHS E-220-2024-0-PC-0132
[0115] At block 502, the computing platform 108 receives or otherwise obtains fMRI data for a subject, and the fMRI data indicates correlations and correlation strengths between the regions of the brain for the subject. The fMRI data may be obtained using an MRI image that is configured to capture blood flow that indicates brain activity in different regions of the brain of the subject (e g. mapping brain activity). In some examples, the fMRI data may be task-based fMRI data representing fMRI data that is captured of a subject while they perform a specific task or respond to a particular stimuli. The fMRI data may be transformed or processed to generate matrices that indicate the correlation strengths between different regions of the brain or as a representative of functional connectivity between the regions of the brain of the subject over the plurality of time steps. An example of a small subset of fMRI data is depicted in FIG. 4. In some examples, the fMRI data may be simulated data for a simulated subject that is generated by the computing platform 108 and / or another computing device / system.
[0116] At block 504, the computing platform 108 generates j ack-knife datasets using a jackknife correlation process by removing one or more data points from the fMRI data to generate each of the jack-knife datasets. An example jack-knife dataset is depicted in FIG. 4. The jack-knife correlation process recalculates the remaining correlation strengths to generate a jack-knife dataset (first jack-knife dataset) and the process is iterated for a number of cycles or loops, where each time a different data point is removed and the remaining data points are recalculated to generate a new jack-knife dataset. The number of cycles or loops to iteratively generate the jack-knife datasets may correspond to a defined variable or parameter or may be based on the amount of data for a removed data point. In other words, if the data point selected for removal is a time step, the cycle will keep generating different jack-knife datasets until a jack-knife dataset has been generated by removing each time step at least once.
[0117] At block 506, the computing platform 108 processes the j ack-knife datasets and the fMRI data using one or more ML - Al models to generate predicted correlation strengths for each region of a brain for the plurality of time steps. As described above with reference to FIG. 3B, the one or more ML - Al models may be trained to use the input (e.g., fMRI data and jack-knife datasets) to generate output that corresponds to predicted correlation strengths for each region of a brain for a plurality of time steps.
[0118] At block 508, the computing platform 108 may output the predicted correlation strengths for each region of the subject’s brain. In some examples, the output of the correlationLevdig 774186HHS E-220-2024-0-PC-0133 strengths may be in the same format as the jackknife datasets for the subject and include regions of the brain, predicted correlation strengths as values, and time steps in a two dimensional (2D) matrix or in a 3D tensor. The output may further be represented as a line graph depicting correlation strength for a particular region of the brain or network over a plurality of time steps for a subject or predicted subject. The output of the computing platform 108 that includes the predicted correlation strengths for each region of the subject’s brain may be compared to baseline correlation strengths associated with one or more subjects to identify a deviation. The baseline correlation strengths may be derived or otherwise generated from the fMRI data associated with the one or more subjects. The deviation may be represented as a value and compared, by the computing platform 108 and / or another computer system, to a threshold value. The threshold value may be associated with a particular region of the brain and the value exceeding the threshold value may correspond to a particular condition such as Alzheimer’s or dementia. The computer system may automatically determine or select a treatment plan for the corresponding condition by obtaining the treatment plan from a database of treatment plans that are mapped to corresponding conditions and recommend one of the treatment plans to a user, doctor, or subject.
[0119] In some examples, longitudinal data may be used to determine the trajectory for a subject. For example, the longitudinal data may indicate fMRI data (e.g., raw and / or processed fMRI data) for a subject (or multiple subjects) that is obtained over multiple years (e.g., a lifespan). For instance, the computing platform 108 may obtain fMRI data associated with the subject, and the fMRI data may be obtained from raw fMRI data that is collected over a time span (e.g., multiple years). The computing platform 108 may use the fMRI data that is obtained over the time span to train the Ml - Al model 306. As such, during inference, the computing platform 108 may obtain patient data for a patient and generate an output. Then, the computing platform 108 may compare the output to a trajectory for a healthy individual (e.g., from raw fMRI data that is collected over a time span) to determine whether the trajectory is normal. For instance, based on the trajectory being statistically significant from the baseline trajectory for the healthy individual, the computing platform 108 may determine a prediction, condition, or prognosis of the patient (e.g., whether the patient is in the early stages of dementia).
[0120] FIG. 5B is an exemplary process 600 for training the one or more ML-AI models in accordance with one or more examples of the present application. The process 600 may be performed by a control system or computer such as MRI training computing systems 102,Levdig 774186HHS E-220-2024-0-PC-0134 computing platform 108, Inference MRI computing system 104, from FIG. 1, the exemplary system and / or device 200 from FIG. 2, and / or other control systems (e.g., mobile computing devices, tablet computers, or other computational and / or control devices, systems, and / or apparatuses). However, it will be recognized that any of the following blocks may be performed in any suitable order, the blocks may be performed by any suitable system, and that the process 600 may be performed in any suitable environment. The descriptions, illustrations, and processes of FIG. 5B are merely exemplary and the process 600 may use other descriptions, illustrations, and processes. The process 600 may represent a training phase for the one or more ML - Al models using training fMRI data of a plurality of subjects across a plurality of time steps and generated training jack-knife datasets.
[0121] At block 602, the computing platform 108 may receive, obtain, or otherwise generate training fMRI data for a plurality of subj ects. The training fMRI data indicates correlation strengths between the regions of the brain for a given subject of the plurality of subjects. The training fMRI data (e.g., training dataset) is described above in FIGs. 3A and 3B.
[0122] At block 604, the computing platform 108 may generate training jack-knife datasets (e g., jackknife datasets or data)using a jack-knife correlation process. This is described above.
[0123] At block 606, the computing platform 108 may process the training jack-knife datasets and training fMRI data using one or more ML - Al models to generate predicted correlation strengths for each region of the subject’s brain. For example, the one or more ML - Al models may include graph convolutional network (GCN) layers that process first input (the training jackknife datasets and training fMRI data) to generate first output. The one or more ML - Al models may include long short term memory (LSTM) layers and one or more dense layers. The training of the one or more ML - Al models may include processing the first output by the LSTM layers to generate second output. The second output may be processed by the one or more ML - Al models using the dense layers to generate third output. The one or more ML - Al models may be trained using the first, second, and third output. The third output may correspond to the predicted correlation strengths for each region of the subj ect’ s brain. This is described in further detail above.
[0124] At block 608, the computing platform 108 may compute losses using the generated predicted correlation strengths and groundtruth data. For example, the computer system may compute a first loss by comparing correlation strengths from a first brain region that is output from the ML - Al model with correlation strengths from the first brain region from the groudntruth data.Levdig 774186HHS E-220-2024-0-PC-0135For example, as depicted in FIG. 6, the predicted correlation strengths may be compared to ground truth or true values for a test subject for a specific brain region. FIG. 6 depicts a line graph representing the predicted correlation strength compared to the correlation strength derived from the training fMRI data for a particular region of the brain over a plurality of time steps. The difference in the values across the time steps may be used to compute the first loss. Similarly, the computer system may compute a second loss by comparing correlation strengths from a second brain region that is output from the ML - Al model with correlation strengths from the second brain region from the groundtruth data. For example, as depicted in FIG. 7, the predicted correlation strengths may be compared to ground truth or true values for a test subject for a specific and different brain region from FIG. 6. FIG. 7 depicts a line graph representing the predicted correlation strength compared to the correlation strength derived from the training fMRI data for a particular region of the brain over a plurality of time steps. The difference in the values across the time steps may be used to compute the second loss. Multiple losses may be generated across all the regions of the brain as depicted in FIG. 8 which illustrates the comparison of predicted correlation strengths to true values of correlation strengths for all the regions of the brain of a subject.
[0125] At block 610, the computing platform 108 may execute a backward pass or perform back propagation to update parameters of the one or more Al models. The training of the one or more ML - Al models may include updating parameters of the one or more ML - Al models based on the first loss and / or the second loss. The updated of parameters may include adjusting or updating weights of the GCN layers, LSTM layers, or dense layers. The backwards pass may use gradient descent to update the weights of the ML - Al models (e.g., GCN layers, LSTM layers, and / or dense layers). Subsequently, process 600 may repeat one or more epochs. This is described in further detail above.
[0126] It will be appreciated that the figures of the present application and their corresponding descriptions are merely exemplary, and that the application is not limited to these exemplary situations.
[0127] It will further be appreciated by those of skill in the art that the execution of the various machine-implemented processes and steps described herein may occur via the computerized execution of processor-executable instructions stored on a non-transitory computer-readable medium, e.g., random access memory (RAM), read-only memory (ROM), programmable read-Levdig 774186HHS E-220-2024-0-PC-0136 only memory (PROM), volatile, nonvolatile, or other electronic memory mechanism. Thus, for example, the operations described herein as being performed by computing devices and / or components thereof may be carried out by according to processor-executable instructions and / or installed applications corresponding to software, firmware, and / or computer hardware.
[0128] The use of the term “at least one” followed by a list of one or more items (for example, “at least one of A and B”) is to be construed to mean one item selected from the listed items (A or B) or any combination of two or more of the listed items (A and B), unless otherwise indicated herein or clearly contradicted by context. The terms “comprising,” “having,” “including,” and “containing” are to be construed as open-ended terms (i.e., meaning “including, but not limited to,”) unless otherwise noted. Recitation of ranges of values herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein, and each separate value is incorporated into the specification as if it were individually recited herein. All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (e.g., “such as”) provided herein, is intended merely to better illuminate the application and does not pose a limitation on the scope of the application unless otherwise claimed. No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the application.
[0129] It will be appreciated that the examples of the application described herein are merely exemplary. Variations of these examples may become apparent to those of ordinary skill in the art upon reading the foregoing description. The inventors expect skilled artisans to employ such variations as appropriate, and the inventors intend for the application to be practiced otherwise than as specifically described herein. Accordingly, this application includes all modifications and equivalents of the subject matter recited in the claims appended hereto as permitted by applicable law. Moreover, any combination of the above-described elements in all possible variations thereof is encompassed by the application unless otherwise indicated herein or otherwise clearly contradicted by context.
Claims
1. Levdig 774186HHS E-220-2024-0-PC-0137CLAIMS:
1. A method for processing functional magnetic resonance imaging (fMRI) data using one or more machine learning - artificial intelligence (ML - Al) models, the method comprising: obtaining blood-oxygen-level-dependent (BOLD) signals associated with changes in blood oxygenation and blood flow in a brain of a subject; determining the fMRI data for the subject based on the BOLD signals, wherein the fMRI data indicates correlation strengths between regions of the brain for the subject; generating jack-knife datasets from the fMRI data using a jack-knife correlation process that removes one or more data points associated with the regions of the brain from the fMRI data to generate each of the jack-knife datasets; processing the jack-knife datasets and the fMRI data using the one or more ML - Al models to generate predicted correlation strengths for each region of the brain for a plurality of time steps; and outputting information associated with the predicted correlation strengths that is generated using the one or more ML - Al models.
2. The method of claim 1, wherein the fMRI data comprises a plurality of entries and each entry indicates a correlation strength between a first region of the brain for the subject and a second region of the brain for the subject, and wherein generating the jack-knife datasets using the jackknife correlation process that removes the one or more data points from the fMRI data comprises: removing at least one entry from the fMRI data; and generating a jack-knife dataset based on recalculating the remaining entries from the fMRI data for the plurality of time steps.
3. The method of claim 1, further comprising: comparing the predicted correlation strengths to baseline correlation strengths to identify a deviation; comparing the deviation with one or more thresholds, wherein the one or more thresholds are associated with one or more regions of the brain and one or more corresponding medical conditions; andLevdig 774186HHS E-220-2024-0-PC-0138 determining one or more medical conditions for the subject based on comparing the deviation with the one or more thresholds, wherein outputting the information comprises outputting information associated with the one or more medical conditions for the subject.
4. The method of claim 3, further comprising: generating a biomarker associated with the medical condition using the deviation, and wherein outputting the information comprises outputting information associated with the biomarker.
5. The method of claim 1, wherein determining the fMRI data based on the BOLD signals comprises: processing raw fMRI data indicating the BOLD signals to generate the fMRI data based on removing noise and motion artifacts from the raw fMRI data.
6. The method of claim 5, wherein the raw fMRI data of the subject is acquired using a magnetic resonance imaging (MRI) system configured to identify the regions of the brain where blood flow indicates activity.
7. The method of claim 1, further comprising: obtaining training fMRI data associated with a plurality of subjects, wherein the training fMRI data comprises static data associated with the plurality of subjects; and prior to determining the fMRI data for the subject, training the one or more ML - Al models using the training fMRI data.
8. The method of claim 7, wherein the one or more ML - Al models comprise one or more graph convolutional network (GCN) layers, and wherein training the one or more ML - Al models comprises: generating training jack-knife datasets for a first subject from the plurality of subjects using first static data for the first subject, wherein the first static data indicates correlation strengths between different regions of a brain of the first subject;Levdig 774186HHS E-220-2024-0-PC-0139 processing the training jack-knife datasets and the first static data using the one or more GNC layers to generate a plurality of node features for a plurality of nodes, wherein each of the plurality of nodes is associated with a region of the brain of the first subject; and training the one or more ML - Al models based on the plurality of node features.
9. The method of claim 8, wherein the one or more ML - Al models further comprise one or more long short term-memory (LSTM) layers and one or more dense layers, and wherein training the one or more ML - Al models further comprises: processing the plurality of node features for the plurality of nodes using the LSTM layers to generate an LSTM layer output; and processing the LSTM layer output using the one or more dense layers to generate an ML - Al output, wherein training the one or more ML - Al models is further based on using the ML - Al output.
10. The method of claim 9, wherein training the one or more ML - Al models is further based on using the ML - Al output comprises: comparing the ML - Al output with groundtruth data; and updating parameters of the one or more ML - Al models based on the comparison.
11. The method of claim 10, wherein comparing the ML - Al output with the groundtruth data comprises: computing a first loss by comparing correlation strengths from a first brain region indicated by the ML - Al output with correlation strengths from the first brain region indicated by the groundtruth data; and computing a second loss by comparing correlation strengths from a second brain region indicated by the ML - Al output with correlation strengths from the second brain region indicated by the groundtruth data, wherein updating the parameters of the one or more ML - Al models is based on the first computed loss and the second computed loss.
12. The method of claim 1, wherein the fMRI data is task-based fMRI data.Levdig 774186HHS E-220-2024-0-PC-014013. A computing platform for processing functional magnetic resonance imaging (fMRI) data using one or more machine learning - artificial intelligence (ML - Al) models, the computing platform comprising: one or more processors; and a non-transitory computer-readable medium having processor-executable instructions stored thereon, wherein the processor-executable instructions, when executed by the one or more processors, facilitate: obtaining blood-oxygen-level-dependent (BOLD) signals associated with changes in blood oxygenation and blood flow in a brain of a subject; determining the fMRI data for the subject based on the BOLD signals, wherein the fMRI data indicates correlation strengths between regions of the brain for the subject; generating jack-knife datasets from the fMRI data using a jack-knife correlation process that removes one or more data points associated with the regions of the brain from the fMRI data to generate each of the jack-knife datasets; processing the jack-knife datasets and the fMRI data using the one or more ML - Al models to generate predicted correlation strengths for each region of the brain for a plurality of time steps; and outputting information associated with the predicted correlation strengths that is generated using the one or more ML - Al models.
14. The computing platform of claim 13, wherein the fMRI data comprises a plurality of entries and each entry indicates a correlation strength between a first region of the brain for the subject and a second region of the brain for the subject, and wherein generating the jack-knife datasets using the jack-knife correlation process that removes the one or more data points from the fMRI data comprises: removing at least one entry from the fMRI data; and generating a jack-knife dataset based on recalculating the remaining entries from the fMRI data for the plurality of time steps.
15. The computing platform of claim 13, wherein the processor-executable instructions, when executed by the one or more processors, further facilitate:Levdig 774186HHS E-220-2024-0-PC-0141 comparing the predicted correlation strengths to baseline correlation strengths to identify a deviation; comparing the deviation with one or more thresholds, wherein the one or more thresholds are associated with one or more regions of the brain and one or more corresponding medical conditions; and determining one or more medical conditions for the subject based on comparing the deviation with the one or more thresholds, wherein outputting the information comprises outputting information associated with the one or more medical conditions for the subject.
16. The computing platform of claim 15, wherein the processor-executable instructions, when executed by the one or more processors, further facilitate: generating a biomarker associated with the medical condition using the deviation, and wherein outputting the information comprises outputting information associated with the biomarker.
17. The computing platform of claim 15, wherein the processor-executable instructions, when executed by the one or more processors, further facilitate: obtaining training fMRI data associated with a plurality of subjects, wherein the training fMRI data comprises static data associated with the plurality of subjects; and prior to determining the fMRI data for the subject, training the one or more ML - Al models using the training fMRI data.
18. The computing platform of claim 17, wherein the one or more ML - Al models comprise one or more graph convolutional network (GCN) layers, and wherein training the one or more ML - Al models comprises: generating training jack-knife datasets for a first subject from the plurality of subjects using first static data for the first subject, wherein the first static data indicates correlation strengths between different regions of a brain of the first subject; processing the training jack-knife datasets and the first static data using the one or more GNC layers to generate a plurality of node features for a plurality of nodes, wherein each of the plurality of nodes is associated with a region of the brain of the first subject; andLevdig 774186HHS E-220-2024-0-PC-0142 training the one or more ML - Al models based on the plurality of node features.
19. The computing platform of claim 13, wherein the fMRI data is task-based fMRI data.
20. A non-transitory computer-readable medium for processing functional magnetic resonance imaging (fMRI) data using one or more machine learning - artificial intelligence (ML - Al) models, the non-transitory computer-readable medium having processor-executable instructions stored thereon, wherein the processor-executable instructions, when executed, facilitate: obtaining blood-oxygen-level-dependent (BOLD) signals associated with changes in blood oxygenation and blood flow in a brain of a subject; determining the fMRI data for the subject based on the BOLD signals, wherein the fMRI data indicates correlation strengths between regions of the brain for the subject; generating jack-knife datasets from the fMRI data using a jack-knife correlation process that removes one or more data points associated with the regions of the brain from the fMRI data to generate each of the jack-knife datasets; processing the jack-knife datasets and the fMRI data using the one or more ML - Al models to generate predicted correlation strengths for each region of the brain for a plurality of time steps; and outputting information associated with the predicted correlation strengths that is generated using the one or more ML - Al models.