Methods and systems for identifying correlates of cognitive function

Abstract

Disclosed is a computer-implemented method for identifying correlates of cognitive function. The method uses a data set comprising neurophysiological signals collected from a plurality of subjects performing one or more cognitive tasks, each cognitive task causing a predetermined change in one or more of the correlates over a duration which the corresponding task is performed. The data set is segmented into a plurality of blocks, some of which are selected for training, where blocks from the start of the data set (temporally) are taken to indicate an opposite cognitive state, or correlate, to blocks from the end of the data set. A machine learning model is trained on the selected blocks, to distinguish between correlates. Further neurophysiological signals can then be categorised and outputted with a confidence probability associated with an accuracy of the categorisation.

Description

METHODS AND SYSTEMS FOR IDENTIFYING CORRELATES OF COGNITIVE FUNCTIONTechnical Field

[0001] The present invention relates, in general terms, to the field of human factors engineering and simulation. More specifically, the invention relates to, but is not limited to, methods and systems for identifying correlates of declining cognitive function that may adversely affect real-world task performance.Background

[0002] Moderate levels of stress motivate a person to perform better and increase productivity. However, excessive durations and / or intensities of stress exposure may lead to distress, which in turn results in a decrease in performance. In some cases, high-pressure working environments cause stress at levels that negatively affect an individual's wellbeing and cause mistakes. An example of a high- pressure environment is a 24-hour command-and-control centre, in which officers are constantly required to juggle several tasks consecutively, such as, receiving incoming calls, extracting important information from callers, inputting caller details and emergency information into an incident management system, performing emergency severity triaging, providing advice, and dispatching the nearest and quickest relevant resources to assist callers. Officers often also navigate multiple computer screens, loaded with extensive realtime information, to prioritise and respond to emergencies in a timely manner. High call volume may cause stress levels to peak.

[0003] Stress levels are typically measured using subjective questionnaires, or through biomarkers such as heart rate variability(HRV), skin conductance, blood pressure, pupillary response and cortisol levels. However, it is impractical for people to regularly fill in a form or survey to report their stress levels. Often, stress levels are better measured in real-time, such as by using wearable devices, without interruption to work. However, there is presently a lack of a reliable and quantitative means to identify and measure fatigue, inattention, stress and other (FISO) correlates of declining cognitive performance. For instance, the design of experiments may not accurately capture the appropriate FISO states. Self-assessments may be subject to personal judgements. Moreover, existing methods require a personalised stress or fatigue baseline for each individual. This is impractical to implement over large workforces and populations. It also delays implementation since the baseline information must be collected and analysed prior to implementation.

[0004] Electroencephalogram (EEG) is typically used to robustly and accurately record neural signals which correlate with motor imagery, affective states, epileptic seizures and other factors. It typically requires full-head gel-based EEG acquisition and may require higher order computational power, making them unsuitable for ease-of-use and real-time processing. An electrocardiogram (ECG) monitor is used for recording heart rate (HR) and HRV, both of which are well- established physiological markers which may indirectly quantify stress. However, the ECG appear to show little correlation with other correlates of FISO.

[0005] There is hence a need for a set of computationally efficient and representative correlates, for accurate FISO detection, and reliable methods and systems for identifying correlates of cognitive function, that can be generally applied without a pre-established, individualised baseline.Summary of the Invention

[0006] Disclosed herein is a computer-implemented method for identifying one or more correlates of cognitive function. The method involves receiving a data set comprising neurophysiological signals. The signals are collected over a duration from each of a plurality of subjects performing one or more cognitive tasks, each cognitive task causing a change in at least one of the correlates. The neurophysiological signals for each subject are segmented into a plurality of blocks and one or both of a first block and a last block in the duration are selected from the neurophysiological data of multiple subjects. These selected blocks are labelled - each selected first block is labelled as indicative of first levels of at least one said correlate and each selected last block as being indicative of second levels of at least one said correlate. The second levels are indicative an opposite state of each said correlate relative to a state indicated by the first levels. Supervised training of a detection model is then performed based on the one or more blocks, to train the detection model to detect the one or more correlates. After training, a further data set comprising further neurophysiological signals collected from a further subject can be subjected to the detection model, to detect the one or more correlates for the further subject. The further subject can then be categorised into one of a plurality of categories based on the one or more correlates, based on an output of the detection model, and a confidence probability associated with an accuracy of the categorisation is then generated.

[0007] The method may further comprise outputting:a risk recommendation for the further subject if the category of the subject corresponds to declining cognitive function and the confidence probability is above a predetermined threshold;a clear recommendation for the further subject if the category of the subject does not correspond to decliningcognitive function and the confidence probability is above the predetermined threshold; anda review recommendation if the confidence probability is below the predetermined threshold.

[0008] Also disclosed is a system for identifying one or more correlates of declining cognitive function, comprising:memory;one or more sensors;a signal processing module, the memory storing instructions that, when executed by the system, cause the system to; receive a data set through the one or more sensors, the data set comprising neurophysiological signals collected over a duration from each of a plurality of subjects performing one or more cognitive tasks, each cognitive task causing a change in at least one of the correlates;use the signal processing module to:segment the neurophysiological signals for each subject into a plurality of blocks;select from the neurophysiological data of multiple said subjects, one or both of a first block and a last block in the duration, and labelling each selected first block as indicative of first levels of at least one said correlate and each selected last block as being indicative of second levels of at least one said correlate, the second levels indicative an opposite state of each said correlate relative to a state indicated by the first levels;perform supervised training of a detection model based on the one or more blocks, to train the detection model to detect the one or more correlates;subject a further data set comprising further neurophysiological signals collected from a further subject, to the detection model, to detect the one or more correlates for the further subject;categorise the further subject into one of a plurality of categories based on the one or more correlates, based on an output of the detection model; andgenerate a confidence probability associated with an accuracy of the categorisation.Brief description of the drawings

[0009] Embodiments of the present invention will now be described, by way of non-limiting example only, with reference to the following figures, in which:

[0010] FIGs. 1(a) and 1(b) provide an illustration of how a cognitive vigilance task (CVT) is carried out. FIG. 1(a) shows a critical number, i.e., 32, included in the trial. FIG. 1(b) shows a no-go trial, in which an indicator appears on the screen to instruct the participant to wait for the trial to time-out.

[0011] FIGs. 2(a) and 2(b) provide an illustration of how a multi-modal integration task (MMIT) is carried out. In FIG. 2(a), all properties of the suspect match the rules, and a participant is instructed to respond by hitting a spacebar. In FIG. 2(b), a participant is instructed to respond by waiting for the trial to time-out.

[0012] FIG. 3 shows a schematic of a high-level architecture of the brain-computer interface (BCI) FISO system.

[0013] FIG. 4 provides an illustration of a smart cognitive monitor.

[0014] FIG. 5 provides an example plot showing trends of engagement, distress and worry for an experiment. The error bars represent the standard error of the mean, and the significance levels were calculated using paired t-tests.

[0015] FIGs. 6(a)-6(c) show plots of engagement, distress and worry, respectively, for CVT. FIGs. 6(d)-6(f) show plots of engagement, distress and worry, respectively, for MMIT. The error bars represent the standard error of the mean, and the significance levels were calculated using paired t-tests. The plots suggest CVT significantly decreased engagement, while MMIT significantly increased distress. All other changes were not significant (a = 0.05).

[0016] FIG. 7 shows a boxplot of classification scores on test data for the three different mental state classification paradigms, namely: attention, fatigue and stress.

[0017] FIG. 8 shows a plot of performance of searching for optimal blocks used for modelling, as a function of the number of rounds. The plot depicts an accuracy of about 86% after about six rounds of search.

[0018] FIG. 9 shows a FISO performance monitoring model.

[0019] FIG. 10 shows a stress detection modelling algorithm.

[0020] FIG. 11 shows a flowchart for a recursive task block.

[0021] FIG. 12 shows a flowchart for a multitude of FISO induction paradigm.Detailed description

[0022] Disclosed herein is a computer-implemented method for identifying one or more correlates of cognitive function. Of particular interest are correlates of declining cognitive function that can be linkedto stress or fatigue levels causing physiological detriment, cognitive impairment or distress. The correlates include factors such as fatigue, stress, attention (or inattention) and alertness (or non-alertness), and combinations of those factors.

[0023] The methodology involves receiving a data set comprising neurophysiological signals collected from a plurality of subjects. The act of "receiving" that data set includes obtaining the data set from memory (e.g., a hard disk or other non-transitory memory), or receiving the dataset from some external device including directly from external devices used to measure the neurophysiological signals.

[0024] The neurophysiological signals are electrical, chemical or mechanical measurements taken from each subject, that are indicative of nervous system function that itself can be used to infer brain or cognitive function, and subject health. The neurophysiological signals are captured as each subject performs multiple cognitive tasks. While, in some embodiments, subjects may only perform a single task, in general it will be more informative if subjects perform multiple tasks, preferably in a series. The series of tasks, when performed consecutively, enable measurement of changes in the neurophysiological signals (and thus of cognitive performance or decline), and thus in the correlates, from one task to the next in the series.

[0025] The cognitive tasks are designed to induce a change in one or more of the correlates over time. For example, over the duration of a series of cognitive tasks, it is expected that fatigue will increase as will stress levels, and alertness and attention will decline. Each cognitive tasks may correspond to a fatigue-inducing task, a stress-inducing task, an attention-modulating task, and an alertness-testing task, or combinations thereof. Thus, each neurophysiological signal corresponds to the subject's response to multiple such tasks, each ofwhich may be designed to induce a similar response (e.g., fatigue or stress) or a different response.

[0026] After receiving the data set, the method involves pre-processing that data set to facilitate subsequent analysis. As the neurophysiological signals are time series data, pre-processing generally involves segmenting the neurophysiological signals into blocks. The blocks are temporal blocks, in which a length (i.e., a duration) of each block corresponds to a fixed period of time. In some embodiments, the length of the blocks varies. For example, the length of each block may correspond to the duration of a cognitive task. The blocks may overlap or be non-overlapping series.

[0027] An intention of the segmenting step is to enable a detection model to differentiate between blocks or epochs indicative of different phases in a subject's experience, during performance of the cognitive tasks. For example, at the start of a series of tasks, the subject may be relaxed and attentive - i.e., a relaxed phase. As task difficulty increases, the subject may begin to feel stressed and have greater difficulty focussing - i.e., a stressed phase. As difficulty peaks and the subject moves towards the end of the tasks in the series, the subject may feel distress and inattentiveness due to information overload - i.e., a distressed phase. Thus, the different phases correspond to different cognitive states of the subject.

[0028] To identify the commencement of a particular phase, the cognitive tasks, and thus the neurophysiological signals measured during performance of those tasks, may be aligned with known experimental phases. This can be achieved by assessing changes in cognitive performance using traditional methods, for a series of cognitive tasks, and assuming similar changes in cognitive performance across individuals performing the present series of cognitive tasks. The invention thus maximizes the likelihood that eachblock corresponds to different stress levels, thereby reducing noise in the training process.

[0029] The neurophysiological signals can be captured using any known method, such as Functional Magnetic Resonance Imaging (fMRI), Event-Related Potentials (ERPs), Electromyography (EMG), Electrocardiogram (ECG / EKG), Galvanic Skin Response (GSR) and electroencephalography (EEG) signals. In some embodiments, EEG (or other) recordings are segmented into blocks, each block corresponding to a phase of the respective subject's experience during performance of the one or more tasks. By identifying blocks containing multiple EEG epochs of time steps, each block length being determined experimentally (e.g., to correspond to a particular phase or task), the different phases of a participant's experience may be identified, and changes in their cognitive states between phases reflected in interblock signal neurophysiological signal variations. For CVT and MMIT, each CVT and MMIT is segmented into about 40 to about 50 blocks, each block having about ten trial questions. Segmenting the EEG recordings into corresponding blocks, corresponding to known experimental phases, noise may be minimised and a likelihood that each block corresponds to different stress levels increases.

[0030] In some implementations, all blocks may be used for training, with labels corresponding to assumed phases - i.e., labels are determined by aligning blocks with experimentally determined or known phases, and labelling the blocks based on the labels of those known phases. In some embodiments, the method involves selecting one or both of the first block and last block for each subject. Labels can be applied to the first and last blocks, with the label applied to the first block indicating opposite levels of the one or more correlates to the levels corresponding to the label applied to the last blocks. In effect, the selected first blocks are indicative of an opposite state of each correlate, to the state represented or indicated by the last blocks - e.g.,the correlate of "attention" being an opposite physiological or cognitive state to the correlate of "inattention" or "distracted ness". In other words, the first and last blocks are assumed to indicate opposite correlates (stressed v relaxed) or opposite values of the same correlate (stressed v 'not' stressed). This labelling procedure is based on the assumption that a subject will start performing a set of cognitive tasks in a relaxed, destressed, attentive state, and stress and inattention will increase, peaking towards the end of the set of cognitive tasks.

[0031] A detection model can then undergo supervised training on the selected blocks, to enable the detection model to detect the correlates, and changes therein, indicative of declining cognitive function or other changes in cognitive function. By distinguishing between states based on the time blocks are measured during performance of tasks, the detection model becomes subject agnostic - i.e., it does not need to develop a subject-specific baseline. To enhance the detection model's ability to distinguish between phases, and thereby identify changes in cognitive performance over performance of a set of tasks, the detection model may be trained in two stages. In the first stage, a binary classifier labels a start state (low stress, high attention, etc) and an end state (high stress, low attention, etc) by labelling first and last blocks as set out above. The binary classifier then employs a recursive approach, to search for more optimal blocks. In a first iteration, that search may involve comparing each block to the first block and last block, to identify a first candidate block that is most similar to the first block while being least similar to the last block and, conversely, a second candidate block that is most similar to the last block while being least similar to the first block. If the first candidate block is less similar to the last block, than the first block to the last block, then the first candidate block is a more optimal block to be indicative of the rest, start or low stress state (optimal rest block). Conversely, if the second candidate block is less similar to the first block, than the last block tothe first block, then the second candidate block is a more optimal block to be indicative of the distress or end state (optimal stress block).

[0032] In some embodiments, the first block is an average of all first blocks selected from the subjects and the last block is an average of all last blocks selected from the subjects. In this case, the search for more optimal blocks is against the averaged first block and averaged last block.

[0033] Similarity between blocks can be tested in a variety of ways. For example, similarity may be measured by Euclidean distance between blocks, cross-correlation methods, time or frequency domain analysis, and any other applicable method.

[0034] FIG. 9 shows an example embodiment of a FISO performance monitoring, and FIG. 10 shows how model optimisation with recursive task block selection is being carried out.

[0035] Advantageously, such a selection process ensures only the most representative blocks for each state are used, thereby enhancing the model performance. For example, as illustrated in FIG. 10, a stressinducing task (e.g., CVT or MMIT) is intentionally designed to increase stress responses. The beginning of a task session is considered to correspond to a no- to low-stress state, while the ending of a task session is considered to correspond to a stress state. Consequently, each subject contributed distinct sets of EEG data from each of the tasks, both labelled for supervised stress classification. To reduce the non-stress- related noise contained in the EEG data, the task blocks most strongly associated with the feature being decoded (i.e. stress) are identified as set out above, and these blocks are used only for training and evaluation. This enables an optimal number of task blocks for constructing the training data set for optimal stress detection.

[0036] After the optimal blocks are identified, the second training stage involves training the detection model to categorise blocks into differentcategories, based on the similarity of those blocks with the optimal stress and rest blocks. Categorisation may involve two categories - e.g., rest and stress - or more than two categories - e.g., rest, stress and categories therebetween based on non-overlapping ranges of similarity and dissimilarity between individual blocks and the optimal rest and stress blocks. During training, the detection model may be tasked with categorising the blocks into multiple categories by predicting a label for each block based on the optimal rest and stress blocks, and comparing the predicted labels with a label assigned to each block based on known phases as set out above. Differences between predicted and assigned labels are associated with a loss that is propagated through layers of the detection model.

[0037] In some embodiments, a leave-one-subject-out validation methodology is used fortraining the detection model (whether the first and last blocks are used directly for training, or the binary classifier is used to first identify optimal blocks for training). This may involve partitioning the data into training and testing sets using a N-l versus the remaining one subject split, ensuring a comprehensive evaluation and validation of the algorithm's performance. Advantageously, the leave-one-subject-out approach allows training of a general crossparticipant model for predicting FISO mental states, by training a classifier on data from all participants except one, then testing it on the omitted participant. Consequently, the classifier was not trained on data from the left-out participant but must predict its mental state. By repeating this for each participant in the data set and rotating the left- out participant each time, the classification accuracies may be averaged across all iterations.

[0038] According to an aspect of the invention, several widely recognised regression algorithms may be used for training the detection model, including Random Forest, Gradient Boosting, Support Vector Machine, K-Nearest Neighbour, and Linear Regressor, amongothers. By exploring a range of methodologies, a comprehensive evaluation of the data is achieved and the potential for identifying optimal predictive models is maximised.

[0039] The detection model is also configured to generate a confidence score for the classification assigned to each subject's neurophysiological signals. The confidence score may be determined by assigning a probability of each category (stress, rest, and potentially other categories therebetween) to the neurophysiological signal, the selected category having highest probability, and the confidence score being based on the highest probability (e.g., high confidence if the label is assigned with greater than 90% confidence), or the probability associated with the second highest category (e.g., if the second most probable category has a probability within 10% of the highest probability, then confidence may be low).

[0040] Based on the confidence score and the category, the detection model may output a recommendation. That recommendation may be a risk recommendation for the further subject. This may occur if the category of the subject corresponds to declining cognitive function and the confidence probability is above a predetermined threshold. The confidence threshold is applied to ensure the recommendation is accurate. The recommendation itself could correspond, for example, to a recommendation not to hire a subject for a particular high-stress job if the subject's category indicates a drop in performance sufficient to cause a mistake (which can sometimes be deadly) and / or undesirable outcome in a high-stress environment. This recommendation may also indicate that the subject's workload or working environment needs to be adapted to enhance the subject's mental well-being. An alternative recommendation could be a clear recommendation where the subject does not show cognitive decline (including cognitive decline that is not significant enough to erode performance during high-stress tasks) and the confidence probability is above the predetermined threshold. Areview recommendation may also be applied, to prompt clinical review where the confidence probability is below the predetermined threshold. Low confidence indicates that the label may be inaccurate, and that a recommendation based on the inaccurate label may result in the wrong recommendation being made.

[0041] Based on the foregoing, the method involves a decision-making process in which the system implementing the method categorises the person and, based on the confidence of that categorisation, makes a recommendation. The categorisation method and thus the confidence in that categorisation, is developed within the system and may not be available or explainable, depending on the model selected for the detection model. As such, a decision (categorisation) is made by the system based on parameters that are potentially unknown to the user, and that decision is critically analysed by the system itself, such that any resulting recommendation is automatically caveated - i.e., based on the confidence of the decision.

[0042] After training, the detection model can be fed a further data set comprising further neurophysiological signals collected from a further subject. The detection model will categorise the further subject into one of the categories, and a confidence score will be generated forthat categorisation. As set out above, a recommendation may also be generated, to facilitate decision-making based on the category of the further subject.

[0043] Advantageously, embodiments of the claimed invention enable monitoring of FISO indicators of multiple subjects, without a need to develop a subject-specific model. The general model may be used on further subjects (i.e., neurophysiological signals that did not form part of the training data set) without pre-calibration, using a data driven approach from neurophysiological signals collected from multiple subjects.

[0044] FIG. 12 shows a flowchart for a multitude of FISO induction states. Some states may be induced by a fatigue task that is an adapted CVT task - typically monotonous yet cognitively demanding. Other states may be induced by a stress task that is an MMIT - typically engaging and immersive, to simulate neurophysiological effects of decision-making under pressure. Attention task may comprise both the CVT and MMIT performed at resting state, to test for inattentiveness. Alert tasks may comprise alternating go and no-go trials, to instruct participants to not respond to no-go trials, integrating both the fatigue task and the stress task, to test for user alertness. Notably, a task may comprise a single trial, or multiple trials. Where tasks are adaptive, a more difficult tasks may be provided if the subject passes the single trial or achieves greater than a predetermined accuracy where a task involves multiple trials. Thus, the difficulty of a "next" task in a series of tasks may be based on the accuracy of response to the "previous" or "current" task, such that greater accuracy results in greater difficulty of the next task and lower accuracy results in lower difficulty of the next task.

[0045] The fatigue-inducing task may be an adapted cognitive vigilance task (CVT). CVT measures a subject's capacity to maintain focus and behavioural alertness over an extended, often monotonous, period, typically involving detection of infrequent and sporadic target stimuli. FIGs. 1(a) and 1(b) illustrate an example of how CVT may be carried out. For example, when a participant is subjected to the CVT, the participant monitors an occurrence of critical signals on a screen. Participants may, for example, be required to detect critical signals which appear in two-digit numbers in which the first digit and the second digit differ by either 0 or 1. For each trial, a panel of eight two- digit numbers may be shown, with participants being required to detect critical numbers which occur infrequently, either in only one of the eight numbers, or not at all. Participants may be required to press a spacebar if a single critical number was present (i.e., a positive trial) as shownin FIG. 1(a), and to press nothing, letting the trial go to time-out, if there were no critical numbers being detected (i.e., a negative trial).

[0046] In addition, a go or no-go task may be integrated into the CVT, by introducing visual or aural cues, instructing subjects not to respond to certain trials (FIG. 1(b)). Advantageously, the no-go task increases the complexity of the task by requiring participants to practice inhibitory control.

[0047] In some embodiments, the stress-inducing task is a multi-modal integration task (MMIT) configured to stimulate neurophysiological effects on the subject. FIGs. 2(a) and 2(b) illustrate an example of how MMIT may be carried out. For example, the participants may be placed in a role of an immigration officer, who must decide whether to admit or deny an entry of procedurally generated characters into a fictional country, based on whether their documents complied with a set of immigration rules. By seeking to explore the neurophysiological effects of decision-making under pressure and cognitive burdens, MMIT creates an engaging and immersive task for the participants.

[0048] In other embodiments, participants of the MMIT may be required to match properties of a procedurally generated suspect (right of FIG.2(a)) against a list of pre-defined rules (left of FIG. 2(a)). Participants may be instructed to press the spacebar or another key if all rules matched (i.e., a positive trial), or to wait for the trial to time-out (i.e., a negative trial).

[0049] In addition, a go or no-go task may be integrated into the MMIT, by introducing visual or aural cues, instructing subjects not to respond to certain trials (for e.g., the grey star in FIG. 2(b)). When a visual or aural cue is present, participants may be instructed to let the trial timeout (i.e., a no-go trial). Advantageously, the no-go task increases the complexity of the task by requiring participants to practice inhibitory control.

[0050] In some embodiments, an adaptive difficulty component is introduced into the MMIT, to promote the continued engagement of each participant, ensuring participants are always adequately cognitively challenged. Advantageously, this accounts for different people with a wide range of mean reaction times. Such an adaptive difficulty component also ensures participants with different levels of cognitive abilities and / or reaction times may experience similar levels of stress.

[0051] The adaptive difficulty component may, for instance, begin a MMIT trial with a trial duration of 5 seconds. When a participant gets 9-10 out of 10 trials correct, the difficulty level of the trial may be raised by decreasing the trial duration by 1 second, to a floor of 1 second, giving the participant less time to interpret each trial and respond accordingly. Conversely, when a participant gets 7 or fewer out of 10 trials correct, the difficulty of the trial may be lowered by increasing the trial duration by 1 second, giving the participant more time to interpret each trial and respond accordingly. The trial duration may remain unchanged when a participant scores exactly 8 out of 10.

[0052] In some embodiments, the attention-modulating tasks are configured to cause progressive inattention over the series of tasks. As such, attention-modulation presumes the subject is starts in a state of calm, then gradually heightens an attention level towards conclusion of the attention-modulating task.

[0053] During each of these tasks (i.e., the CVT and MMIT), participants may wear a non-invasive EEG headband, for recording neurophysiological signals, an ECG monitor to record their heart rate (HR) and heart rate variability (HRV), and / or other devices. Trials may be presented in blocks of ten (or other number), each block containing six positive trials, two negative trials and two no-go trials. Results may be briefly presented to participants, describing how many trials of tenthey had responded correctly to (or, for the case of negative trials and no-go trials, allowed to proceed to time-out).

[0054] In some embodiments, the participant's experiences may be captured by a Dundee Stress State Questionnaire (DSSQ) - e.g., during collection of labels that can be aligned to subsequently captured neurophysiological signals and thereby used in training the detection model. DSSQ measures states of engagement, distress and worry, with engagement corresponding to a measure of attention, distress corresponding to a measure of stress, and worry corresponding to how a participant views their performance. Both engagement and distress may be directly correlated to FISO, and hence, DSSQ may be used as a reliable means for scoring and sorting questions. Table 1 shows an example of the questions and how each question is scored, for the DSSQ.Table 1. Questions and categories for each question, for the DSSQ.

[0055] Preferably, participants complete the DSSQ before, during and after the tasks. By providing sufficient rest time prior to commencing a task, participants are relaxed, and the experimental data corresponding to a starting block may be said to be that of low stress level. Conversely, analysis of the DSSQ suggests an increase in stress level after completion of the CVT or MMIT, indicating that the experimental data corresponding to an end block may be said to be that of high stress level.

[0056] For example, in some experiments, it was observed that as participants performed the tasks, the engagement level decreased, while the distress level increased. The worry level also decreased, though the drop was found to not be statistically significant. Theexperiment was further randomised such that half the participants performed CVT first, while the other half performed MMIT first. The participants then provided their responses to the DSSQ again, and the outcomes suggest CVT and MMIT affect the responses to varying degrees. FIG. 5 illustrates the trends in levels of engagement, distress and worry, with the corresponding significance levels.

[0057] By assembling the trends in how CVT and MMIT affect the engagement, distress and worry levels, the plots suggest performing CVT results in participants reporting significantly lower levels of engagement, with their distress and worry levels remaining generally unchanged. The plots also suggest performing MMIT results in participants reporting significantly higher distress, while their levels of engagement and worry remain constant. While the former suggests participants of CVT may suffer from inattention fatigue, possibly due to the low-intensity and repetitive nature of the task, participants of MMIT may cause participants to experience more stress, due to the higher intensity and cognitively more demanding nature of the tasks.

[0058] Table 2 presents an example of model performance metrics, following a leave-one-subject-out training and testing.Table 2. Model performance metrics, after leave-one-subject-out training and testing.

[0059] FIG. 7 also illustrates a boxplot of classification scores on test data for the three different mental states. The predicted scores on class 1 (i.e. yes) are higher than the predicted scores for class 0 (i.e. no), in the left-out class, indicating that a model built for attention, fatigueand stress may be generalisable to unseen participants and therefore predict the mental states in the participants, accurately. Each box and whisker plot represents a distribution of prediction scores on the left- out participant. FIG. 8 illustrates the performance of searching for optimal blocks, when used in modelling. An accuracy of about 86% was achieved after about six rounds of search.

[0060] FIG. 3 shows a schematic of an example of a high-level architecture of the brain-computer interface (BCI) FISO system, used for developing a BCI model for detecting FISO. The system comprises the following modules: (1) an EEG and an ECG sensor to which the subject is connected, and that interact with the data acquisition module via Bluetooth 1. (2) The data acquisition module receives, via Bluetooth, data packets that is converts into continuous time-series data, for e.g., raw EEG data and ECG data. (3) A signal processing module, that performs quality checks - e.g., applies filters and distortion correction to ensure neurophysiological signals are in a condition for subsequent analysis. (4) A data storage module for storing the neurophysiological data - e.g., raw EEG data and the raw ECG data marked with stim codes and saved into *.cnt files in a binary format. Log files record task performance and are saved into *.log files in a text format. (5) An experimenter GUI, comprising an EEG / ECG viewer and a task controller, allows the experimenter to control the task workflow. (6) A subject GUI, displays the tasks and allows subjects to perform the tasks. (7) A task scheduler, scheduling resting tasks (for e.g., eye open, eye closed) and cognitive tasks (for e.g., CVT and MMIT).

[0061] Table 3 outlines an experimental workflow for participants of the BCI experiment. In an example of the BCI experiment with an estimated total duration of approximately two hours, half the participants were randomly assigned to do CVT followed by MMIT, while the other half of the participants were randomly assigned to do MMIT followed by CVT. Each of the groups of participants undertook a seriesof eight steps, as indicated by SI to S8, with experimental EEG and ECG data being recorded at steps S3, S4 and S7.Table 3. An experimental workflow for participants of the BCI experiment. CVT: Cognitive vigilance task. MMIT: Multi-modal integration task. Each group of participants is subjected to eight steps, indicated S1-S8. EEG and ECG data are collected at steps S3, S4 and S7.

[0062] In some embodiments, and as illustrated in FIG. 4, the BCI further comprises a smart cognitive monitor interface, showing the EEG and ECG signals in real-time, and providing controls for adjusting the signal display and experimental tasks. The monitor interface displays EEG signals, ECG signals, ECG signal statistics and summaries, as well as signal quality measurements and indicators, in real-time.

[0063] In some embodiments, a stress score may be derived from the data, using a Support Vector Regressor (SVR) model, and subsequently employing a majority voting mechanism to ascertain the stress states of the task block. A voting system ensures the system assesses the entirety of each block by examining its epochs. If more than half of the epochs are within a block indicating stress, the overall state of the block would be classified as such. Conversely, if less than half of the epochs are within a block indicating stress, the overall state of the block would be classified as "not stressed". An SVR classifier may be trained on all but one participant and tested on the left-out participant.This process may be repeated for every participant. FIG. 11 provides a flowchart for such a recursive task block.

[0064] It will be appreciated that many further modifications and permutations of various aspects of the described embodiments are possible. Accordingly, the described aspects are intended to embrace all such alterations, modifications and variations that fall within the spirit and the scope of the appended claims.

[0065] Throughout this disclosure, and unless the context requires otherwise, the word "comprise" and variations such as "comprises" and "comprising" will be understood to imply the inclusion of a stated integer or step or group of integers or steps, but not the exclusion of any other integer or step or group of integers or steps.

[0066] The reference to any prior art in this disclosure is not, and should not be taken as, an acknowledgement or any form of suggestion that the prior art forms part of the common general knowledge.

Claims

CLAIMS1. A computer-implemented method for identifying one or more correlates of cognitive function, the method comprising:receiving a data set comprising neurophysiological signals collected over a duration from each of a plurality of subjects performing one or more cognitive tasks, each cognitive task causing a change in at least one of the correlates; segmenting the neurophysiological signals for each subject into a plurality of blocks;selecting, from the neurophysiological data of multiple said subjects, one or both of a first block and a last block in the duration, and labelling each selected first block as indicative of first levels of at least one said correlate and each selected last block as being indicative of second levels of at least one said correlate, the second levels indicative an opposite state of each said correlate relative to a state indicated by the first levels; performing supervised training of a detection model based on the one or more blocks, to train the detection model to detect the one or more correlates;subjecting a further data set comprising further neurophysiological signals collected from a further subject, to the detection model, to detect the one or more correlates for the further subject;categorising the further subject into one of a plurality of categories based on the one or more correlates, based on an output of the detection model; andgenerating a confidence probability associated with an accuracy of the categorisation and.

2. The method of claim 1, further outputting:a risk recommendation for the further subject if the category of the subject corresponds to declining cognitive function and the confidence probability is above a predetermined threshold;a clear recommendation for the further subject if the category of the subject does not correspond to declining cognitive function and the confidence probability is above the predetermined threshold; anda review recommendation if the confidence probability is below the predetermined threshold.

3. The method of claim 2, wherein each correlate comprises one or more of fatigue, stress, inattention, and non-alertness.

4. The method of claim any one of claims 1 to 3, wherein segmenting the neurophysiological signals for each subject into a plurality of blocks comprises segmenting electroencephalography (EEG) recordings into blocks, each block corresponding to a phase of the respective subject's experience during performance of the one or more tasks.

5. The method of any one of claims 1 to 4, wherein performing supervised training on the one or more blocks comprises using a binary classifier and labelling the first block and the last block with low values and high values of the one or more correlates, respectively.

6. The method of claim 5, wherein the binary classifier recursively searches for optimal said blocks, based on, for each block for a particular said subject, a similarity of the said block with the first block, or last block, for the respective subject and a dissimilarity of the said block with the last block, of first block, for the respective subject.

7. The method of claim 6, where similarity and dissimilarity is determined by Euclidean distance between the said block and each of the respective first block and last block.

8. The method of any one of claims 1 to 7, wherein the data set comprises neurophysiological signals collected from each said subject performing a series of said cognitive tasks, the series of cognitive tasks correspond to one or more of: fatigue-inducing tasks, stress-inducing tasks, attention-modulating tasks, and alertness-testing tasks.

9. The method of claim 8, wherein the fatigue-inducing tasks are adapted cognitive vigilance tasks (CVT) in which a difficulty of the cognitive tasks in the series is adapted relative to an accuracy of performance of an immediately preceding said task in the series.

10. The method of claim 8, wherein the stress-inducing tasks are multimodal integration tasks (MMIT) configured to stimulate a neurophysiological response of the subject to decision-making under pressure, the neurophysiological data for the subject comprising that neurophysiological response.

11. The method of claim 8, wherein the attention-modulating tasks are configured to cause progressive inattention over the series of tasks.

12. The method of any one of claims 8 to 11, wherein, for the neurophysiological signals for each subject, the first block is labelled as indicative of one or more of low fatigue, low stress, high alertness and high attention, and the last block is labelled as indicative of one or more of high fatigue, high stress, low alertness and low attention.

13. The method of any one of claims 8 to 12, wherein the series of cognitive tasks for at least one subject comprises a no-go task.

14. A system for identifying one or more correlates of declining cognitive function, comprising:memory;one or more sensors;a signal processing module, the memory storing instructions that, when executed by the system, cause the system to; receive a data set through the one or more sensors, the data set comprising neurophysiological signals collected over a duration from each of a plurality of subjects performing one or more cognitive tasks, each cognitive task causing a change in at least one of the correlates;use the signal processing module to:segment the neurophysiological signals for each subject into a plurality of blocks;select from the neurophysiological data of multiple said subjects, one or both of a first block and a last block in the duration, and labelling each selected first block as indicative of first levels of at least one said correlate and each selected last block as being indicative of second levels of at least one said correlate, the second levels indicative an opposite state of each said correlate relative to a state indicated by the first levels;perform supervised training of a detection model based on the one or more blocks, to train the detection model to detect the one or more correlates;subject a further data set comprising further neurophysiological signals collected from a further subject, to the detection model, to detect the one or more correlates for the further subject;categorise the further subject into one of a plurality of categories based on the one or more correlates, based on an output of the detection model; andgenerate a confidence probability associated with an accuracy of the categorisation.

15. The system of claim 14, being configured to:output a risk recommendation comprising one of:a risk recommendation for the further subject if the category of the subject corresponds to declining cognitive function and the confidence probability is above a predetermined threshold;a clear recommendation for the further subject if the category of the subject does not correspond to declining cognitive function and the confidence probability is above the predetermined threshold; and a review recommendation if the confidence probability is below the predetermined threshold.

16. The system of claim 14 or 15, wherein the signal processing module segments the neurophysiological signals for each subject into a plurality of blocks by segmenting electroencephalography (EEG) recordings into blocks, each block corresponding to a phase of the respective subject's experience during performance of the one or more tasks.

17. The system of any one of claims 14 to 17, wherein supervised training on the one or more blocks involves using a binary classifier and labelling the first block and the last block with low values and high values of the one or more correlates, respectively.

18. The system of claim 17, wherein the binary classifier is used to recursively search for optimal said blocks, based on, for each block for a particular said subject, a similarity of the said block with the first block, or last block, for the respective subject and a dissimilarity of the said block with the last block, of first block, for the respective subject.

19. The system of claim 18, wherein, for the neurophysiological signals for each subject, the signal processing module labels the first block as indicative of one or more of low fatigue, low stress, high alertness and high attention, and the last block as indicative of one or more of high fatigue, high stress, low alertness and low attention.

Images