Computer-implemented method, device, and system

Abstract

A computer-implemented method for determining a subject's neurological status by calculating a score indicative of the subject's neurological status. The method comprising steps of: (a) performing an encoding phase, in which the subject is presented with a plurality of items to be recalled in a retrieval phase; (b) obtaining, from the subject, a metacognitive judgement score, and / or performing an intermediate task phase, in which the subject is presented with a task to perform and which results in an intermediate task score; (c) performing the retrieval phase, in which the subject is asked to recall the plurality of items presented in the encoding phase, and which results in a retrieval phase score; and (d) calculating the score indicative of the subject's neurological status based on the retrieval phase score and one or both of: the metacognitive judgement score and the intermediate task score.

Description

COMPUTER-IMPLEMENTED METHOD, DEVICE, AND SYSTEMTECHNICAL FIELDThe present disclosure relates to a computer-implemented method, device, and system.BACKGROUNDThe diagnosis of types of dementia, particularly Alzheimer’s Disease (AD), and the monitoring of a subject’s mental capacity before and after diagnosis is an active and multifaceted area of research. Alzheimer’s disease is a progressive neurological disease whereby excess protein builds up in the brain, impairing neuronal function and eventually leading to cell death. The disease is characterised by continual progression, but the rate of progression is individual. Being able to identify the early stages of Alzheimer’s disease is important, however there are many barriers that make early diagnosis of Alzheimer’s disease difficult. A first is that significant damage to the brain may have already occurred before the disease is detectable by conventional methods (e.g. imaging, regular health checks, etc.). A second is that, typically, neurological assessments (such as the Mini-Mental State Exam - MMSE, or the Alzheimer’s Disease Assessment Scale-Cognitive subscale - ADAS- Cog) take a significant amount of time on the part of the subject and may require clinical resources such as a nurse or physician to undertake the testing.The present disclosure was arrived at in light of the above considerations.SUMMARYAccordingly, in a first aspect, embodiments of the invention provide a computer-implemented method for determining a subject’s neurological status by calculating a score indicative of the subject’s neurological status, the method comprising steps of:(a) performing an encoding phase, in which the subject is presented with a plurality of items to be recalled in a later phase;(b) obtaining, from the subject, a metacognitive judgement score, and / or performing an intermediate task phase, in which the subject is presented with a task to perform and which results in an intermediate task score;(c) performing the retrieval phase, in which the subject is asked to recall the plurality of items presented in the encoding phase, and which results in a retrieval phase score; and(d) calculating the score indicative of the subject’s neurological status based on the retrieval phase score and one or both of: the metacognitive judgement score and the intermediate task score.Such a method is able to accurately and swiftly determine a subject’s neurological status in a selfadministered manner.In some examples, the method includes both obtaining, from the subject, the metacognitive judgement score, and performing the intermediate task phase in which the subject is presented with a task to perform and which results in the intermediate task score. In such examples the calculation of the score indicative of the subject’s neurological status is based on the retrieval phase score, the metacognitive judgment score, and the intermediate task score. Obtaining the metacognitive judgment score may be indexed as step (b) and performing the intermediate task phase may be indexed as step (c), with subsequent steps being reindexed.The metacognitive judgment score can be obtained before, during, or after performing the encoding phase. The metacognitive judgment score could be obtained before, during, or after the intermediate task phase. The metacognitive judgment score could be obtained before, during, or after the retrieval phase.The metacognitive judgment score, intermediate task score, and retrieval phase score may be independently weighted when used to calculate the score. The retrieval phase score may be weighted more heavily than the intermediate task score and the metacognitive judgement score. The intermediate task score may be more lightly weighted than the metacognitive judgement score. The metacognitive judgement score may be more lightly weighted than the retrieval phase score. The metacognitive judgement score may have a weighting in between the weightings of the intermediate task score judgement score and the retrieval phase score.The score indicative of the subject’s neurological status may be referred to as a composite score.The score may be compared to a reference score or range of scores, and the subject may be classified according to this comparison. For example, a score of between a first value and a second value may be indicative that the subject is healthy. A score below the first value may be indicative that the subject has or is likely to develop Alzheimer’s disease. For example, a score of between 55 and 80 may indicate that the subject is healthy with scores below that range indicating the subject has or is likely to develop Alzheimer’s disease. Scores in excess of this range may be indicative of a problem with the testing process (the subject was helped, etc).The score may be calculated based on an equation selected on the basis of the subject’s sex.Components of the retrieval phase score may be weighted based on ordinal position and / or location of each item to be recalled. For example, where the encoding phase is implemented as a grid of items which are, one by one, revealed to the user for encoding, the weight assigned to any given correct answer may be weighted based on: (i) the ordinal position of the item in the order of revealed items; and / or (ii) the position of the item in the grid of items.The neurological status may be a cognitive status of the subject.The metacognitive judgement score may be an indication of how the subject believes they will perform during the retrieval phase. The metacognitive judgment score may be an indication of how well or how poorly they believe will perform during the retrieval phase.The score may be further calculated based on the age of the subject.The intermediate task may be a reaction speed test. The encoding phase may be implemented as a grid of items which are sequentially revealed to the user for encoding. By sequentially, it may be meant that the items are revealed one at a time but not in any order (i.e. , not in a spatial sequence).The computer-implemented method may further comprise transforming the calculated score into a different score indicative of the subject’s neurological status. The computer-implemented method may further comprise a step of calculating a predicted MMSE or ADAS-Cog value from the score. The computer-implemented method may further comprise calculating a predicted score selected from the list comprising: (1 ) Addenbrooke’s Cognitive Examination (ACE-Ill); (2) Montreal Cognitive Assessment (MOCA); (3) The Repeatable Battery for Assessment of Neuropsychological Status (RBANS); (4) The Preclinical Alzheimer’s Cognitive Composite (PACC5); (5) The Alzheimer’s Prevention Initiative Preclinical Cognitive Composite (APCC); (6) The Cambridge Assessment of Memory and Cognition (CAMDEX); (7) The Alzheimer’s disease (AD) composite score (ADCOMS);(8) The Neuropsychological Test Battery (NTB); and (9) The General Practitioner assessment of Cognition (GPCOG). The method may then include a step of classifying the neurological status of the subject according to the corresponding cut-offs in the MMSE or ADAS-Cog, or any of the other predicted scores.The computer-implemented method may further include repeating steps (a), (b), and (c) for a plurality of testing cycles. Step (a) in each testing cycle may be modified to only show the subject the items missed in the step (c) of the preceding cycle. Step (b) in each testing cycle may be modified so that only the intermediate task phase is performed for each testing cycle after the first testing cycle, with the first testing cycle including both the intermediate task phase and the step of obtaining the metacognitive judgement score. Where there are no missing items from the previous step (c), step (a) is skipped as there are no items to display. After repeating steps (a), (b), and (c) for the plurality of testing cycles, the method may further comprise obtaining a further metacognitive judgement score indicative of how the subject believes they will perform during a delayed retrieval phase. The method may then comprise performing the delayed retrieval phase after obtaining the further metacognitive judgement score. The method may further comprise performing one or more filler tasks after obtaining the further metacognitive judgement score and before the delayed retrieval phase.The score may be calculated either according to equation (1): ScOT'ernale 0.606 ■ (10 ■ (-0.0738 ■ (70 - age) + Ecorrect itemsZ, ■ P) +2 ■ (15 - [0.00196 ■ (70 - age) + |3 + jol - Xcorrect items l l ]) +1 ■ (15 + 6 ■ (-0.00828 ■ (70 - age) + 0.5 -) or according to equation (2):ScoreNoTmaie ~ 0.606 ■ (10 ■ (-0.0296 ■ (70 - age) - 1.05 + Xcorrect itemsZ, ■ P) +2 ■ (15 - [-0.00767 ■ (70 - age) + 0.0907 + |3 + jol - £correctitemsl|]) + 1 ■ (15 + 6 ■ (—0.00797 ■ (70 - age) + 0.00177 + 0.5 -) wherein equation (1) is used when the subject is male, and equation (2) is used when the subject is not male, and where age is the age of the subject, L is a weighting for the location in which a given item was presented during the encoding phase, P is a weighting for the order in which a given item was presented during the encoding phase, jol is the metacognitive judgment score, and RT is an average reaction time which is the intermediate task score. In one example, the age of the subject may be calculated based on the difference between the year of the assessment date and the subject’s year of birth (which the subject may be requested to provide).The method may further include communicating the calculated score to a remote device, for example a clinician’s terminal or web portal through which a clinician may access the score. The method may include outputting the calculated score on the device, e.g. via display means.The method may include an initial step of requiring that the subject provide one or more credentials to either register themselves or identify themselves.The calculated score may be referred to as a HiPAL score.In a second aspect, embodiments of the invention provide a device for determining a subject’s neurological status by calculating a score indicative of the subject’s neurological status, the device including a display, a processor, and a user input component; wherein the device further includes memory containing machine-executable instructions which, when executed on the processor, cause the device to:(a) perform an encoding phase, in which a plurality of items to be recalled by the subject in a retrieval phase are displayed via the display;(b) obtain form the subject, via the user input component, a metacognitive judgement score, and / or perform an intermediate task phase using the display and the user input component, in which the user is presented with a task to perform and which results in an intermediate task score;(c) perform the retrieval phase using the display and the user input component, in which the subject is asked to recall the plurality of items presented in the encoding phase, and which results in a retrieval phase score; and(d) calculate the score indicative of the subject’s neurological status based on the retrieval phase score and one or both of: the metacognitive judgement score and the intermediate task score.Such device method is able to accurately and swiftly determine a subject’s neurological status in a self-administered manner.In some examples, the method includes both obtaining, from the subject, the metacognitive judgement score, and performing the intermediate task phase in which the subject is presented with a task to perform and which results in the intermediate task score. In such examples the calculation of the score indicative of the subject’s neurological status is based on the retrieval phase score, the metacognitive judgment score, and the intermediate task score. Obtaining the metacognitive sjudgment score may be indexed as step (b) and performing the intermediate task phase may be indexed as step (c), with subsequent steps being reindexed.The metacognitive judgment score can be obtained before, during, or after performing the encoding phase. The metacognitive judgment score could be obtained before, during, or after the intermediate task phase. The metacognitive judgment score could be obtained before, during, or after the retrieval phase.In calculating the score indicative of the subject’s neurological status, the metacognitive judgment score, the intermediate task score, and the retrieval phase score may be independently weighted. The retrieval phase score may be weighted more heavily than the intermediate task score and the metacognitive judgement score. The intermediate task score may be more lightly weighted than the metacognitive judgement score. The metacognitive judgement score may be more lightly weighted than the retrieval phase score. The metacognitive judgment score may have a weighting in between the weightings of the intermediate task score and the retrieval phase score.In calculating the score, an equation may be used by the processor, and the equation may be selected on the basis of the subject’s sex.The score may be compared by the processor to a reference score or range of scores, and the subject may be classified according to this comparison. For example, a score of between a first value and a second value may be indicative that the subject is healthy. A score below the first value may be indicative that the subject has or is likely to develop Alzheimer’s disease. For example, a score of between 55 and 80 may indicate that the subject is healthy with scores below that range indicating the subject has or is likely to develop Alzheimer’s disease. Scores in excess of this range may be indicative of a problem with the testing process (the subject was helped, etc).Components of the retrieval phase score may be weighted based on ordinal position and / or location of each item to be recalled. For example, where the encoding phase is implemented as a grid of items which are, one by one, revealed to the user for encoding, the weight assigned to any given correct answer may be weighted based on: (i) the ordinal position of the item in the order of revealed items; and / or (ii) the position of the item in the grid of items.The neurological status may be a cognitive status of the subject.AThe metacognitive judgement score may be an indication of how the subject believes they will perform during the retrieval phase. The metacognitive judgment score may be an indication of how well or how poorly they will perform during the retrieval phase.Calculating the score may be further based on the age of the subject.The intermediate task may be a reaction speed test. The encoding phase may be implemented as a grid of items which are, one by one, revealed to the user for encoding.The memory may contain further machine-executable instructions which may cause the device (for example the processor) to transform the calculated score into a different score indicative of the subject’s neurological status. For example, the device may be caused to calculate a predicted MMSE or ADAS-Cog value from the score. The device may be caused to calculate a predicted score selected from the list comprising: (1 ) Addenbrooke’s Cognitive Examination (ACE-Ill); (2) Montreal Cognitive Assessment (MOCA); (3) The Repeatable Battery for Assessment of Neuropsychological Status (RBANS); (4) The Preclinical Alzheimer’s Cognitive Composite (PACC5); (5) The Alzheimer’s Prevention Initiative Preclinical Cognitive Composite (APCC); (6) The Cambridge Assessment of Memory and Cognition (CAMDEX); (7) The Alzheimer’s disease (AD) composite score (ADCOMS);(8) The Neuropsychological Test Battery (NTB); and (9) The General Practitioner assessment of Cognition (GPCOG).The memory may contain further machine-executable instructions which may cause the device to repeat steps (a), (b), and (c) for a plurality of testing cycles. Step (a) in each testing cycle may be modified to only show the subject the items missed in the step (c) of the preceding cycle. Step (b) in each testing cycle may be modified so that only the intermediate task phase is performed for each testing cycle after the first, with the first including both the intermediate task phase and the step of obtaining the metacognitive judgement score. Where there are no missing items from the previous step (c), step (a) is skipped as there are no items to display. The memory may contain further machine-executable instructions which may cause the device, after repeating steps (a), (b), and (c) for the plurality of testing cycles, to obtain a further metacognitive judgement score indicative of how the subject believes they will perform during a delayed retrieval phase. The memory may contain further machine-executable instructions which may cause the device to perform the delayed retrieval phase after obtaining the further metacognitive judgement score. The memory may contain furthermachine-executable instructions which may cause the device to perform one or more filler tasks after obtaining the further metacognitive judgement score and before the delayed retrieval phase.The score may be calculated either according to equation (1):Scoremale— 0.606 ■ (10 ■ (-0.0738 ■ (70 - age) + ^correct items1- ’P) +2 ■ (15 - [0.00196 ■ (70 - age) + |3 + jol - £COrrect items l | ]) + 1 ■ (15 + 6 ■ (-0.00828 ■ (70 - age) + 0.5 -) or according to equation (2):ScoreNOTmaie— 0.606 ■ (10 ■ (-0.0296 ■ (70 - age) - 1.05 + Scorrect itemsL ■ P) +2 ■ (15 — [—0.00767 ■ (70 — age) + 0.0907 + |3 + jol — £Correct items l |]) + 1 ■ (15 + 6 ■ (—0.00797 ■ (70 - age) + 0.00177 + 0.5 -) wherein equation (1) is used when the subject is male, and equation (2) is used when the subject is not male, and where age is the age of the subject, L is a weighting for the location in which a given item was presented during the encoding phase, P is a weighting for the order in which a given item was presented during the encoding phase, jol is the metacognitive judgment score, and RT is an average reaction time which is the intermediate task score. In one example, the age of the subject may be calculated based on the difference between the year of the assessment date and the subject’s year of birth (which the subject may be requested to provide).In a third aspect, embodiments of the invention provide a system for determining a subject’s neurological status by calculating a score indicative of the subject’s neurological status, the system comprising one or more processors, a display, and a user input component, wherein the system further includes memory containing machine-executable instructions which, when executed on the one or more processors, cause the system to:(a) perform an encoding phase, in which a plurality of items to be recalled by the subject in a later phase a displayed via the display;(b) obtain from the subject, via the user input component, a metacognitive judgement score and / or perform an intermediate phase using the display and the user input component, in which the user is presented with a task to perform and which results in an intermediate task score;(c) perform a retrieval phase using the display and the user input component, in which theRsubject is asked to recall the plurality of items presented in the encoding phase, and which results in a retrieval phase score; and(d) calculate the score indicative of the subject’s neurological status based on the retrieval phase score and one or both of: the metacognitive judgement score and the intermediate task score.The processor performing steps (a) - (c) may be a different processor to the processor performing step (d). The two processors may be connected via a local or wide area network.The system may include a further processor, connected to the processor performing steps (a) - (d), and the calculated score may be communicated to this further processor.The memory may contain machine readable-instructions which, when executed on the one or more processors, cause the system to perform the computer-implemented method according to the first aspect and including any one, or any combination insofar as they are compatible, of the optional features set out with reference thereto.The invention includes the combination of the aspects and optional features described except where such a combination is clearly impermissible or expressly avoided.Further aspects of the present invention provide: a computer program comprising code which, when run on a computer, causes the computer to perform the method of the first aspect; a computer readable medium storing a computer program comprising code which, when run on a computer, causes the computer to perform the method of the first aspect; and a computer system programmed to perform the method of the first aspect.BRIEF DESCRIPTION OF THE DRAWINGSFigure 1 shows a method;Figure 2 is a schematic of a device;Figure 3 is a schematic of a system;Figure 4 is scatter plot of number of items correctly recalled (y-axis) over age (x-axis);Figure 5 is a smoothed histogram for items correctly recalled;Figure 6 is a scatter plot of number of items correctly recalled after a 10-minute delay (y-axis) over age (x-axis);Figure 7 is a smoothed histogram for items correctly recalled after a 10-minute delay;Figure 8 is a scatter plot of composite score (y-axis) over age (x-axis);QFigure 9 is a smoothed histogram for composite scores;Figure 10 is a smoothed histogram plot for the absolute value of the accuracy of the metacognitive judgement score (also referred to as a judgement of learning task, or JOL);Figure 11 is a smoothed histogram plot for reaction time;Figure 12 is a scatter plot of composite score (x-axis) and measured MMSE score (y-axis), with a curve showing the predicted MMSE score;Figure 13 is a scatter plot of composite score (x-axis) and measured ADAS-Cog score, with a curve showing the predicted ADAS-Cog score;Figure 14 is a scatter plot of measured MMSE score (x-axis) and predicted MMSE score (y-axis);Figure 15 is an example interface of a display when performing the encoding phase;Figure 16 is an example interface of a display when obtaining the metacognitive judgment score;Figure 17 is an example interface of a display when performing the intermediate task phase;Figure 18 is an example interface of a display when performing the retrieval phase; andFigures 19 - 22 are example interfaces of a display during various filler tasks.DETAILED DESCRIPTIONAspects and embodiments of the present invention will now be discussed with reference to the accompanying figures. Further aspects and embodiments will be apparent to those skilled in the art.Figure 1 shows a method 400. In a first step of the method, step S402, an encoding phase is performed. Items are presented or displayed to a subject, who is asked to remember them. In some examples, and as is discussed in more detail below, the items are presented in a grid and the subject is asked to remember the locations of the item. The items can be presented sequentially, one at a time. The encoding phase may form a part of a paired associates learning test, which includes displaying 12 cards in a three by four grid. Cards are ‘opened’ in a random order revealing an item (e.g. a basketball). Once the item has been briefly shown it is then concealed and the next object is revealed. Items are shown one by one until all 12 have been revealed. The subject is instructed to try to remember each of the items and their location in the grid. Herein, item and object may be considered synonyms.Next, in step S404, the subject is asked to provide a metacognitive judgement score. For example, the subject may be asked to give the number of items they think they will remember the correct location of. This is followed by step S406 in which an intermediate task phase is performed, in whichthe user is presented with a task to perform and which results in an intermediate task score. Steps S404 and S406 may be considered together as distraction tasks to ensure that the subject does not go directly from the encoding phase to the retrieval phase. In one example, the intermediate task phase is a reaction task during which the subject is asked to tap on an element (e.g. dot), that appears in a random location on the screen, as quickly as possible. The intermediate task results in an intermediate task score.The method then moves to step S408, where the retrieval phase is performed. The subject is asked to recall the items shown to them in S402, with the result being a retrieval phase score. For example, where the items have been shown in a grid the subject is asked to indicate where in the grid a given item was located. In some examples, the result of the retrieval phase is an indication as to how many items were correctly recalled. The method then moves to step S410 where the score is calculated based on: the metacognitive judgement score, the intermediate task score, and the retrieval phase score.When calculating the score, it is important to consider the sequence and location of the items as shown. The scoring of the items may need to be weighted to account for one or both of these factors. Firstly, the order in which stimuli are presented can have a significant effect on which items a subject will be able to recall. The Serial Position Effect indicates that information presented at the beginning (primacy) and end (recency) of a task will likely be recalled more readily than the information in the middle (Troyer, 2011). Accordingly, weighting objects according to the presentation order ensures that the method accounts for this phenomenon. Secondly, the location of items within the grid (when a grid is used) is likely to have an effect on the user’s recall ability. For example, stimuli presented adjacent to corners receive more cognitive resources than those appearing along straight edges (Cole, Skarratt, & Gellatly, 2007). It is useful then to weight the object stimuli according to both the presentation order (ordinal location) but also their location when presented (presentation location).In an example, the weights for the presentational order were as follows:Table 1 : Weights used for initial presentation order of objectsThe weights for presentation order were based on the odds ratio between a healthy group of subjects and a group of subjects diagnosed with Alzheimer’s disease. They were normalised such that their sum equals 12 (the number of items).For the location of the items, a weight of 0.6 was used for corner items and 1 .2 for non-corner items. In this way corners, which are easier to remember, are given less weight than non-corners. Again, the sum of the weights was set as equal to 12 (the number of items).In addition to the weighted recall score, the metacognitive judgement score (also referred to as a judgement of learning task score or value) was taken into account. As discussed in more detail below, the subjects diagnosed with AD are more accurate with their predictions of how many items they will remember the correct location of. Healthy subjects were found to underestimate their performance on average by three items. The metacognitive judgement score can be used then together with the number of recalled items as an additional part of an overall output score for the subject. It was further found that subjects diagnosed with AD were slower in their reaction time than healthy subjects, and so this intermediate task score is also taken into account when calculating the overall output score.All three sub-scores (metacognitive judgement score, intermediate task score, and retrieval phase score) are dependent on the age and sex of the subject. Correction was therefore introduced for age and sex for each subject by normalising each sub-score to a reference age and sex. Each component of the overall score is then assigned a weight to account for the fact that the recall task is the most accurate of the three in separating the two subject groups, and therefore should contribute more to the overall output score.In some examples, the score is then normalised such that the overall output score is a number between 0 and 100. This conveniently provides a well-defined scale with a clear start and end.The resulting score is calculated based on one of the following equations:(1) Scoremaie—0.606 - (10. (-0.0738 ■ (70 - age) + Scorrect itemsL ■ P) +2 ■ (15 - [0.00196 ■ (70 - age) + |3 + jol - ^correct items* ID +1 ■ f 15 + 6 ■ f -0.00828 ■ (70 - age) + 0.5 -)17(2) ScoreNOTmale—0.606 ■ (10. (-0.0296 ■ (70 - age) + XCorrect items£’P) +2 ■ (15— 0.00767 ■ (70 - age) + 0.0907 + |3 + jol - £COrrect itemSl |]) +1 ■ ( 15 + 6 ■ (-0.00797 ■ (70 - age) + 0.00177 + 0.5 - J)Where age = year of assessment date - subject's year of birth, is the age of the subject based on their year of birth and the year of the assessment. L is the weight for the location in which the item was presented and P is the weight for the order in which the items were presented during the encoding phase, jol is the value the subject entered during the metacognitive judgement task. RT is the average reaction time (in ms, as a mean of the values from a plurality of repetitions of the intermediate task) from the first instance of the intermediate task.In some examples the result of the equation is rounded. It is then set to 0 should it be below zero, and set to 100 in case the chosen equation results in a value above 100.As is discussed in more detail below, there is a strong correlation between the score calculated using the equations above and the Mini-Mental State Examination (MMSE) as well as the Alzheimer’s Dementia Assessment Scale - Cognitive subscale (ADAS-Cog). As a consequence, the overall score calculated using the equation above can be used to predict a subject’s MMSE and ADAS-Cog score. Advantageously, this facilitates the interpretation of the calculated score as it can be translated into other known scales. In both cases, the relationship was found to be better described by use of nonlinear functions. For predicting the MMSE score a logarithmic function was used, and for predicting the ADAS-Cog score an exponential function was used. The predicted MMSE and ADAS-Cog scores can be calculated with the following equations:Predicted MMSE Score = 30 + 4.632(ln[Score] — ln

[0100] )Predicted ADAScog score = 24.3 ■ e-D 016'[ScorelWhere Score is the score as calculated using equation (1) or (2) above.In some examples, steps S402, S406 and 408 are repeated for a plurality of testing cycles (for example six cycles total) with only the scores from the first testing cycle being used to calculate the Score as discussed above. The steps may be modified so that only the items not correctly recalled are shown in the subsequent encoding phase (this can be referred to as a selective reminding test).After these further testing cycles, the subject is given different distraction tasks for a period (e.g. 10 minutes) before a final delayed recall of the items. During investigations into how classification varies for the total number recalled after five selective reminding phases and a 10 minute delay, 190 data sets were randomly selected (95 diagnosed with AD and 95 healthy control subjects) to train a linear classifier and the remaining 96 data sets (43 diagnosed with AD and 53 healthy control) were used to validate the classification performance. The procedure was repeated 100 times to obtain a median performance of the classification. The median accuracy based on a delayed recall task is 74%, with the median area under the ROC curve for AD subjects being 0.76. This is comparable to the results for only a single testing cycle and shows that the assessment does not need to be lengthy in order to gain insight into the subject’s cognitive status.Figure 2 is a schematic of a device 200. The device includes a processor 202, memory 204, longterm storage 206, a display 208, a user input component 210, and (optionally) a network interface 212. The memory 204 contains machine-readable instructions which, when executed on the processor 202, cause the processor 202 to perform the method as discussed above. In some examples the device 200 is a tablet or smart phone, and so the display 208 and user input component 210 may be combined as a touch screen display. The network interface 212 may be a wired or wireless connection and can allow connection to a local area network or a wide area network. The long-term storage may also contain machine-executable instructions, or a copy thereof, but also scores as calculated using the equations above.Figure 3 is a schematic of a system. The system includes a device 200 of the type referred to in Figure 2, which is connected via network 300 to a further device 302. In some examples, device 200 is used to perform steps S402 - S408. After the device 200 has collected the sub-scores, it transmits them to device 302 across the network (which may be a local area network or a wide area network) to the further device 302. This further device can then calculate the score, and optionally provide the score back across the network to device 200 and / or store the results.Figure 4 is a scatter plot of the number of items correctly recalled (y-axis) over age (x-axis). Data was collected from 286 subjects. Figure 5 is a smoothed histogram for items correctly recalled.The recall phase score data was randomly split into a subset for training and a subset for validation. 190 data sets (95 corresponding to subjects diagnosed with AD and 95 healthy control subjects) to train a linear classification algorithm, and the remaining 96 data sets (43 diagnosed with AD and 53 healthy control subjects) to test the classification capability in data not trained on. The process was repeated 100 times to explore how well the classification works on average. Based on this procedure,the median accuracy achieved for the number of items correctly recalled was 74%. The median area under the ROC curve for AD subjects is 0.77.Figure 6 is a scatter plot of the number of items correctly recalled after a 10-minute delay (y-axis) over age (x-axis). Figure 7 is a smoothed histogram for items correctly recalled after a 10-minute delay.Figure 8 is a scatter plot of the composite score (y-axis) over age (x-axis). Figure 9 is a smoothed histogram for composite scores. Figure 10 is a smoothed histogram plot for the absolute value of the accuracy of the metacognitive judgement scores (also referred to as a judgement of learning task, or JOL). Figure 11 is a smoothed histogram plot for reaction times.For the composite score, the same procedure as discussed above with respect to Figures 4 and 5 was repeated. Namely, subjects were randomly selected to be in a training data set and the accuracy and area under the ROC curve was calculated for the remaining subjects. This was repeated 100 times, and the median accuracy for this was 75.5% with the median area under the ROC curve for AD subjects being 0.82.Figure 12 is a scatter plot of the composite score (x-axis) and measured MMSE score (y-axis), with a curve showing the predicted MMSE score. Figure 13 is a scatter plot of the composite score (x-axis) and measured ADAS-Cog score, with a curve showing the predicted ADAS-Cog score. Figure 14 is a scatter plot of measured MMSE score (x-axis) and predicted MMSE score (y-axis).In the 286-element dataset, it was found that the predicted MMSE score differed from the actual MMSE score by three or more points for only 25.8% of subjects. For 35.2% of subjects, the difference was less than one point on the MMSE scale. For the predicted ADAS-Cog score in the 286 subjects, it was found that 30.1% of subjects have a difference greater than or equal to five points, whilst for 31 .6% of subjects the difference was less than two points on the ADAS-Cog scale. Table 2 shows the results of this comparison for MMSE, and Table 3 shows the results of this comparison for ADAS-Cog:Table 2 For each score difference of MMSE and predicted MMSE this table shows the percentage of subjects with difference lower than the respective score difference and the percentage of subjects with difference equal to or higher than the respective score difference.18Table 3: For each score difference of ADAS-Cog and predicted ADAS-Cog this table shows the percentage of subjects with difference lower than the respective score difference and the percentage of subjects with difference equal to or higher than the respective score difference.For further comparison, a separate data set was used. The separate data set consisted of 187 subjects who were recruited in Kuala Lumpur, Malaysia. Diagnostic status was not available for these subjects, as they were volunteers recruited without any inclusion or exclusion criteria. Using the composite score to calculate a predicted MMSE score, it was possible to compare this to the measured MMSE score obtained from assessing the subjects with the MMSE. It was found that for 33.7% of subjects the predicted MMSE score differed from the measured MMSE score by three points or more. For 19.8% of subjects the difference was found to be less than one point on the MMSE scale. Table 4 shows the results of this comparison in more detail:Table 4: For each score difference of MMSE and predicted MMSE this table shows the percentage of subjects from the Malaysian cohort with difference lower than the respective score difference and the percentage of subjects from the Malaysian cohort with difference equal to or higher than the respective score difference.Figure 15 is an example interface of a display when performing the encoding phase. A device 150 of the type discussed previously shows on its display 152 a four by three grid of cards 154. One at a time, items 156 are revealed to the subject during the encoding phase. For example, basketball 156 is presented in the third column third row. The order in which items are revealed, and the position of one item relative to the next revealed item are randomly selected.Figure 16 is an example interface of the display 152 when the device 150 is obtaining the metacognitive judgment score. The display prompts the user, in this example, to enter the number of item locations they had memorised. In this example, the user input component is a touchscreen and so the subject can slide their finger across the screen to select the number of items they believe they have memorised. Once they have selected the appropriate number, they press the submit button.Figure 17 is an example interface of the display 152 when performing the intermediate task phase. A green dot 158 appears on the display 152, together with a prompt to tap the dot as quickly as possible. This process repeats until an average reaction speed is calculated, which is recorded asthe intermediate task phase score. The process can be repeated for a fixed duration (e.g., 30s) or until a fixed number of reaction speeds have been measured.Figure 18 is an example interface of the display 152 when performing the retrieval phase. An item 182 is presented above the four by three grid of cards, and the subject is prompted to indicate where in the grid they believe the item was displayed during the encoding phase. In this example, the subject indicates card 184 by tapping on the touch screen display at the relevant location.Figures 19 - 22 are example interfaces of the display during various filler tasks. Figure 19 shows a display partway through a questionnaire on the subject’s subjective cognitive performance. Figure 20 shows a symbol matching exercise, where a key or index is presented in showing symbols with corresponding numerical values. A prompt value (e.g. 4) is shown on the display, and the subject is requested to indicate which symbol corresponds to the prompt value according to the key or index. In a subsequent or variant version of this exercise, the exercise is reversed and the subject must match prompted symbols to numerical values. Figure 21 shows a complex reaction filler task, which has three versions with increasing difficulty. Each version is introduced with a short tutorial. The screen layout for all three versions is the same, in the middle of the display is a large button with the word ‘Ready’ on it. In a semi-circle around the top of this large button are eight circles arranged equidistantly. The subject is asked to place their finger on the ready button, and to use the same finger to react to the stimulus. As a stimulus, one of the eight circles are highlighted. In the first version, the same location is highlighted and the subject is asked to tap that location as quickly as possible. The task is repeated until the subject has correctly responded 16 times. In the second version, the four different locations are highlighted one at a time. Each time, one of the four locations is selected at random. Again, the subject is asked to tap the highlighted location as quickly as possible. The task is repeated until the subject has correctly responded 16 times. In a third, and the most complex, version of the task, any of the eight locations can be highlighted. However, the subject is asked to only tap if the highlighted location is not one of the so called ‘forbidden locations’ (see the locations highlighted in red in Figure 22). The task continues until a 10 minute timer (begun at the start of the filler tasks and / or after completion of the encoding phase corresponding to the delayed recall phase) has expired.The systems and methods of the above embodiments may be implemented in a computer system (in particular in computer hardware or in computer software) in addition to the structural components and user interactions described.The term “computer system” includes the hardware, software and data storage devices for embodying a system or carrying out a method according to the above described embodiments. For example, a1Qcomputer system may comprise a central processing unit (CPU), input means, output means and data storage. The computer system may have a monitor to provide a visual output display. The data storage may comprise RAM, disk drives or other computer readable media. The computer system may include a plurality of computing devices connected by a network and able to communicate with each other over that network.The methods of the above embodiments may be provided as computer programs or as computer program products or computer readable media carrying a computer program which is arranged, when run on a computer, to perform the method(s) described above.The term “computer readable media” includes, without limitation, any non-transitory medium or media which can be read and accessed directly by a computer or computer system. The media can include, but are not limited to, magnetic storage media such as floppy discs, hard disc storage media and magnetic tape; optical storage media such as optical discs or CD-OMs; electrical storage media such as memory, including RAM, ROM and flash memory; and hybrids and combinations of the above such as magnetic / optical storage media.While the disclosure has been described in conjunction with the exemplary embodiments described above, many equivalent modifications and variations will be apparent to those skilled in the art when given this disclosure. Accordingly, the exemplary embodiments of the disclosure set forth above are considered to be illustrative and not limiting. Various changes to the described embodiments may be made without departing from the spirit and scope of the disclosure.In particular, although the methods of the above embodiments have been described as being implemented on the systems of the embodiments described, the methods and systems of the present disclosure need not be implemented in conjunction with each other, but can be implemented on alternative systems or using alternative methods respectively.The features disclosed in the description, or in the following claims, or in the accompanying drawings, expressed in their specific forms or in terms of a means for performing the disclosed function, or a method or process for obtaining the disclosed results, as appropriate, may, separately, or in any combination of such features, be utilised for realising the disclosure in diverse forms thereof.While the disclosure has been described in conjunction with the exemplary embodiments described above, many equivalent modifications and variations will be apparent to those skilled in the art when given this disclosure. Accordingly, the exemplary embodiments of the disclosure set forth above are70considered to be illustrative and not limiting. Various changes to the described embodiments may be made without departing from the spirit and scope of the disclosure.For the avoidance of any doubt, any theoretical explanations provided herein are provided for the purposes of improving the understanding of a reader. The inventors do not wish to be bound by any of these theoretical explanations.Any section headings used herein are for organizational purposes only and are not to be construed as limiting the subject matter described.Throughout this specification, including the claims which follow, unless the context requires otherwise, the word “comprise” and “include”, and variations such as “comprises”, “comprising”, and “including” will be understood to imply the inclusion of a stated integer or step or group of integers or steps but not the exclusion of any other integer or step or group of integers or steps.It must be noted that, as used in the specification and the appended claims, the singular forms “a,” “an,” and “the” include plural referents unless the context clearly dictates otherwise. Ranges may be expressed herein as from “about” one particular value, and / or to “about” another particular value. When such a range is expressed, another embodiment includes from the one particular value and / or to the other particular value. Similarly, when values are expressed as approximations, by the use of the antecedent “about,” it will be understood that the particular value forms another embodiment. The term “about” in relation to a numerical value is optional and means for example + / - 10%.ReferencesCole, G. G., Skarratt, P. A., & Gellatly, A. R. (2007). Object and spatial representations in the corner enhancement effect. Perception & psychophysics, 69(3), 400-412.Troyer, A. K. (2011). Serial Position E ect. In J. S. Kreutzer, J. DeLuca, & B. Caplan (Eds.), Encyclopaedia of Clinical Neuropsychology (pp. 2263-2264). New York, NY: Springer New York.71

Claims

CLAIMS1 . A computer-implemented method for determining a subject’s neurological status by calculating a score indicative of the subject’s neurological status, the method comprising steps of:(a) performing an encoding phase, in which the subject is presented with a plurality of items to be recalled in a retrieval phase;(b) obtaining, from the subject, a metacognitive judgement score, and / or performing an intermediate task phase, in which the subject is presented with a task to perform and which results in an intermediate task score;(c) performing the retrieval phase, in which the subject is asked to recall the plurality of items presented in the encoding phase, and which results in a retrieval phase score; and(d) calculating the score indicative of the subject’s neurological status based on the retrieval phase score and one or both of: the metacognitive judgement score and the intermediate task score.

2. The computer-implemented method of claim 1 , wherein the metacognitive judgement score, intermediate task score, and retrieval phase scores are independently weighted when used to calculate the score.

3. The computer-implemented method of claim 1 or claim 2, wherein the score is calculated based on an equation selected on the basis of the subject’s sex.

4. The computer-implemented method of any preceding claim, wherein components of the retrieval phase score are weighted based on ordinal position and / or location of each item to be recalled.

5. The computer-implemented method of any preceding claim, wherein the neurological status is a cognitive status of the subject.

6. The computer-implemented method of any preceding claim, wherein the metacognitive judgement score is an indication of how the subject believes they will perform during the retrieval phase.

7. The computer-implemented method of any preceding claim, wherein the score is further calculated based on the age of the subject.

8. The computer-implemented method of any preceding claim, wherein the intermediate task is a reaction speed test.

9. The computer-implemented method of any preceding claim, further comprising a step of calculating a predicted MMSE or ADAS-Cog value from the score10. The computer-implemented method of any preceding claim, wherein the method further includes repeating steps (a), (c), and (d) for a plurality of testing cycles.11 . The computer-implemented method of claim 10, wherein step (a) in each testing cycle is modified to only show the subject the items missed in the step (d) of the preceding cycle.

12. The computer-implemented method of claim 10 or 11 , wherein, after repeating steps (a), (c), and (d) for the plurality of testing cycles, the method further comprises obtaining a further metacognitive judgment score indicative of how the subject believes they will perform during a delayed retrieval phase.

13. The computer-implemented method of claim 12, further comprising performing the delayed retrieval phase after obtaining the further metacognitive judgement score.

14. The computer-implemented method of claim 13, further comprising performing one or more filler tasks after obtaining the further metacognitive judgement score and before the delayed retrieval phase.

15. The computer-implemented method of any preceding claim, where in the score is calculated either according to equation (1):ScOT'ernale 0.606 ■ (10 ■ (-0.0738 ■ (70 - age) + £correctitemsL ' f) +2 ■ (15 - [0.00196 ■ (70 - age) + |3 + jol - Ecorrect items l l ]) +1 ■ (15 + 6 ■ (-0.00828 ■ (70 - age) + 0.5 -) or according to equation (2):ScoreNOTmaie— 0.606 ■ (10 ■ (-0.0296 ■ (70 - age) - 1.05 + Xcorrect itemsZ, ■ P) +2 ■ (15 - [-0.00767 ■ (70 - age) + 0.0907 + |3 + jol - £correctitems* ID +1 ■ (15 + 6 ■ (—0.00797 ■ (70 - age) + 0.00177 + 0.5 -); wherein equation (1) is used when the subject is male, and equation (2) is used when the subject is not male, and where age is the age of the subject, L is a weighting for the location in which a given item was presented during the encoding phase, P is a weighting for the order in which a given item was presented during the encoding phase, jol is the metacognitive judgment score, and RT is an average reaction time which is the intermediate task score.

16. A device for determining a subject’s neurological status by calculating a score indicative of the subject’s neurological status, the device including a display, a processor, and a user input component; wherein the device further includes memory containing machine-executable instructions which, whenexecuted on the processor, cause the device to:(a) perform an encoding phase, in which a plurality of items to be recalled by the subject in a later phase are displayed via the display;(b) obtain from the subject, via the user input component, a metacognitive judgement score, and / or perform an intermediate task phase using the display and the user input component, in which the user is presented with a task to perform and which results in an intermediate task score;(c) perform the retrieval phase using the display and the user input component, in which the subject is asked to recall the plurality of items presented in the encoding phase, and which results in a retrieval phase score; and(d) calculate the score indicative of the subject’s neurological status based on the retrieval phase score and one or both of : the metacognitive judgement score and the intermediate task score.

17. The device according to claim 16, wherein, in calculating the score indicative of the subject’s neurological status, the metacognitive judgement score, intermediate task score, and retrieval phase scores are independently weighted.

18. The device according to claim 16 or 17, wherein in calculating the score, an equation is used by the processor, and the equation is selected on the basis of the subject’s sex.

19. The device according to any of claims 16 - 18, wherein components of the retrieval phase score are weighted based on ordinal position and / or location of each item to be recalled.

20. The device according to any of claims 16 - 19, wherein the neurological status is a cognitive status of the subject.21 . The device according to any of claims 16 - 20, wherein the metacognitive judgment score is an indication of how the subject believes they will perform during the retrieval phase.

22. The device according to any of claims 16 - 21 , wherein calculating the score is further based on the age of the subject.

23. The device according to any of claims 16 - 22, wherein the intermediate task is a reaction speed test.

24. The device according to any of claims 16 - 23, wherein memory contains further machineexecutable instructions which cause the device to calculate a predicted MMSE or ADAS-Cog value from the score.7425. The device according to any of claims 16 - 24, wherein the memory contains further machineexecutable instructions which cause the device to repeat steps (a), (c), and (d) for a plurality of testing cycles.

26. The device according to claim 25, wherein step (a) in each testing cycle is modified to only show the subject the items missed in the step (d) of the preceding cycle.

27. The device according to claim 25 or 26, wherein the memory contains further machineexecutable instructions which cause the device, after repeating steps (a), (c), and (d) for the plurality of cycles, to obtain a further metacognitive judgement score of how the subject believes they will perform during a delayed retrieval phase.

28. The device according to claim 27, wherein the memory contains further machine-executable instructions which cause the device to perform the delayed retrieval phase after obtaining the further metacognitive judgement score.

29. The device according to claim 28, wherein the memory contains further machine-executable instructions which cause the device to perform a filler phase in which the subject is asked to perform one or more filler tasks, after obtaining the further metacognitive judgement score and before the delayed retrieval phase.

30. The device according to any of claims 16 - 29, where in the score is calculated either according to equation (1):Scoremale— 0.606 ■ (10 ■ (-0.0738 ■ (70 - age) + Ecorrect itemsZ, ■ P) +2 ■ (15 - [0.00196 ■ (70 - age) + |3 + jol - Ecorrect items1! ]) +1 ■ (15 + 6 ■ (-0.00828 ■ (70 - age) + 0.5 - ^-))) or according to equation (2):ScoreNOTmaie— 0.606 ■ (10 ■ (-0.0296 ■ (70 - age) - 1.05 + Xcorrect itemsL ■ P) +2 ■ (15 - [-0.00767 ■ (70 - age) + 0.0907 + |3 + jol - Xcorrect items1!]) +1 ■ (15 + 6 ■ (—0.00797 ■ (70 - age) + 0.00177 + 0.5 -); wherein equation (1) is used when the subject is male, and equation (2) is used when the subject is not male, and where age is the age of the subject, L is a weighting for the location in which a given item was presented during the encoding phase, P is a weighting for the order in which a given item was presented during the encoding phase, jol is the metacognitive judgment score, and RT is an average reaction time which is the intermediate task score.31 . A system for determining a subject’s neurological status by calculating a score indicative of the subject’s neurological status, the system comprising one or more processors, a display, and a user input component, wherein the system further includes memory containing machine-executable instructions which, when executed on the one or more of the processors, causes the system to:(a) perform an encoding phase, in which a plurality of items to be recalled by the subject in a later phase a displayed via the display;(b) obtain from the subject, via the user input component, a metacognitive judgement score, and / or perform an intermediate phase using the display and the user input component, in which the user is presented with a task to perform and which results in an intermediate task score;(c) perform a retrieval phase using the display and the user input component, in which the subject is asked to recall the plurality of items presented in the encoding phase, and which results in a retrieval phase score; and(d) calculate the score indicative of the subject’s neurological status based on the retrieval phase score and one or both of: the metacognitive judgement score and the intermediate task score.

32. The system of claim 31 , wherein the processor performing steps (a) - (c) is a different processor to the processor performing step (d).

33. The system of claim 31 or 32, wherein the memory contains machine readable-instructions which, when executed on the one or more processors, cause the system to perform the computer- implemented method according to any of claims 1 - 16.76

Images