System and method for device-based prediction of a cognitive disability

Abstract

A processor-implemented method for prediction of a cognitive disability using a device-based game, including: displaying, with a display module of a communication device, a game executed on the communication device, wherein the game includes game components and at least one of: auditory game objects and visual game objects; receiving, with the game, via the communication device or a device connected with the communication device, input signals throughout at least a portion of the duration of the game; extracting, with the game, information on cognitive state based on the input signals; and measuring, with the game, the information on cognitive state against neurological or psychiatric conditions; and predicting, with the game, a cognitive disability based on the information on cognitive state measured against the neurological or psychiatric conditions.

Description

SYSTEM AND METHOD FOR DEVICE-BASED PREDICTION OF A COGNITIVE DISABILITYFIELD

[0001] The present disclosure relates to systems and methods for device-based prediction of a cognitive disability, and more particularly to, systems and methods for device-based prediction of a cognitive disability using machine learning.BACKGROUND

[0002] Reading, writing, and speech are fundamental cognitive skills critical for academic success and broader life outcomes. However, many individuals face cognitive disabilities that hinder their ability to develop these skills effectively. Among school - aged children, approximately one in ten students is considered neurodiverse, often exhibiting challenges related to reading. These challenges may be the result of unidentified conditions such as dyslexia, learning disabilities, or attention- deficit / hyperactivity disorder (ADHD). Reading difficulties, in particular, remain one of the most prevalent learning disabilities encountered in educational environments. It is estimated that students with reading disabilities constitute approximately 80% of the school-age population identified with learning disabilities.

[0003] Learning disabilities are understood to stem from psychological processing deficits that are neurologically based (National Association of School Psychologists. (2011, July). Identification of students with specific learning disabilities (Position Statement). Bethesda, MD: Author.). Academic performance is influenced by a combination of lower-order and higher-order cognitive constructs, including but not limited to phonological and orthographic processing, oral-motor functioning, language abilities, verbal and non-verbal reasoning, immediate and working memory, and longterm memory retrieval (Anderson, P. (2002). Assessment and development of executive function (EF) during childhood. Child Neuropsychology, 8(2), 71-82.; Berninger, V. W., & Richards, T. L. (2002). Brain literacy for educators and psychologists. San Diego,CA: Academic Press.). Executive functions — such as attention regulation, impulse control, planning, organization, and strategy implementation — play a pivotal role in academic learning and output, particularly in reading development (Berninger, V. W., & Richards, T. L. (2002). Brain literacy for educators and psychologists . San Diego, CA: Academic Press.; McCloskey, G., & Perkins, L. A. (2012). Essentials of executive functions assessment. Hoboken, NJ: John Wiley & Sons, Inc.; McCloskey., G., Perkins, L. A., & Van Divner, B. (2009) Assessment and intervention for executive function difficulties. New York, NY: Routledge.). Deficits in executive functioning can significantly impair a student’s ability to acquire, apply, and retain reading skills.

[0004] Executive functions are believed to form part of a broader cognitive framework that supports the acquisition and consistent application of learned skills (McCloskey, G., & Perkins, L. A. (2012). Essentials of executive functions assessment. Hoboken, NJ: John Wiley & Sons, Inc.). These functions are particularly critical in reading, as they support mastery of foundational skills such as alphabet recognition and word comprehension. When executive functioning is weak, specific difficulties with reading often emerge. Successful reading requires students to sustain attention, manage time, remain motivated, organize information, inhibit impulses, and employ decoding and comprehension strategies effectively (Joseph, L. M., & Schisler, R. A. (2006). Reading and the whole student. Student Counseling, 11-15.; Maricle, D. E., Johnson, W., & Avirett, E. (2010). They are assessing and intervening in children with executive function disorders. In D.C. Miller & D.C. Miller (Eds.), Best practices in school neuropsychology: Guidelines for effective practice, assessment, and evidence -based intervention (pp. 599-640). Hoboken, NJ: John Wiley & Sons, Inc.). Furthermore, reading comprehension depends on a student's ability to engage in higher-level reasoning, such as formulating inferences and recognizing categorical relationships (McCloskey, G., & Perkins, L. A. (2012). Essentials of executive junctions assessment. Hoboken, NJ: John Wiley & Sons, Inc). Despite advances in the research of these cognitive processes, diagnostic practices for identifying related impairments remain limited and inconsistent.

[0005] The persistence of inaccurate or delayed diagnosis raises concerns among affected individuals and caregivers about prognosis and treatment options. However, patient diagnosis is complicated by inter- and intra-individual variability. As such, the development of personalized treatment plans typically requires labor-intensive evaluation protocols to assess the presence and extent of cognitive disability and to monitor the effectiveness of selected intervention strategies. In both clinical and research contexts, the analysis of these diagnostic results is frequently performed manually and based on qualitative criteria, making the process vulnerable to inter- and intra-ob server variability. This challenge underscores the need for more automated or semi-automated methods capable of analyzing data consistently and efficiently, thereby supporting improved diagnosis, treatment, and follow-up.

[0006] The global proliferation of digital technologies has driven the creation of e- Health platforms and software tools designed to support individuals with cognitive and behavioral disabilities, including those related to executive function and working memory. Examples include interventions for Autism Spectrum Disorder, ADHD, and dementia. As technology continues to evolve, machine learning has emerged as a powerful tool, having the potential to enhance diagnostic and therapeutic capabilities.

[0007] To address the limitations of traditional assessment models, a non-invasive, objective method that leverages machine learning to assist in identifying new markers or features predictive of cognitive disabilities is desirable. Incorporating automated or semi-automated artificial intelligence can enhance diagnostic accuracy, increase operational efficiency, and reduce manual input by providing clinicians and specialists with pre-screened data and feature-based insights. This work aligns with broader research efforts and policy initiatives aimed at integrating machine learning technology into healthcare and educational domains.BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate the present invention and, together with the description,further serve to explain the principles of the invention and to enable a person skilled in the pertinent art to make and use the invention.

[0009] Figures 1-7 illustrate examples of device-based games for prediction of a cognitive disability, according to embodiments of the present disclosure.

[0010] Figure 8 illustrates an example of a device-based game for prediction of a cognitive disability, according to an embodiment of the present disclosure.

[0011] Figures 9 and 10 illustrate examples of a method for predicting a cognitive disability using a device-based game according to an embodiment of the present disclosure.BRIEF DESCRIPTION

[0012] The following presents a simplified summary of some embodiments of the techniques described herein in order to provide a basic understanding of the invention. This summary is not an extensive overview of the invention. It is not intended to identify key / critical elements of the invention or to delineate the scope of the invention. Its sole purpose is to present some embodiments of the invention in a simplified form as a prelude to the more detailed description that is presented below.

[0013] In an embodiment, the present disclosure provides a processor-implemented method for prediction of a cognitive disability using a device-based game, including: displaying, with a display module of a communication device, a game executed on the communication device, wherein the game includes game components and at least one of: auditory game objects and visual game objects; receiving, with the game, via the communication device or a device connected with the communication device, input signals throughout at least a portion of the duration of the game; extracting, with the game, information on cognitive state based on the input signals; and measuring, with the game, the information on cognitive state against neurological or psychiatric conditions; and predicting, with the game, a cognitive disability based on the information on cognitive state measured against the neurological or psychiatric conditions.

[0014] In an example embodiment, the input signals are collected by at least one sensor of the communication device or an input device connected with the communication device. In an example embodiment, the at least one sensor includes a microphone and a touchscreen sensor or a sensor of an input device connected to the communication device. In an example embodiment, the game is accessible via a software application downloaded on the communication device.

[0015] In an example embodiment, a machine-learning based prediction model is used in predicting the cognitive disability; and the information on cognitive state and control measures are provided as input to the machine-learning based prediction model. In an example embodiment, the information of cognitive state includes information relating to at least one of: attention, hyperactivity, irritability, self-regulation, eye-hand coordination, perception, reading, and listening. In an example embodiment, the control measures include at least one of: age, gender, grade, word count per minute, genetics, and epigenetics. In an example embodiment, the machine-learning based prediction model is further used in predicting a learning disability.

[0016] In an embodiment, the present disclosure provides a processor-implemented method for prediction of a cognitive disability using a device-based game, including: displaying, with a display module of a communication device, a game executed on the communication device, wherein the game includes game components and at least one of: auditory game objects and visual game objects; receiving, with the game, via the communication device, a first set of inputs designating control measures; receiving, with the game, via the communication device or a device connected with the communication device, a second set of input signals throughout at least a portion of the duration of the game; extracting, with the game, performance measures based on the second set of input signals; and predicting, with the game, a cognitive disability using a machine learningbased prediction model, wherein the performance measures and the control measures are provided as input to the machine learning -based prediction model.

[0017] In an example embodiment, the input signals are collected by at least one sensor of the communication device or an input device connected with the communication device. In an example embodiment, the at least one sensor includes a microphone and atouchscreen sensor or a sensor of an input device connected to the communication device. In an example embodiment, the game is accessible via a software application downloaded on the communication device. In an example embodiment, the game further predicts a learning disability using the machine learning -based prediction model. In an example embodiment, the game further predicts dyslexia and ADHD using the machine learning-based prediction model. In an example embodiment, the performance measures include at least one of: attention, timeliness, hyperactivity, impulsiveness, and reading fluency; and the control measures include at least one of: age, gender, grade, word count per minute, genetics, and epigenetics.

[0018] In an example embodiment, the game includes a plurality of games assessing reading fluency and executive function. In an example embodiment, the performance measures include attention, timeliness, hyperactivity, impulsiveness, and reading fluency; and the control measures include age, gender, grade, and word count per minute. In an example embodiment, attention is measured based on a number of correct responses to a target stimuli including failures to respond to a target stimuli; timeliness is measured based on amount of time between responses; hyperactivity is measured based speed of responses and correct responses; impulsiveness is measured based on a number of responses to a non-target stimuli; and reading fluency is measured based on words correct per minute.

[0019] In an example embodiment, for a game of the plurality of games assessing reading fluency: the visual game objects include a plurality of columns of words; the second set of input signals include a first audio input of an individual reading the words in each column from top to bottom, then in each row from left to right; the visual game objects further include a passage; the second set of input signals further include a second audio input of the individual reading the passage at normal speed; and the words correct per minute is measured based on the second set of input signals. In an example embodiment, a game of the plurality of games assessing executive function includes continuously and iteratively displaying a visual game object including one of: a red hexagon, a blue hexagon, the word red, or the word blue within one of two play areas, wherein the second set of input signals includes a first input designating a play area,wherein a goal is to, when the red hexagon or the word red is displayed, select the play area in which the red hexagon or the word red is displayed, and when the blue hexagon or the word blue is displayed, select the play area in which the blue hexagon or the word blue is not displayed; the number of correct responses to a target stimuli including the failures to respond to a target stimuli and the number of responses to a non-target stimuli are measured based on the second set of input signals.

[0020] In an example embodiment, scenes and assets of the game including the auditory game objects and the visual game objects are generated independent of a language of game delivery. In an example embodiment, displaying the game includes displaying both translucent and opaque moving masks to induce attention. In an example embodiment, displaying the game includes providing a cognition inducing component selected from a group consisting of rhymes, pidgin English, and homophones.

[0021] In an example embodiment, displaying the game includes interspersing repetitive language tasks with non-language mosaics. In an example embodiment, displaying the game includes displaying motion graphics with embedded text. In an example embodiment, displaying the game includes providing a reward including visual and aural feedback. In an example embodiment, displaying the game includes displaying fast motion graphics in addition to the auditory game objects and the visual game objects. In an example embodiment, displaying the game includes providing an auditory pathway challenge to address one or more phonological components. In an example embodiment, displaying the game includes providing the auditory pathway challenge to address pronunciation. In an example embodiment, displaying the game includes providing a visual pathway challenge to address one or more orthographic aspects. In an example embodiment, displaying the game includes providing the visual pathway challenge to address spelling. In an example embodiment, displaying the game includes providing a cognitive pathway challenge to address one or more motor or working memory aspects. In an example embodiment, displaying the game includes providing the cognitive pathway challenge to address holding and processing information for a brief period of time.

[0022] In an embodiment, the present disclosure provides a non-transitory machine- readable memory storing statements and instructions for execution by the one or more processors to perform any one of the above-described embodiments.

[0023] In an embodiment, the present disclosure provides an apparatus for providing a device-based game for prediction of a cognitive disability including: one or more processors; and a non-transitory machine-readable memory storing statements and instructions for execution by the one or more processors to perform any one of the above-described embodiments.DETAILED DESCRIPTION

[0024] The present invention will now be described in detail with reference to a few embodiments thereof as illustrated in the accompanying drawings. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, to one skilled in the art, that the present invention may be practiced without some or all of these specific details. In other instances, well known process steps and / or structures have not been described in detail in order to not unnecessarily obscure the present invention. Further, it should be emphasized that several inventive techniques are described, and embodiments are not limited to systems implanting all of those techniques, as various cost and engineering trade-offs may warrant systems that only afford a subset of the benefits described herein or that will be apparent to one of ordinary skill in the art.

[0025] In the following detailed description of exemplary embodiments of the invention, reference is made to accompanying drawings (where like numbers represent like elements), which form a part hereof, and in which is shown by way of illustration specific exemplary embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in art to practice the invention, but other embodiments may be utilized, and logical, mechanical, electrical, and other changes may be made without departing from the scope of the present invention. The following detailed description is, therefore, not to be taken in alimiting sense, and the scope of the present invention is defined only by appended claims.

[0026] In the following description, numerous specific details are set forth to provide a thorough understanding of the invention. However, it is understood that the invention may be practiced without these specific details. In other instances, well-known structures and techniques known to one of ordinary skill in art have not been shown in detail in order not to obscure the invention.

[0027] In one or more embodiments, a system and method provide a device-based game for prediction of a cognitive disability. A display module of a communication device (e.g., smart phone, tablet, laptop, etc.) displays a game executed on the communication device, the game including game components and at least one of: auditory game objects and visual game objects. The game may be accessed via a software application downloaded on the communication device or a web application accessed via a web browser downloaded on the communication device or by another means. The software application may operate in an online-mode (e.g., via connection to the Internet) and in an offline mode. The operations and instructions associated with the game are embodied in computer program code that forms at least part of the software application. When executed by one or more processors of the communication device, the computer program code causes the device to perform various operations and / or functions of the game described herein, including but not limited to input signal processing, game state management, rendering of graphics and audio, and communication with remote servers or other devices, as required. The computer program code may be stored in a non- transitory computer-readable medium associated with the communication device. The game receives input signals throughout at least a portion of the duration of the game, from which the game extracts information on cognitive state (e.g., attention, hyperactivity, irritability, self-regulation, eye-hand coordination, perception, reading and listening, etc.). The game may receive a diverse range of input signals, such as input signals from a sensor (e.g., image sensor) capturing hand gestures of an individual, a mouse, a sensor (e.g., microphone) capturing a voice of an individual, a sensor (e.g., image sensor) capturing eye blinking of an individual, etc. Input signals may be capturedor collected using sensors of the communication device (e.g., tactile sensor of a touch screen, a camera, a microphone, etc.) or input devices (e.g., mouse, a controller, a touch pad, etc.) connected with the communication device. The information on the cognitive state is measured against neurological or psychiatric conditions to predict a cognitive disability. For example, an individual may be required to respond to a written, verbal, and / or visual instruction, where information is processed using auditory pathways of the brain. In another example, an individual may be required to respond to attention and executive function games, where information is processed using visual pathways of the brain. The game may output the information on cognitive state and / or the predicted cognitive disability to a user and / or an operator (e.g., via the display module of the communication device). The game may store the information on cognitive state and / or the predicted cognitive disability such that it is accessible to the user and / or the operator at another time. The cognitive state extracted may be compared with someone with neurological and cognitive deficits to predict a cognitive deficit. In some embodiments, the game includes a plurality of games wherein the information on cognitive state are extracted from input signals received throughout at least a portion of the duration of each of the plurality of games.

[0028] In one or more embodiments, a system and method provide a device-based game for prediction of a cognitive disability. A display module of a communication device (e.g., smart phone, tablet, laptop, etc.) displays a game executed on the communication device (e.g., accessed via a software application downloaded on the communication device or a web application accessed via a web browser downloaded on the communication device), the game including game components and at least one of: auditory game objects and visual game objects. The game may receive input designating one or more control measures (e.g., age, gender, grade of an individual, word count per minute, genetics, and epigenetics). The game may further receive input signals throughout at least a portion of the duration of the game, from which the game extracts one or more performance measures (e.g., attention, hyperactivity, impulsiveness, timeliness, reading fluency, etc.). A novel machine learning-based prediction model of the game embodied in the computer program code forming part of the softwareapplication predicts a cognitive disability based on one or more of the performance measures and one or more of the control measures. In some embodiments, the game includes a plurality of games (e.g., reading fluency games, executive function games, etc.) wherein one or more performance measures are extracted from input signals received throughout at least a portion of the duration of each of the plurality of games. The game may output the performance measures and / or the predicted cognitive disability to a user and / or an operator (e.g., via the display module of the communication device). The game may store the performance measures and / or the predicted cognitive disability such that it is accessible to the user and / or the operator at another time.

[0029] In some embodiments, the machine learning-based prediction model predicts a learning disability based on the performance measures derived from at least some of the input signals received throughout the game and one or more of the control measures. In some embodiments, the machine learning-based prediction model further predicts dyslexia or Attention Deficit Hyperactivity Disorder (ADHD) based on one or more of the performance measures and further detailed characterizations of such disabilities. For instance, dyslexia may be characterized by difficulties with accurate and / or fluent word recognition, poor spelling, etc.

[0030] In some embodiments, the game for prediction of the learning disability includes a first game for assessing reading fluency and a second game for assessing executive function. An example of the first game for assessing reading fluency includes two tasks. In a first task, the display module displays visual game objects including a plurality of columns of words. The individual is required to read the words in each column from top to bottom, then in each row from left to right while the game receives input signals from a microphone of the communication device capturing a voice of the individual as they read. Figure 1 illustrates a screenshot of an example of a plurality of columns of words 100 displayed by a display module 110 of a communication device 120. In a second task, the display module displays a visual game object including a passage. The individual is required to read the passage at normal speed while the game receives input signals from the microphone of the communication device capturing the voice of the individual as they read the passage. Based on the input signals received during the firstand second tasks, the game extracts one or more of: words correct per minute, specific words missed, and number of errors, each of which may be recorded and stored to track progress over time. An example of the second game for assessing executive function includes the display module of the communication device continuously and iteratively displaying a visual game object including one of: a red hexagon, a blue hexagon, the word red, or the word blue. The visual object is displayed within one of two areas. If a red hexagon or the word red is displayed, the individual is to provide an input signal (e.g., via a tactile sensor that may be used to detect contact between a finger and a touchscreen of the communication device or a mouse) designating the same area within which the red hexagon or the word red is displayed. If a blue hexagon or the word blue is displayed, the individual is to provide an input signal designating the area within which the blue hexagon or the word blue is not displayed. Based on input signals received throughout the second game, one or more of: omission error (i.e., failing to respond to target stimulus), commission error (i.e., responding to a non-target stimulus), and correct response to stimulus are extracted. Figures 2A and 2B illustrate a screenshot of the display module 110 of the communication device 120 during execution of the second game. FIG. 2 A illustrates two play areas 200 and 210 and a red hexagon 220 displayed within play area 210. FIG. 2B illustrates the word red 230 displayed within the first play area 200. In some embodiments, the performance measures extracted from the input signals received during the first reading fluency game and the second executive function game comprise one or more of: attention (number of correct responses to target stimuli, including the rate of omission errors), timeliness (amount of time between responses), hyperactivity (a measure of motor activity based on speed of responses and correct responses), impulsiveness (responses to non-target stimuli, including the rate of commission errors), and reading fluency and the control measures comprise one or more of: age, gender, grade, and word count per minute.

[0031] Other reading fluency and executive function games may be used in extracting one or more performance measures. The games are designed to stimulate and target the same cognitive pathways and measure the same actions through different tasks. For example, games may measure how fast an individual can make new words, spellingabilities, ability in remembering card placements, etc., from which the game may extract one or more performance measures. Brief descriptions of examples of additional reading fluency and executive function games according to embodiments of the present disclosure are now provided, some of which have associated figures to assist in understanding. These are non-limiting examples intended to illustrate sample implementations of different combinations of aspects of embodiments of the present disclosure.

[0032] Path Finder Game: Figure 3 illustrates a screenshot of the display module 110 of the communication device 120 during execution of the Path Finder Game. The display module 110 displays a koala 300 and food 310 of the koala 300. An individual is to provide input signals (e.g., via a tactile sensor that may be used to detect contact between a finger and a touchscreen of the communication device or a mouse) to the game designating a path that a koala 300 needs to travel to find its food 310. Each input signal designates one of four directional arrows 320 defining a next direction of travel of the koala 300 until reaching its food 310.

[0033] Memory Builder Game: Figure 4 illustrates a screenshot of the display module 110 of the communication device 120 during execution of the Memory Builder Game. An individual is to provide input signals (e.g., via a tactile sensor that may be used to detect contact between a finger and a touchscreen of the communication device or a mouse) to the game designating cards to reveal. Each input signal designates a card to reveal, with a goal of selecting two cards, such as 400 and 410, sequentially that reveal matching animals.

[0034] Word Builder Game: Figure 5 illustrates a screenshot of the display module 110 of the communication device 120 during execution of the Word Builder Game. An individual is to provide input signals (e.g., via a tactile sensor that may be used to detect contact between a finger and a touchscreen of the communication device or a mouse) to the game designating prefixes or suffixes within balloons 500 to add to a currently displayed word 510 to build a new word or an instruction to display a next word when no more prefixes or suffixes can be added to the currently displayed word 510 to build another new word.

[0035] Silly Machine Game: Figure 6 illustrates a screenshot of the display module 110 of the communication device 120 during execution of Silly Machine Game. An individual is to provide input signals (e.g., via a tactile sensor that may be used to detect contact between a finger and a touchscreen of the communication device or a mouse) designating a movement of a first displayed word 600 (e.g., by dragging the word across the display module of the communication device using a finger on the touchscreen of the communication device or a mouse) to a second word 610 of a plurality of displayed words 620 including the same letters as the first displayed word.

[0036] Colour Hopper Game: Figure 7 illustrates a screenshot of the display module 110 of the communication device 120 during execution of Colour Hopper Game. The display module displays two or more buttons of the plurality of buttons 700 lit up one after the other in a particular sequence. An individual is to provide inputs signals e.g., via a tactile sensor that may be used to detect contact between a finger and a touchscreen of the communication device or a mouse) designating two or more buttons to light up one after the other in sequence that matches the game displayed sequence of buttons lit up.

[0037] The machine learning -based prediction model for predicting the learning disability was developed based on a parallel, double-blind, randomized controlled trial. The first game for assessing reading fluency and the second game for assessing executive function described above and illustrated in Figures 1 and 2 were used to assess reading fluency and executive function of participants over five ten minutes sessions per week for five weeks. An assessment of executive attention, working memory, and reading fluency were taken based on results of one or more games described herein at baseline and post-training for each participant. The executive function game provided a serial presentation of visual target and non-target stimuli and required participants to sustain attention over a continuous stimuli stream and respond to a prespecified target, where visual stimuli served as a measurable distractor. Failing to respond to a target stimulus was assumed to measure inattention and was recorded as an omission error. A response to a non-target stimulus was considered to measure impulsivity and was recorded as a commission error. The number of correct responses were also recorded.Reading fluency was measured through recording of word count per minute. From the first game for assessing reading fluency and the second game for assessing executive function, performance measures recorded for each participant were attention (number of correct responses to target stimuli, including the rate of omission errors); hyperactivity (a measure of motor activity); impulsiveness (responses to non -target stimuli, including the rate of commission errors); timeliness; and reading fluency (word correct per minute). The machine learning-based prediction model was trained based on the performance measures (attention, timeliness, hyperactivity, impulsiveness, and reading fluency) and additional control variables (age, gender, grade, word count per minute) to predict a learning disability.

[0038] Participants included 286 children aged 4-13 years (mean = 10.8). Of them, 197 children were diagnosed with a learning disability, and 89 were typically developed children, which corresponds to 68% of children not having a learning disability and 32% having a learning disability. The data was partitioned randomly into training data and holdout data. The training data consisted of 80% of the data and was used for model training. The holdout data consisted of the remaining 20% of the data and was used for model evaluation. The activity data was manipulated to correct the inherent biases due to the imbalance in the number of children with and without learning disabilities. Specifically, the inflated number of training records (N = 500) was bootstrapped using bootstrap sampling with repetition, following the Synthetic Minority Oversampling Technique. The oversampling probability per record was set as inverse to its appearance in the data concerning levels (1 - m / N) and diagnosis [1 - pi(LD)] for children with a learning disorder (LD) and pi(LD) for children without LD. This sampling procedure generated a training data set that was: (1) synthetically large enough to allow the use of robust machine learning techniques that operate on large amounts of data; and (2) balanced within and across levels, thus was free of statistical biases. The holdout data remained untouched to allow a fair prediction model evaluation.

[0039] The machine learning -based prediction model f was trained to the training set that mapped the performance measures (attention, timeliness, hyperactivity, and impulsiveness) and additional control measures (age, gender, grade level, word countper minute) to the diagnosis class: p(diagnosis = L ) = f (features, controls) . Specifically, f in the analysis was either a random forest or neural network with cross- validation. After training the machine learning-based prediction model, the trained model was used to predict the detection of learning disability in the holdout data. The accuracy, sensitivity, and specificity of the machine learning-based prediction model was compared to ground truth results. As a benchmark, the function f was compared with a random Base Model and the machine learning-based prediction model. The machine learning-based prediction model had an accuracy rate of 72%, a sensitivity rate of 78%, and a specificity rate of 77%. The results indicate that the machine learningbased prediction model has high accuracy.

[0040] The machine learning-based prediction model may be validated using other executive function and reading fluency games. For instance, inhibition, the ability to suppress impulses and control distracting stimuli, is another executive function that is a key feature for ADHD that may be measured with a game, the results of which may be used to validate the machine learning -based prediction model.

[0041] Figure 8 illustrates a device-based game for prediction of a cognitive disability of a user according to an embodiment of the present disclosure. In an environment 800, a user 810 accesses the device-based game via a software application 820 downloaded and executed on a communication device 840. A display module 860 of the communication device 840 displays a user interface of the software application 820 through which the user 810 provides at least some input via a touchscreen of the communication device 840. In an example embodiment, a computer-readable memory storing statements and instructions (e.g., program code) are executed by a processor to effectuate operations of the device-based game described herein. In an example embodiment, the software application 820 includes a backend engine including one or more of a scripting language; an asset engine; a workflow engine; one or more machine learning algorithms; an application programming interface (API); a master data management module; and a data store (e.g., Hadoop, No-SQL, Relational). In an example embodiment, the software application 820 includes a generator module including one or more of a scene generator; a configurator; an asset generator; a triggergenerator; and a game generator. In an example embodiment, a scene in a displayed game includes a plurality of specific asset definitions that trigger, or call, a plurality of generic asset definitions, such as objects, background, color, audio, and video.

[0042] In an example embodiment, the display module 860 displays a game that includes a program, sessions, and / or exercises. In an example embodiment, the display module 860 interfaces with the user interface, which includes or provides one or more of: game modules; security; menus; and a management portal. In an example embodiment, the display module 860 interfaces with a deployment engine for deploying the display on any number of channels, such as Slack™, Facebook™, web, mobile, SMS, iOS™, Android™, Windows™, Chrome OS™, Linux, augmented reality (AR), mixed reality (MR) and virtual reality (VR) platforms.

[0043] Figure 9 illustrates an example of a method for prediction of a cognitive disability using a device-based game. At a first step 900, a display module of a communication device displays a game executed on the communication device, the game including game components and at least one of: auditory game objects and visual game objects. At a second step 910, the game receives input signals throughout the duration of the game. At a third step 920, the game extracts information on cognitive state. At a fourth step 930, the game measures the information on cognitive state against neurological or psychiatric conditions. At a fifth step 940, the game predicts a cognitive disability based on the information on cognitive state measured against the neurological or psychiatric conditions. The game may comprise a plurality of games wherein the information on cognitive state is extracted from input signals received throughout at least a portion of the duration of the plurality of games.

[0044] Figure 10 illustrates an example of a method for prediction of a cognitive disability using a device-based game. At a first step 1000, a display module of a communication device displays a game executed on the communication device, the game including game components and at least one of: auditory game objects and visual game objects. At a second step 1010, the game receives input designating control measures. At a third step 1020, the game further receives input signals throughout the duration of the game. At a fourth step 1030, the game extracts performance measures ofbased on the input signals received. At a fifth step 1040, a machine learning-based prediction model of the game predicts a cognitive disability based on the performance measures and the control measures. The game may comprise a plurality of games (e.g., reading fluency games, executive function games, etc.) wherein the performance measures are extracted from input signals received throughout at least a portion of the duration of the plurality of games.

[0045] Embodiments of the present disclosure provide a system and method for providing a device-based game in any number of languages, with the underlying method, modules and components being the same for any language of program delivery, such that the game is language independent. For example, in an embodiment, the same system and method can provide a device-based game in English, Arabic, Hebrew, Spanish, French, Hindi or any other language, or variants / dialects of base languages. A system and method according to an embodiment of the present disclosure provide a device-based game in a user’s first language or native language.

[0046] In an implementation, a system or method according to an embodiment of the present disclosure employs one or more of the following design features: attention; cognition; novelty; and reward. In an example embodiment, displaying the game includes displaying both translucent and opaque moving masks to induce attention. In another example embodiment, displaying the game includes providing a cognition inducing component selected from the group consisting of rhymes, pidgin English and homophones. In an example embodiment, displaying the game includes interspersing repetitive language tasks with non-language mosaics. In an example embodiment, displaying the game includes displaying motion graphics with embedded text. In an example embodiment, displaying the game includes providing a reward including visual and aural feedback.

[0047] In an example embodiment, displaying the game includes displaying fast motion graphics in addition to the auditory game objects and / or the visual game objects. In an example embodiment, the fast motion graphics comprise movement of about 700° / s or higher, or between approximately 700% and approximately 900%, considering the peak angular speed of the eye during a saccade reaches up to 900%.

[0048] In an example embodiment, displaying the game includes displaying both translucent and opaque moving masks to induce attention; providing a cognition inducing component selected from a group consisting of rhymes, pidgin English and homophones; interspersing repetitive language tasks with non-language mosaics; displaying motion graphics with embedded text; providing a reward including visual and aural feedback; displaying fast motion graphics in addition to the auditory game objects and the visual game objects; providing an auditory pathway challenge to address one or more phonological components; providing the auditory pathway challenge to address pronunciation; providing a visual pathway challenge to address one or more orthographic aspects; providing the visual pathway challenge to address spelling; providing a cognitive pathway challenge to address one or more motor or working memory aspects; providing the cognitive pathway challenge to address holding and processing information for a brief period of time.

[0049] Embodiments of the present disclosure provide benefits in one or more of the following environments: educational institutions, such as elementary schools, middle schools, secondary / high schools, colleges, or universities; hospital facilities dealing with conditions such as intellectual disability, ADHD, stress, anxiety, depression etc.; private medical / healthcare practitioners and their clinics, such as clinical psychologists, occupational therapists, vision therapists, reading clinics; and employment readiness programs, such as learning based skill development.

[0050] In the preceding description, for purposes of explanation, numerous details are set forth in order to provide a thorough understanding of the embodiments. However, it will be apparent to one skilled in the art that these specific details are not required. In other instances, well-known electrical structures and circuits are shown in block diagram form in order not to obscure the understanding. For example, specific details are not provided as to whether the embodiments described herein are implemented as a software routine, hardware circuit, firmware, or a combination thereof.

[0051] While various embodiments have been described herein with reference to specific features and implementations, it should be understood that the scope of the invention is not limited to these particular examples. The embodiments may beimplemented in software, hardware, firmware, or any combination thereof. In cases where processes or methods are described, the steps may be performed in any suitable order, and not necessarily in the order illustrated or described, unless explicitly stated otherwise.

[0052] Embodiments of the disclosure can be represented as a computer program product stored in a machine-readable medium (also referred to as a computer-readable medium, a processor-readable medium, or a computer usable medium having a computer-readable program code embodied therein). The machine-readable medium can be any suitable tangible, non-transitory medium, including magnetic, optical, or electrical storage medium including a diskette, compact disk read only memory (CD- ROM), memory device (volatile or non-volatile), or similar storage mechanism. The machine-readable medium can contain various sets of instructions, code sequences, configuration information, or other data, which, when executed, cause a processor to perform steps in a method according to an embodiment of the disclosure. Those of ordinary skill in the art will appreciate that other instructions and operations necessary to implement the described implementations can also be stored on the machine-readable medium. The instructions stored on the machine-readable medium can be executed by a processor or other suitable processing device and can interface with circuitry to perform the described tasks. Further, the various modules, components, or functions described herein may be implemented using any suitable programming language or framework, and may be distributed across multiple devices, systems, or networked environments.

[0053] The above-described embodiments are intended to be examples only. Alterations, modifications and variations can be effected to the particular embodiments by those of skill in the art without departing from the scope, which is defined solely by the claims appended hereto.

Claims

CLAIMS1. A processor-implemented method for prediction of a cognitive disability using a devicebased game, comprising: displaying, with a display module of a communication device, a game executed on the communication device, wherein the game comprises game components and at least one of: auditory game objects and visual game objects; receiving, with the game, via the communication device or a device connected with the communication device, input signals throughout at least a portion of the duration of the game; extracting, with the game, information on cognitive state based on the input signals; measuring, with the game, the information on cognitive state against neurological or psychiatric conditions; and predicting, with the game, a cognitive disability based on the information on cognitive state measured against the neurological or psychiatric conditions.

2. The method of claim 1, wherein the input signals are collected by at least one sensor of the communication device or an input device connected with the communication device.

3. The method of claim 2, wherein the at least one sensor comprises a microphone and a touchscreen sensor or a sensor of an input device connected to the communication device.

4. The method of claim 1, wherein the game is accessible via a software application downloaded on the communication device.

5. The method of claim 1, wherein: a machine-learning based prediction model is used in predicting the cognitive disability; and the information on cognitive state and control measures are provided as input to the machine-learning based prediction model.

6. The method of claim 5, wherein the information of cognitive state comprises information relating to at least one of: attention, hyperactivity, irritability, selfregulation, eye-hand coordination, perception, reading, and listening.

7. The method of claim 5, wherein the control measures comprise at least one of: age, gender, grade, word count per minute, genetics, and epigenetics.

8. The method of claim 5, wherein the machine-learning based prediction model is further used in predicting a learning disability.

9. A processor-implemented method for prediction of a cognitive disability using a devicebased game, comprising: displaying, with a display module of a communication device, a game executed on the communication device, wherein the game comprises game components and at least one of: auditory game objects and visual game objects; receiving, with the game, via the communication device, a first set of inputs designating control measures; receiving, with the game, via the communication device or a device connected with the communication device, a second set of input signals throughout at least a portion of the duration of the game; extracting, with the game, performance measures based on the second set of input signals; and predicting, with the game, a cognitive disability using a machine learning-based prediction model, wherein the performance measures and the control measures are provided as input to the machine learning -based prediction model.

10. The method of claim 9, wherein the input signals are collected by at least one sensor of the communication device or an input device connected with the communication device.

11. The method of claim 10, wherein the at least one sensor comprises a microphone and a touchscreen sensor or a sensor of an input device connected to the communication device.

12. The method of claim 9, wherein the game is accessible via a software application downloaded on the communication device.

13. The method of claim 9, wherein the game further predicts a learning disability using the machine learning -based prediction model.

14. The method of claim 13, wherein the game further predicts dyslexia and ADHD using the machine learning-based prediction model.

15. The method of claim 9, wherein: the performance measures comprise at least one of: attention, timeliness, hyperactivity, impulsiveness, and reading fluency; and the control measures comprise at least one of: age, gender, grade, word count per minute, genetics, and epigenetics.

16. The method of claim 9, wherein the game comprises a plurality of games assessing reading fluency and executive function.

17. The method of claim 16, wherein: the performance measures comprise attention, timeliness, hyperactivity, impulsiveness, and reading fluency; and the control measures comprise age, gender, grade, and word count per minute.

18. The method of claim 17, wherein: attention is measured based on a number of correct responses to a target stimuli including failures to respond to a target stimuli; timeliness is measured based on amount of time between responses; hyperactivity is measured based speed of responses and correct responses; impulsiveness is measured based on a number of responses to a non-target stimuli; and reading fluency is measured based on words correct per minute.

19. The method of claim 17, wherein for a game of the plurality of games assessing reading fluency: the visual game objects comprise a plurality of columns of words; the second set of input signals comprise a first audio input of an individual reading the words in each column from top to bottom, then in each row from left to right; the visual game objects further comprise a passage;the second set of input signals further comprise a second audio input of the individual reading the passage at normal speed; and the words correct per minute is measured based on the second set of input signals.

20. The method of claim 17, wherein a game of the plurality of games assessing executive function comprises continuously and iteratively displaying a visual game object comprising one of: a red hexagon, a blue hexagon, the word red, or the word blue within one of two play areas, wherein: the second set of input signals comprises a first input designating a play area, wherein a goal is to, when the red hexagon or the word red is displayed, select the play area in which the red hexagon or the word red is displayed, and when the blue hexagon or the word blue is displayed, select the play area in which the blue hexagon or the word blue is not displayed; the number of correct responses to a target stimuli including the failures to respond to a target stimuli and the number of responses to a non-target stimuli are measured based on the second set of input signals.

21. The method of claim 9, wherein scenes and assets of the game including the auditory game objects and the visual game objects are generated independent of a language of game delivery.

22. The method of claim 9, wherein displaying the game comprises displaying both translucent and opaque moving masks to induce attention.

23. The method of claim 9, wherein displaying the game comprises providing a cognition inducing component selected from a group consisting of rhymes, pidgin English, and homophones.

24. The method of claim 9, wherein displaying the game comprises interspersing repetitive language tasks with non-language mosaics.

25. The method of claim 9, wherein displaying the game comprises displaying motion graphics with embedded text.

26. The method of claim 9, wherein displaying the game comprises providing a reward comprising visual and aural feedback.

27. The method of claim 9, wherein displaying the game comprises displaying fast motion graphics in addition to the auditory game objects and the visual game objects.

28. The method of claim 9, wherein displaying the game comprises providing an auditory pathway challenge to address one or more phonological components.

29. The method of claim 28, wherein displaying the game comprises providing the auditory pathway challenge to address pronunciation.

30. The method of claim 9, wherein displaying the game comprises providing a visual pathway challenge to address one or more orthographic aspects.

31. The method of claim 30, wherein displaying the game comprises providing the visual pathway challenge to address spelling.

32. The method of claim 9, wherein displaying the game comprises providing a cognitive pathway challenge to address one or more motor or working memory aspects.

33. The method of claim 32, wherein displaying the game comprises providing the cognitive pathway challenge to address holding and processing information for a brief period of time.

34. A non-transitory machine-readable memory storing instructions that when executed by one or more processors effectuates operations comprising: displaying, with a display module of a communication device, a game executed on the communication device, wherein the game comprises game components and at least one of: auditory game objects and visual game objects; receiving, with the game, via the communication device, a first set of inputs designating control measures; receiving, with the game, via the communication device or a device connected with the communication device, a second set of input signals throughout at least a portion of the duration of the game; extracting, with the game, performance measures based on the second set of input signals; and predicting, with the game, a cognitive disability using a machine learning-based prediction model, wherein the performance measures and the control measures are provided as input to the machine learning -based prediction model.

5. An apparatus for providing a device-based game for prediction of a cognitive disability comprising: one or more processors; and a non-transitory machine-readable memory storing instructions that when executed by the one or more processors effectuates operations comprising: displaying, with a display module of a communication device, a game executed on the communication device, wherein the game comprises game components and at least one of: auditory game objects and visual game objects; receiving, with the game, via the communication device, a first set of inputs designating control measures; receiving, with the game, via the communication device or a device connected with the communication device, a second set of input signals throughout at least a portion of the duration of the game; extracting, with the game, performance measures based on the second set of input signals; and predicting, with the game, a cognitive disability using a machine learning-based prediction model, wherein the performance measures and the control measures are provided as input to the machine learning -based prediction model.

Images