Methods, systems and devices for the diagnosis of behavioral disorders, developmental delays and nervous system dysfunction

Abstract

Described herein are methods, devices, systems, software, and platforms used to evaluate individuals, such as children, for behavioral disorders, developmental delays, and nervous system dysfunction. Specifically, described herein are methods, devices, systems, software, and platforms used to analyze video and / or audio recordings of individuals with one or more behavioral disorders, developmental delays, and nervous system dysfunctions.

Description

[Technical field]

[0001] cross reference This application claims the benefit of U.S. Provisional Patent Application No. 62 / 897,217, filed September 6, 2019, which is incorporated herein by reference in its entirety. [Background technology]

[0002] Many individuals, including children, suffer from behavioral disorders, developmental delays, and nervous system dysfunction, examples of which include attention deficit hyperactivity disorder ("ADHD"), autism (including autism spectrum disorders), and language disorders.

[0003] Health care providers typically use traditional observational techniques to assess for behavioral disorders, developmental delays, and neurological dysfunction. Summary of the Invention

[0004] Described herein are methods, devices, systems, software, and platforms that are used to evaluate individuals, such as children, for behavioral disorders, developmental delays, and nervous system dysfunctions.Specifically, described herein are methods, devices, systems, software, and platforms that are used to analyze the video and / or audio of individuals with one or more behavioral disorders, developmental delays, and nervous system dysfunctions.Compared to conventional techniques for evaluating individuals for one or more behavioral disorders, developmental delays, and nervous system dysfunctions, the methods, devices, systems, software, and platforms described herein are highly efficient and accurate.

[0005] Traditionally, behavioral disorders, developmental delays, and nervous system dysfunctions are difficult to assess accurately and efficiently, especially due to the relatedness of these disease types: each disease type category (e.g., behavioral disorders, developmental delays, and nervous system dysfunctions) involves multiple disease types that are typically related within and across different disease type categories such that the disease types have one or more overlapping symptoms or other identifiers.

[0006] Diseases within each single disease type category (e.g., behavioral disorders, developmental delays, or nervous system dysfunction) tend to be associated with one or more overlapping symptoms or other identifiers. For example, a first developmental delay, such as autism, has overlap with a second developmental delay, such as speech delay. As a result, autism may be difficult to distinguish from speech delay using conventional techniques, and thus individuals may be misdiagnosed. Similarly, individuals with both of these developmental delays (e.g., autism and speech delay) may be diagnosed with only one developmental delay, but not both, because the presence of one may be overlooked by the presence of the other (e.g., speech delay may be overlooked in an individual diagnosed with autism, and vice versa).

[0007] Similarly, disease types within a disease type category tend to be associated with one or more overlapping symptoms or other identifiers. For example, ADHD, a type of behavioral disorder, tends to have overlap with autism, a type of developmental delay. As a result, ADHD may be difficult to distinguish from autism using traditional techniques, and thus individuals may be misdiagnosed. Similarly, individuals with both ADHD and autism may be diagnosed with only one condition, but not both, because the presence of one may be overlooked by the presence of the other (e.g., autism may be overlooked in individuals diagnosed with ADHD, and vice versa).

[0008] Conventional techniques for assessing an individual for at least one disease type selected from the disease type categories of behavioral disorders, developmental delays, and nervous system dysfunction generally involve repeated assessment of the individual, often with collection of multiple types of data, including various test findings. For example, conventional techniques may involve a relatively long series of questions given to the individual and / or his / her caregivers. Thus, in addition to being inaccurate due to the relevance of the disease types assessed (as described above), conventional techniques are generally time-consuming and inefficient.

[0009] In contrast to conventional techniques, this specification describes a method, device, system, software, and platform for accurately and efficiently assessing an individual for at least one disease type selected from the disease type categories of behavioral disorder, developmental delay, and nervous system dysfunction.More specifically, this specification describes a method, device, system, software, and platform for analyzing an individual's video and / or audio data for at least one disease type selected from the disease type categories of behavioral disorder, developmental delay, and nervous system dysfunction to determine whether or not at least one disease type exists in the individual, or the likelihood that at least one disease type exists.

[0010] Some embodiments described herein analyze input sources, such as video and / or audio sources, of or associated with the individual to identify behavioral units. For example, video recorded using a mobile computing device is analyzed to identify behavioral units in the recorded video and / or audio. One or more behavioral units constitute higher order behaviors. For example, a behavioral unit including smiling may, in some circumstances, constitute a higher order behavior related to happiness. Additionally, in some embodiments, behavioral units are mapped over time to create a timeline that provides additional context for one or more behavioral units. In some embodiments, a behavioral pattern includes behavioral units and / or higher order behaviors mapped over time.

[0011] In some embodiments of the methods as described herein, video and / or audio of an individual is recorded by a mobile computing device and analyzed using a machine learning software module that identifies one or more behavioral units in the recorded video and / or audio. In some embodiments, the behavioral units are associated with one or more other behavioral units. In some embodiments, the behavioral units are classified as constituting higher order behaviors using a machine learning classifier or other type of machine learning modeling.

[0012] In some embodiments, the identified behavior units are tagged or labeled, for example with respect to the timing of the occurrence of the behavior unit in the recorded video and / or audio, In some embodiments, the tags or labels are used by a machine learning software module to contextualize the behavior units.

[0013] In general, some embodiments described herein comprise a machine learning software module trained to receive input including video and / or audio data, analyze the video and / or audio data, and generate output including at least one of an identified behavior unit, an identified plurality of behavior units, a map of the plurality of behavior units, an identified higher order behavior, an identified plurality of higher order behaviors, and a map of the plurality of higher order behaviors.

[0014] Disclosed herein is a computer-implemented method for automated video assessment of an individual, the method comprising: receiving, with a computing device, input data comprising at least one of audio or video information regarding the individual for whom the automated video assessment is to be performed; identifying, with the computing device, a plurality of behavioral units in the input data, each behavioral unit of the plurality of behavioral units comprising a behavior constituting a higher order behavior; and identifying, with the computing device, the higher order behavior based on at least one behavioral unit of the plurality of behavioral units. In some embodiments, at least one of the plurality of behavioral units comprises a machine-detectable movement or sound made by an individual. In some embodiments, at least one of the plurality of behavioral units comprises a facial movement, a body movement, or a sound made by an individual. In some embodiments, at least one of the higher order behavior comprises a verbal or non-verbal communication. In some embodiments, the non-verbal communication comprises a facial expression, a posture, a gesture, an eye contact, or a touch. In some embodiments, identifying the plurality of behavioral units comprises analyzing the video information using facial recognition to detect one or more facial movements. In some embodiments, identifying the plurality of behavioral units includes analyzing the audio information using speech recognition to detect one or more sounds, words, or phrases. In some embodiments, identifying the higher order behaviors includes generating a timeline of a plurality of behavioral units. In some embodiments, the method further includes generating the behavioral pattern based on the timeline or higher order behaviors of the plurality of behavioral units. In some embodiments, the input data includes information or responses provided by a caregiver of the individual. In some embodiments, the information or responses include answers to questions. In some embodiments, the method further includes generating a prediction including a positive, negative, or indeterminate classification for behavioral disorder, developmental delay, or nervous system dysfunction.In some embodiments, the method further includes obtaining at least one of the audio or video information via a mobile computing device. In some embodiments, the mobile computing device includes a smartphone, tablet computer, laptop, smart watch, or other wearable computing device. In some embodiments, obtaining at least one of the audio or video information includes capturing a video film or audio recording of the individual or of an interaction between the individual and others. In some embodiments, the behavioral disorder, developmental delay, or nervous system dysfunction comprises Pervasive Developmental Disorder (PDD), Autism Spectrum Disorder (ASD), Social Communication Disorder, Restricted Repetitive Behaviors, Interests, and Activities (RRB), Autism ("Classic Autism"), Asperger's Syndrome ("High Functioning Autism"), Pervasive Developmental Disorder Not Otherwise Specified (PDD-NOS, "Atypical Autism"), Attention Deficit Disorder (ADD), Attention Deficit Hyperactivity Disorder (ADHD), Speech and Language Delay, Obsessive Compulsive Disorder (OCD), Depression, Schizophrenia, Alzheimer's Disease, Dementia, Intellectual Disability, or Learning Disability. In some embodiments, the behavioral disorder, developmental delay, or nervous system dysfunction is Autism Spectrum Disorder or Autism. In some embodiments, identifying the plurality of behavioral units in the input data and / or identifying the higher order behaviors is performed using a machine learning software module. In some embodiments, the machine learning algorithm is a supervised learning algorithm. In some embodiments, the machine learning software module is selected from nearest neighbor, naive Bayes, decision tree, linear regression, support vector machine, or neural network. In some embodiments, the method further comprises providing an activity configured to elicit a response by the individual, the response comprising the input data comprising at least one of the audio information or the video information. In some embodiments, the activity provides a virtual character that guides the individual through the activity.In some embodiments, the individual's higher order behaviors or behavioral patterns include speech articulation, fluency, phonation, language comprehension / decoding, language production / use, or any combination thereof. In some embodiments, identifying the plurality of behavioral units includes analyzing the speech information using speech recognition to detect one or more sounds, words, or phrases. In some embodiments, the method further includes adjusting the activity based on the identified higher order behaviors in the individual. In some embodiments, the activity is dynamically adjusted while the individual is engaged in the activity.

[0015] Disclosed herein is a device for automated assessment of an individual, the device comprising a display and a processor, the processor configured with instructions for receiving input data including at least one of audio or video information for the individual for which the automated video assessment is performed, identifying a plurality of behavioral units in the input data, each behavioral unit including behaviors constituting a higher order behavior, and identifying the higher order behavior based on at least one behavioral unit of the plurality of behavioral units. In some embodiments, at least one of the plurality of behavioral units includes a machine-detectable movement or sound made by the individual. In some embodiments, at least one of the plurality of behavioral units includes a facial movement, a body movement, or a sound made by the individual. In some embodiments, at least one of the higher order behavior includes a verbal or non-verbal communication. In some embodiments, the non-verbal communication includes a facial expression, a posture, a gesture, an eye contact, or a touch. In some embodiments, identifying the plurality of behavioral units includes analyzing the video information using facial recognition to detect one or more facial movements. In some embodiments, identifying the plurality of behavioral units includes analyzing the audio information using speech recognition to detect one or more sounds, words, or phrases. In some embodiments, identifying the higher order behavior includes generating a timeline of a plurality of behavioral units. In some embodiments, the processor is further configured with instructions for generating the behavioral pattern based on the timeline or higher order behaviors of the plurality of behavioral units. In some embodiments, the input data includes information or responses provided by a caregiver of the individual. In some embodiments, the information or responses include answers to questions. In some embodiments, the processor is further configured with instructions for generating a prediction including a positive, negative, or indeterminate classification for a behavioral disorder, developmental delay, or nervous system dysfunction. In some embodiments, the device is a mobile computing device.In some embodiments, the mobile computing device comprises a smartphone, tablet computer, laptop, smartwatch, or other wearable computing device. In some embodiments, obtaining at least one of the audio or video information comprises capturing a video film or audio recording of the individual or an interaction between the individual and another person. In some embodiments, the behavioral disorder, developmental delay, or nervous system dysfunction comprises a pervasive developmental disorder (PDD), an autism spectrum disorder (ASD), a social communication disorder, restricted repetitive behaviors, interests, and activities (RRB), autism ("classic autism"), Asperger's syndrome ("high functioning autism"), pervasive developmental disorder not otherwise specified (PDD-NOS, "atypical autism"), attention deficit disorder (ADD), attention deficit hyperactivity disorder (ADHD), speech and language delay, obsessive-compulsive disorder (OCD), depression, schizophrenia, Alzheimer's disease, dementia, intellectual disability, or learning disability. In some embodiments, the behavioral disorder, developmental delay, or nervous system dysfunction is an autism spectrum disorder or autism. In some embodiments, identifying the plurality of behavioral units in the input data and / or identifying the higher order behavior is performed using a machine learning software module. In some embodiments, the machine learning algorithm is a supervised learning algorithm. In some embodiments, the machine learning software module is selected from nearest neighbor, naive Bayes, decision tree, linear regression, support vector machine, or neural network. In some embodiments, the processor is further configured with instructions for providing an activity configured to elicit a response by the individual, the response comprising the input data including at least one of the audio information or the video information. In some embodiments, the activity provides a virtual character that guides the individual through the activity. In some embodiments, the higher order behavior or behavior pattern of the individual comprises speech articulation, fluency, phonation, ability to understand and decode language, ability to produce and use language, or any combination thereof.In some embodiments, identifying the plurality of behavioral units includes analyzing the audio information using speech recognition to detect one or more sounds, words, or phrases. In some embodiments, the processor is further configured with instructions for adjusting the activity based on the identified higher order behaviors in the individual. In some embodiments, the activity is dynamically adjusted while the individual is engaged in the activity.

[0016] Disclosed herein is a platform for automated behavioral assessment of an individual, the platform comprising: a capture application comprising video and / or audio recording software for use on a device capable of capturing one or more video or audio recordings of the individual; and an assessment application configured to receive video and / or audio recordings from the capture application and analyze the video and / or audio recordings using one or more machine learning algorithms, the algorithms configured to receive input data comprising at least one of audio or video information for the individual for whom the automated video assessment is performed; identify a plurality of behavioral units in the input data comprising behaviors that constitute a higher order behavior; and identify the higher order behavior based on at least one of the plurality of behavioral units. In some embodiments, at least one of the plurality of behavioral units comprises a machine-detectable movement or sound made by the individual. In some embodiments, at least one of the plurality of behavioral units comprises a facial movement, a body movement, or a sound made by the individual. In some embodiments, at least one of the higher order behaviors comprises a verbal or non-verbal communication. In some embodiments, the non-verbal communication comprises a facial expression, a posture, a gesture, an eye contact, or a touch. In some embodiments, identifying the plurality of behavior units includes analyzing the video information using facial recognition to detect one or more facial movements. In some embodiments, identifying the plurality of behavior units includes analyzing the audio information using speech recognition to detect one or more sounds, words, or phrases. In some embodiments, identifying the higher order behaviors includes generating a timeline of a plurality of behavior units. In some embodiments, the assessment application is configured to generate the behavior pattern based on the timeline or higher order behaviors of the plurality of behavior units. In some embodiments, the input data includes information or responses provided by a caregiver of the individual. In some embodiments, the information or responses include answers to questions.In some embodiments, the assessment application is configured to generate a prediction comprising a positive, negative, or indeterminate classification for behavioral disorder, developmental delay, or neurological dysfunction. In some embodiments, the device is a mobile computing device. In some embodiments, the mobile computing device comprises a smartphone, tablet computer, laptop, smart watch, or other wearable computing device. In some embodiments, obtaining at least one of the audio or video information comprises capturing a video film or audio recording of the individual or an interaction between the individual and others. In some embodiments, the behavioral disorder, developmental delay, or nervous system dysfunction comprises Pervasive Developmental Disorder (PDD), Autism Spectrum Disorder (ASD), Social Communication Disorder, Restricted Repetitive Behaviors, Interests, and Activities (RRB), Autism ("Classic Autism"), Asperger's Syndrome ("High Functioning Autism"), Pervasive Developmental Disorder Not Otherwise Specified (PDD-NOS, "Atypical Autism"), Attention Deficit Disorder (ADD), Attention Deficit Hyperactivity Disorder (ADHD), Speech and Language Delay, Obsessive Compulsive Disorder (OCD), Depression, Schizophrenia, Alzheimer's Disease, Dementia, Intellectual Disability, or Learning Disability. In some embodiments, the behavioral disorder, developmental delay, or nervous system dysfunction is Autism Spectrum Disorder or Autism. In some embodiments, identifying the plurality of behavioral units in the input data and / or identifying the higher order behaviors is performed using a machine learning software module. In some embodiments, the machine learning algorithm is a supervised learning algorithm. In some embodiments, the machine learning software module is selected from nearest neighbor, naive Bayes, decision tree, linear regression, support vector machine, or neural network. In some embodiments, the assessment application is configured to provide an activity configured to elicit a response by the individual, the response including the input data including at least one of the audio information or the video information.In some embodiments, the activity provides a virtual character that guides the individual through the activity. In some embodiments, the higher order behavior or behavioral pattern of the individual includes speech articulation, fluency, voice, language comprehension / decoding, language production / use, or any combination thereof. In some embodiments, identifying the plurality of behavioral units includes analyzing the speech information using speech recognition to detect one or more sounds, words, or phrases. In some embodiments, the assessment application is configured to adjust the activity based on the higher order behaviors identified in the individual. In some embodiments, the activity is dynamically adjusted while the individual is engaged in the activity.

[0017] Disclosed herein is a computer-implemented method for automated behavioral assessment of an individual, the method comprising: receiving input data, comprising at least one of audio or video information, with a computing device, for the individual; processing the input data, comprising at least one of audio or video information, with the computing device; evaluating the plurality of behavioral units to determine one or more higher order behaviors, with the computing device; evaluating the plurality of behavioral units and higher order behaviors over time to determine one or more behavioral patterns, with the computing device; and generating a prediction of behavioral disorder, developmental delay, or nervous system dysfunction based on the one or more behavioral patterns. In some embodiments, at least one of the plurality of behavioral units comprises an observable action or sound made by the individual. In some embodiments, at least one of the plurality of behavioral units comprises a facial action, a body action, or a sound made by the individual. In some embodiments, at least one of the higher order behaviors corresponds to a verbal or non-verbal communication. In some embodiments, the non-verbal communication comprises a facial expression, a posture, a gesture, an eye contact, or a touch. In some embodiments, evaluating the plurality of behavioral units includes analyzing the video information using facial recognition to detect one or more facial movements. In some embodiments, identifying the plurality of behavioral units includes analyzing the audio information using speech recognition to detect one or more sounds, words, or phrases. In some embodiments, identifying the one or more higher order behaviors includes generating a timeline of a plurality of behavioral units. In some embodiments, the behavioral pattern is generated based on the timeline of the plurality of behavioral units and / or the higher order behaviors. In some embodiments, the input data includes information or responses provided by a caregiver of the individual. In some embodiments, the information or responses include answers to questions. In some embodiments, the prediction includes a positive, negative, or indeterminate classification for behavioral disorder, developmental delay, or nervous system dysfunction.In some embodiments, the device obtains at least one of the audio or video information via a mobile computing device. In some embodiments, the mobile computing device includes a smartphone, tablet computer, laptop, smart watch, or other wearable computing device. In some embodiments, obtaining at least one of the audio or video information includes capturing a video film or audio recording of the individual or of an interaction between the individual and others. In some embodiments, the behavioral disorder, developmental delay, or nervous system dysfunction comprises Pervasive Developmental Disorder (PDD), Autism Spectrum Disorder (ASD), Social Communication Disorder, Restricted Repetitive Behaviors, Interests, and Activities (RRB), Autism ("Classic Autism"), Asperger's Syndrome ("High Functioning Autism"), Pervasive Developmental Disorder Not Otherwise Specified (PDD-NOS, "Atypical Autism"), Attention Deficit Disorder (ADD), Attention Deficit Hyperactivity Disorder (ADHD), Speech and Language Delay, Obsessive Compulsive Disorder (OCD), Depression, Schizophrenia, Alzheimer's Disease, Dementia, Intellectual Disability, or Learning Disability. In some embodiments, the behavioral disorder, developmental delay, or nervous system dysfunction is Autism Spectrum Disorder or Autism. In some embodiments, one or more of the feature detection algorithm, the higher order behavior detection algorithm, the behavior pattern detection algorithm, or the evaluation algorithm is a supervised learning algorithm. One or more of the feature detection algorithm, the higher order behavior detection algorithm, the behavior pattern detection algorithm, or the evaluation algorithm is a supervised learning algorithm selected from nearest neighbor, naive Bayes, decision tree, linear regression, support vector machine, or neural network. In some embodiments, the device includes a camera or microphone capable of capturing audio or video recordings. In some embodiments, the processor is further configured with instructions for providing an activity configured to elicit a response by the individual, the response including the input data including at least one of the audio information or the video information.In some embodiments, the activity provides a virtual character that guides the individual through the activity. In some embodiments, the higher order behavior or behavioral patterns of the individual include speech articulation, fluency, voice, language comprehension / decoding, language production / use, or any combination thereof. In some embodiments, the processor is further configured with instructions for identifying the plurality of behavioral units by analyzing the speech information using speech recognition to detect one or more sounds, words, or phrases. In some embodiments, the processor is further configured with instructions for adjusting the activity based on higher order behavior or behavioral patterns determined in the individual. In some embodiments, the activity is dynamically adjusted while the individual is engaged in the activity.

[0018] Disclosed herein is a device for automated behavioral assessment of an individual, the device comprising a display and a processor, the processor configured with instructions for receiving input data including at least one of audio or video information about the individual, processing the input data to identify a plurality of behavioral units, evaluating the plurality of behavioral units to determine one or more higher order behaviors, evaluating the plurality of behavioral units and higher order behaviors over time to determine one or more behavioral patterns, and making a prediction of behavioral disorder, developmental delay, or nervous system dysfunction based on the one or more behavioral patterns. In some embodiments, at least one of the plurality of behavioral units comprises an observable action or sound made by the individual. In some embodiments, at least one of the plurality of behavioral units comprises a facial action, a body action, or a sound made by the individual. In some embodiments, at least one of the higher order behavior corresponds to a verbal or non-verbal communication. In some embodiments, the non-verbal communication comprises a facial expression, a posture, a gesture, an eye contact, or a touch. In some embodiments, evaluating the plurality of behavioral units comprises analyzing the video information using facial recognition to detect one or more facial actions. In some embodiments, identifying the plurality of behavioral units comprises analyzing the audio information using speech recognition to detect one or more sounds, words, or phrases. In some embodiments, identifying the one or more higher order behaviors comprises generating a timeline of a plurality of behavioral units. In some embodiments, the behavioral pattern is generated based on the timeline of the plurality of behavioral units and / or the higher order behaviors. In some embodiments, the input data comprises information or responses provided by a caregiver of the individual. In some embodiments, the information or responses comprise answers to questions. In some embodiments, the prediction comprises a positive, negative, or indeterminate classification for behavioral disorder, developmental delay, or nervous system dysfunction.In some embodiments, the device obtains at least one of the audio or video information via a mobile computing device. In some embodiments, the mobile computing device includes a smartphone, tablet computer, laptop, smart watch, or other wearable computing device. In some embodiments, obtaining at least one of the audio or video information includes capturing a video film or audio recording of the individual or of an interaction between the individual and others. In some embodiments, the behavioral disorder, developmental delay, or nervous system dysfunction comprises Pervasive Developmental Disorder (PDD), Autism Spectrum Disorder (ASD), Social Communication Disorder, Restricted Repetitive Behaviors, Interests, and Activities (RRB), Autism ("Classic Autism"), Asperger's Syndrome ("High Functioning Autism"), Pervasive Developmental Disorder Not Otherwise Specified (PDD-NOS, "Atypical Autism"), Attention Deficit Disorder (ADD), Attention Deficit Hyperactivity Disorder (ADHD), Speech and Language Delay, Obsessive Compulsive Disorder (OCD), Depression, Schizophrenia, Alzheimer's Disease, Dementia, Intellectual Disability, or Learning Disability. In some embodiments, the behavioral disorder, developmental delay, or nervous system dysfunction is Autism Spectrum Disorder or Autism. In some embodiments, one or more of the feature detection algorithm, the higher order behavior detection algorithm, the behavior pattern detection algorithm, or the evaluation algorithm is a supervised learning algorithm. One or more of the feature detection algorithm, the higher order behavior detection algorithm, the behavior pattern detection algorithm, or the evaluation algorithm is a supervised learning algorithm selected from nearest neighbor, naive Bayes, decision tree, linear regression, support vector machine, or neural network. In some embodiments, the device includes a camera or microphone capable of capturing audio or video recordings. In some embodiments, the processor is further configured with instructions for providing an activity configured to elicit a response by the individual, the response including the input data including at least one of the audio information or the video information.In some embodiments, the activity provides a virtual character that guides the individual through the activity. In some embodiments, the higher order behavior or behavioral patterns of the individual include speech articulation, fluency, voice, language comprehension / decoding, language production / use, or any combination thereof. In some embodiments, the processor is further configured with instructions for identifying the plurality of behavioral units by analyzing the speech information using speech recognition to detect one or more sounds, words, or phrases. In some embodiments, the processor is further configured with instructions for adjusting the activity based on higher order behavior or behavioral patterns determined in the individual. In some embodiments, the activity is dynamically adjusted while the individual is engaged in the activity.

[0019] Disclosed herein is a platform for automated behavioral assessment of an individual, the platform comprising: a capture application comprising video and / or audio recording software for use on a device capable of capturing one or more video or audio recordings of the individual; an assessment application configured to receive video and / or audio recordings from the capture application and analyze the video and / or audio recordings using one or more machine learning algorithms, the algorithms configured to receive input data including at least one of audio or video information about the individual, process the input data to identify a plurality of behavioral units, evaluate the plurality of behavioral units to determine one or more higher order behaviors, evaluate the plurality of behavioral units and higher order behaviors over time to determine one or more behavioral patterns, generate a prediction of behavioral disorder, developmental delay, or neurological dysfunction based on the one or more behavioral patterns, and provide a user with a prediction of behavioral disorder, developmental delay, or neurological dysfunction present in the individual; and a health care practitioner application configured to receive data from the assessment application and display a treatment recommendation. In some embodiments, at least one of the plurality of behavioral units comprises an observable action or sound made by the individual. In some embodiments, at least one of the plurality of behavioral units comprises a facial movement, a body movement, or a sound made by an individual. In some embodiments, at least one of the higher order behaviors corresponds to verbal or non-verbal communication. In some embodiments, the non-verbal communication comprises facial expression, posture, gesture, eye contact, or touch. In some embodiments, evaluating the plurality of behavioral units comprises analyzing the video information using facial recognition to detect one or more facial movements. In some embodiments, identifying the plurality of behavioral units comprises analyzing the audio information using speech recognition to detect one or more sounds, words, or phrases.In some embodiments, identifying the one or more higher order behaviors includes generating a timeline of a plurality of behavioral units. In some embodiments, the behavioral pattern is generated based on the timeline of the plurality of behavioral units and / or the higher order behaviors. In some embodiments, the input data includes information or responses provided by a caregiver of the individual. The platform of claim 40, wherein the information or responses include answers to questions. In some embodiments, the prediction includes a positive, negative, or indeterminate classification of a behavioral disorder, developmental delay, or nervous system dysfunction. In some embodiments, the platform obtains at least one of the audio or video information via a mobile computing device. In some embodiments, the mobile computing device includes a smartphone, tablet computer, laptop, smartwatch, or other wearable computing device. In some embodiments, obtaining at least one of the audio or video information includes capturing a video film or audio recording of the individual or an interaction between the individual and others. In some embodiments, the behavioral disorder, developmental delay, or nervous system dysfunction comprises Pervasive Developmental Disorder (PDD), Autism Spectrum Disorder (ASD), Social Communication Disorder, Restricted Repetitive Behaviors, Interests, and Activities (RRB), Autism ("Classic Autism"), Asperger's Syndrome ("High Functioning Autism"), Pervasive Developmental Disorder Not Otherwise Specified (PDD-NOS, "Atypical Autism"), Attention Deficit Disorder (ADD), Attention Deficit Hyperactivity Disorder (ADHD), Speech and Language Delay, Obsessive Compulsive Disorder (OCD), Depression, Schizophrenia, Alzheimer's Disease, Dementia, Intellectual Disability, or Learning Disability. In some embodiments, the behavioral disorder, developmental delay, or nervous system dysfunction is Autism Spectrum Disorder or Autism. In some embodiments, one or more of the feature detection algorithm, the higher order behavior detection algorithm, the behavior pattern detection algorithm, or the evaluation algorithm is a supervised learning algorithm.One or more of the feature detection algorithm, the higher order behavior detection algorithm, the behavior pattern detection algorithm, or the evaluation algorithm are supervised learning algorithms selected from nearest neighbor, naive Bayes, decision tree, linear regression, support vector machine, or neural network. In some embodiments, the platform is capable of capturing one or more video or audio recordings of the individual and includes a capture application and an evaluation application. In some embodiments, the evaluation application is configured to provide an interactive module including an activity configured to elicit a response by the individual, the response including the input data including at least one of the audio information or the video information. In some embodiments, the activity provides a virtual character that guides the individual through the activity. In some embodiments, the higher order behavior or behavior pattern of the individual includes speech sound articulation, fluency, phonation, language comprehension and decoding, language production and use, or any combination thereof. In some embodiments, the evaluation application is configured to identify the plurality of behavioral units by analyzing the audio information using speech recognition to detect one or more sounds, words, or phrases. In some embodiments, the interactive module is configured to adjust the activity based on higher order behaviors or behavioral patterns determined in the individual, hi some embodiments, the activity is dynamically adjusted while the individual is engaged in the activity.

[0020] Disclosed herein is a computer-implemented method for automated speech or language assessment and treatment of an individual, the method including: (a) using one or more computing devices, providing an interactive module that encourages the individual to engage in one or more digital therapeutic activities; (b) using the one or more computing devices, receiving input data including at least one of audio or video information regarding the individual engaged in the one or more digital therapeutic activities; (c) using the one or more computing devices, identifying a plurality of behavioral units in the input data, each behavioral unit of the plurality of behavioral units including a behavior that constitutes a higher order behavior; (d) using the one or more computing devices, identifying the higher order behavior based on at least one behavioral unit of the plurality of behavioral units; and (e) using the one or more computing devices, adjusting at least one of the one or more digital therapeutic activities based on the higher order behavior. In some embodiments, at least one of the plurality of behavioral units includes a machine-detectable movement or sound made by an individual. In some embodiments, at least one of the plurality of behavioral units includes a facial movement, a body movement, or a sound made by an individual. In some embodiments, at least one of the higher order behaviors includes verbal or non-verbal communication. In some embodiments, the non-verbal communication includes facial expression, posture, gesture, eye contact, or touch. In some embodiments, identifying the plurality of behavior units includes analyzing the video information using facial recognition to detect one or more facial movements. In some embodiments, identifying the plurality of behavior units includes analyzing the audio information using speech recognition to detect one or more sounds, words, or phrases. In some embodiments, identifying the higher order behaviors includes generating a timeline of a plurality of behavior units. In some embodiments, the method further includes generating a behavior pattern based on the timeline or higher order behaviors of the plurality of behavior units.In some embodiments, the input data includes information or responses provided by a caregiver of the individual. In some embodiments, the information or responses include answers to questions. In some embodiments, the method further includes generating a prediction including a positive, negative, or indeterminate classification for behavioral disorder, developmental delay, or nervous system dysfunction. In some embodiments, the method further includes obtaining at least one of the audio or video information via a mobile computing device. In some embodiments, the mobile computing device includes a smartphone, tablet computer, laptop, smartwatch, or other wearable computing device. In some embodiments, obtaining at least one of the audio or video information includes capturing a video film or audio recording of the individual or an interaction between the individual and others. In some embodiments, the behavioral disorder, developmental delay, or nervous system dysfunction comprises Pervasive Developmental Disorder (PDD), Autism Spectrum Disorder (ASD), Social Communication Disorder, Restricted Repetitive Behaviors, Interests, and Activities (RRB), Autism ("Classic Autism"), Asperger's Syndrome ("High Functioning Autism"), Pervasive Developmental Disorder Not Otherwise Specified (PDD-NOS, "Atypical Autism"), Attention Deficit Disorder (ADD), Attention Deficit Hyperactivity Disorder (ADHD), Speech and Language Delay, Obsessive Compulsive Disorder (OCD), Depression, Schizophrenia, Alzheimer's Disease, Dementia, Intellectual Disability, or Learning Disability. In some embodiments, the behavioral disorder, developmental delay, or nervous system dysfunction is Autism Spectrum Disorder or Autism. In some embodiments, identifying the plurality of behavioral units in the input data and / or identifying the higher order behaviors is performed using a machine learning software module. In some embodiments, the machine learning algorithm is a supervised learning algorithm. In some embodiments, the machine learning software module is selected from nearest neighbor, naive Bayes, decision trees, linear regression, support vector machines, or neural networks.In some embodiments, the one or more digital therapeutic activities are configured to elicit a response by the individual, the response being detected as the input data including at least one of the audio or video information. In some embodiments, the activity provides a virtual character to guide the individual through the activity. In some embodiments, the individual's higher order behavior or behavioral patterns include speech articulation, fluency, voice, language comprehension / decoding, language production / use, or any combination thereof. In some embodiments, identifying the plurality of behavioral units includes analyzing the audio information using speech recognition to detect one or more sounds, words, or phrases. In some embodiments, the activity is dynamically adjusted while the individual is engaged in the activity.

[0021] Disclosed herein is a device for automated speech or language assessment and treatment of an individual, the device comprising: (a) a display; and (b) a processor configured with instructions for: (i) providing an interactive module for encouraging the individual to engage in one or more digital therapeutic activities; (ii) receiving input data including at least one of audio or video information regarding the individual engaged in the one or more digital therapeutic activities; (iii) identifying a plurality of behavioral units in the input data, each behavioral unit including a behavior constituting a higher order behavior; (iv) identifying the higher order behavior based on at least one behavioral unit of the plurality of behavioral units; and (v) adjusting at least one of the one or more digital therapeutic activities based on the higher order behavior. In some embodiments, at least one of the plurality of behavioral units includes a machine-detectable movement or sound made by an individual. In some embodiments, at least one of the plurality of behavioral units includes a facial movement, a body movement, or a sound made by an individual. In some embodiments, at least one of the higher order behaviors includes verbal or non-verbal communication. In some embodiments, the non-verbal communication includes facial expressions, posture, gestures, eye contact, or touch. In some embodiments, identifying the plurality of behavioral units includes analyzing the video information using facial recognition to detect one or more facial movements. In some embodiments, identifying the plurality of behavioral units includes analyzing the audio information using speech recognition to detect one or more sounds, words, or phrases. In some embodiments, identifying the higher order behavior includes generating a timeline of a plurality of behavioral units. In some embodiments, the processor is further configured with instructions for generating the behavioral pattern based on the timeline or higher order behaviors of a plurality of behavioral units. In some embodiments, the input data includes information or responses provided by a caregiver of the individual. In some embodiments, the information or responses include answers to questions.In some embodiments, the processor is further configured with instructions for generating a prediction comprising a positive, negative, or indeterminate classification for behavioral disorder, developmental delay, or nervous system dysfunction. In some embodiments, the processor is further configured with instructions for obtaining at least one of the audio or video information via a mobile computing device. In some embodiments, the mobile computing device comprises a smartphone, tablet computer, laptop, smartwatch, or other wearable computing device. In some embodiments, obtaining at least one of the audio or video information comprises capturing a video film or audio recording of the individual or an interaction between the individual and others. In some embodiments, the behavioral disorder, developmental delay, or nervous system dysfunction comprises Pervasive Developmental Disorder (PDD), Autism Spectrum Disorder (ASD), Social Communication Disorder, Restricted Repetitive Behaviors, Interests, and Activities (RRB), Autism ("Classic Autism"), Asperger's Syndrome ("High Functioning Autism"), Pervasive Developmental Disorder Not Otherwise Specified (PDD-NOS, "Atypical Autism"), Attention Deficit Disorder (ADD), Attention Deficit Hyperactivity Disorder (ADHD), Speech and Language Delay, Obsessive Compulsive Disorder (OCD), Depression, Schizophrenia, Alzheimer's Disease, Dementia, Intellectual Disability, or Learning Disability. In some embodiments, the behavioral disorder, developmental delay, or nervous system dysfunction is Autism Spectrum Disorder or Autism. In some embodiments, identifying the plurality of behavioral units in the input data and / or identifying the higher order behaviors is performed using a machine learning software module. In some embodiments, the machine learning algorithm is a supervised learning algorithm. In some embodiments, the machine learning software module is selected from nearest neighbor, naive Bayes, decision tree, linear regression, support vector machine, or neural network. In some embodiments, the one or more digital therapeutic activities are configured to elicit a response by the individual, the response being detected as the input data includes at least one of the audio or video information.In some embodiments, the activity provides a virtual character that guides the individual through the activity. In some embodiments, the higher order behavior or behavioral pattern of the individual includes speech articulation, fluency, voice, language comprehension / decoding, language production / use, or any combination thereof. In some embodiments, identifying the plurality of behavioral units includes analyzing the speech information using speech recognition to detect one or more sounds, words, or phrases. In some embodiments, the activity is dynamically adjusted while the individual is engaged in the activity.

[0022] Disclosed herein is a platform for automated speech or language assessment and treatment of an individual, the platform comprising: (a) a capture application comprising video and / or audio recording software for use on a device capable of capturing one or more video or audio recordings of the individual; and (b) an assessment and treatment application configured to (i) provide an interactive module for encouraging the individual to engage in one or more digital therapeutic activities; (ii) receive input data comprising one of the video or audio recordings of the individual engaged in the one or more digital therapeutic activities; (iii) identify a plurality of behavioral units in the input data, each behavioral unit comprising a behavior constituting a higher order behavior; (iv) identify the higher order behavior based on at least one behavioral unit of the plurality of behavioral units; and (v) adjust at least one of the one or more digital therapeutic activities based on the higher order behavior. In some embodiments, at least one of the plurality of behavioral units comprises a machine-detectable movement or sound made by an individual. In some embodiments, at least one of the plurality of behavioral units comprises a facial movement, a body movement, or a sound made by an individual. In some embodiments, at least one of the higher order behaviors includes verbal or non-verbal communication. In some embodiments, the non-verbal communication includes facial expression, posture, gesture, eye contact, or touch. In some embodiments, identifying the plurality of behavioral units includes analyzing the video information using facial recognition to detect one or more facial movements. In some embodiments, identifying the plurality of behavioral units includes analyzing the audio information using speech recognition to detect one or more sounds, words, or phrases. In some embodiments, identifying the higher order behaviors includes generating a timeline of a plurality of behavioral units. In some embodiments, the processor is further configured with instructions for generating the behavior pattern based on the timeline or higher order behaviors of a plurality of behavioral units.In some embodiments, the input data includes information or responses provided by a caregiver of the individual. In some embodiments, the information or responses include answers to questions. In some embodiments, the processor is further configured with instructions for generating a prediction including a positive, negative, or indeterminate classification for behavioral disorders, developmental delay, or nervous system dysfunction. In some embodiments, the processor is further configured with instructions for obtaining at least one of the audio or video information via a mobile computing device. In some embodiments, the mobile computing device includes a smartphone, tablet computer, laptop, smartwatch, or other wearable computing device. In some embodiments, obtaining at least one of the audio or video information includes capturing a video film or audio recording of the individual or an interaction between the individual and others. In some embodiments, the behavioral disorder, developmental delay, or nervous system dysfunction comprises Pervasive Developmental Disorder (PDD), Autism Spectrum Disorder (ASD), Social Communication Disorder, Restricted Repetitive Behaviors, Interests, and Activities (RRB), Autism ("Classic Autism"), Asperger's Syndrome ("High Functioning Autism"), Pervasive Developmental Disorder Not Otherwise Specified (PDD-NOS, "Atypical Autism"), Attention Deficit Disorder (ADD), Attention Deficit Hyperactivity Disorder (ADHD), Speech and Language Delay, Obsessive Compulsive Disorder (OCD), Depression, Schizophrenia, Alzheimer's Disease, Dementia, Intellectual Disability, or Learning Disability. In some embodiments, the behavioral disorder, developmental delay, or nervous system dysfunction is Autism Spectrum Disorder or Autism. In some embodiments, identifying the plurality of behavioral units in the input data and / or identifying the higher order behaviors is performed using a machine learning software module. In some embodiments, the machine learning algorithm is a supervised learning algorithm. In some embodiments, the machine learning software module is selected from nearest neighbor, naive Bayes, decision trees, linear regression, support vector machines, or neural networks.In some embodiments, the one or more digital therapeutic activities are configured to elicit a response by the individual, the response being detected as the input data including at least one of the audio or video information. In some embodiments, the activity provides a virtual character to guide the individual through the activity. In some embodiments, the individual's higher order behavior or behavioral pattern includes speech articulation, fluency, voice, language comprehension / decoding, language production / use, or any combination thereof. In some embodiments, identifying the plurality of behavioral units includes analyzing the audio information using speech recognition to detect one or more sounds, words, or phrases. In some embodiments, the activity is dynamically adjusted while the individual is engaged in the activity.

[0023] The novel features of the invention are set forth with particularity in the appended claims. To better understand the features and advantages of the present invention, reference should be made to the following detailed description that sets forth illustrative embodiments in which the principles of the invention are utilized and the accompanying drawings in which: [Brief description of the drawings]

[0024] [Figure 1] FIG. 1 shows an image representation from which behavioral units are extracted. [Diagram 2] FIG. 1 shows a representation of how behavioral units and higher order behaviors are mapped over time to produce behavioral patterns. [Diagram 3] FIG. 1 is a schematic diagram of an exemplary data processing module for providing the machine learning algorithms and methods described herein. [Figure 4] FIG. 1 illustrates a computing device suitable for incorporation into the platforms, devices, and methods described herein. [Figure 5A] 11A-11C are diagrams illustrating non-limiting examples of graphic displays in which a virtual character provides guidance to a user during an action. [Figure 5B]FIG. 13 shows a non-limiting example of a graphic display in which a story sequence is presented during an activity. [Figure 5C] FIG. 13 illustrates a non-limiting example of a graphical display in which questions are presented during an activity. [Figure 6A] FIG. 13 illustrates a non-limiting example of a graphic display in which a virtual character gives instructions to a user during an action. [Figure 6B] FIG. 1 shows a non-limiting example of a graphic display with a story sequence corresponding to a story. [Figure 7A] 13A-13C are diagrams illustrating non-limiting examples of graphic displays in which a virtual character prompts a user to review visual narrative sequences and come up with a story that corresponds to the narrative sequences. [Figure 7B] FIG. 7B shows a non-limiting example of the visual story sequence of FIG. 7A consisting of a single picture. [Figure 8] FIG. 2 shows a non-limiting example of a graphic display with a series of images arranged by a user into the correct narrative sequence. [Figure 9] FIG. 1 illustrates a non-limiting example of a graphical display with an interface through which a user creates a story or narrative. [Figure 10] FIG. 1 illustrates a non-limiting example of a graphical display with activities providing an embedding intervention via social stories. [Figure 11] FIG. 1 illustrates a non-limiting example of a graphical display with activities that provide game-based life skill training. [Figure 12A] FIG. 13 illustrates a non-limiting example of a graphical display with the results of an evaluation. [Figure 12B] FIG. 13 illustrates a non-limiting example of a graphical display with the results of an evaluation. [Figure 12C] FIG. 13 illustrates a non-limiting example of a graphical display with the results of an evaluation. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] Described herein are methods, devices, systems, software, and platforms used to evaluate individuals, such as children, for behavioral disorders, developmental delays, and neurological dysfunction.

[0026] In embodiments described herein, at least one video and / or audio input is analyzed and at least one behavior unit is identified therein, which in some embodiments is paired or associated with one or more other behavior units.

[0027] In some embodiments, one or more behavioral units make up the higher order behavior that is identified.

[0028] In some embodiments, a timeline representation is generated that indicates the location of at least one behavioral unit within it.

[0029] Action Unit Generally, a behavioral unit is an observable characteristic or parameter displayed by an individual being evaluated for a particular disease and / or by individuals interacting with said individual. Additionally, a behavioral unit is generally a particular characteristic or parameter useful for evaluating an individual being evaluated for a particular disease. That is, one or more behavioral units that are observed or determined are used in embodiments of the methods, devices, systems, software, and platforms described herein to arrive at an assessment of the presence of a disease in the individual being evaluated.

[0030] In some embodiments, a behavioral unit is a movement or sound of an individual that is detectable and clinically relevant to the possible presence or absence of a disease, such as a behavioral disorder, developmental delay, or nervous system dysfunction.

[0031] In general, in embodiments of the methods, devices, systems, software, and platforms described herein, one or more behavioral units are used to assess an individual to determine the likely presence or absence of one or more behavioral disorders, developmental delays, and nervous system dysfunction in said individual.

[0032] In some embodiments, the behavioral unit is a movement or sound of the individual being analyzed. In some embodiments, the movement or sound is observed by a person viewing a video or audio recording of the individual. In some embodiments, the movement or sound is detected by a software module that analyzes a video or audio recording of the individual. In some embodiments, the movement is in response to a stimulus, such as a question, a game, or an event.

[0033] For example, a child being evaluated for behavioral disorders, developmental delays, and / or neurological dysfunction is asked a question and responds with a smile, a shrug of the shoulders, and an audible "I don't know." In some embodiments, the smile, the shrug of the shoulders, and the verbal communication are each individual behavioral units.

[0034] In some embodiments, the behavioral units are facial movements made by the individual being rated or by other individuals interacting with the individual being rated. In some embodiments, behavioral units such as facial movements are used to determine higher order behaviors such as facial expressions.

[0035] Non-limiting examples of facial actions that constitute behavioral units include smiling, frowning, moving lips, raising or lowering lip height, tightening lips, biting lip, opening or closing mouth, pursing lips, pursing lips, licking lips, showing teeth, showing inside of mouth, moving face in relation to speech or language noises, frowning, relaxing eyebrows, sucking or rubbing cheeks (buffing), raising eyebrows, lowering eyebrows, raising inner eyebrow, lowering inner eyebrow, raising outer eyebrow, lowering outer eyebrow, raising one or both cheeks, raising one or both eyelids, lowering one or both eyelids, tightening one or both eyelids, pulling lip corners, and indenting lip corners. , chin up, tongue out, lips stretched, lips pressed, lips parted, chin down, mouth stretched, looking in any direction, eyes rolling, eyes crossed, eyes wide open, eyes focused, ignoring people or objects, eyes moving quickly, blinking, eyelid fluttering, winking, eyes closed, eyes open, eyes narrowed or squinted, nose movement, nostril flaring, nostril pinching, nose sniffing, cheek puffing, mouth blowing, cheek sucking, cheek puffing with tongue, jaw clenching, jaw moving to the side, jaw jutting or tucking, swallowing, chewing, ear movement, or facial twitching. In some embodiments, some examples, such as smiling, can be a behavioral unit in itself while being a combination of other individual behavioral units. For example, a smile may include the individual behavioral units of baring teeth, squinting, raising eyebrows, hollowing cheeks, raising skin lines on the forehead, and skin coloration.

[0036] In some embodiments, a behavioral unit is a physical movement made by the individual being assessed or by an individual interacting with the individual being assessed.

[0037] Non-limiting examples of physical movements that make up a behavioral unit include head movement from side to side, head movement up and down, head tilt, head thrust forward, head pull back, head shaking, nodding, head shaking up and down, head tilting from side to side, shrugging, trembling, shuddering, moving arms, moving legs, moving hands, moving feet, moving abdomen, moving neck, moving shoulders, moving pelvis, tilting head, stretching neck, jerking head or neck, repetitive movements, tapping fingers, repeatedly opening and closing hands, and gesturing with one or both hands. Examples of behavioral behaviors include: making a gesture, waving one or both hands, pointing, making involuntary movements, losing balance, compensating for lost balance, writhing face or limbs, twitching body or limbs, rocking body, clapping arms, rolling body, running or walking back and forth, staring at a light, biting lips or body, hitting head, being unable to move one side of the body without moving the other, moving sluggishly or lethargically, shifting eyes frequently, bending joints, hitting other people or objects, controlling objects in an abnormal way, and tensing muscles. In some embodiments, some examples such as rocking body can be a behavioral unit in itself while being a combination of other individual behavioral units. For example, rocking body can include individual behavioral units such as moving body, thrusting head forward, and pulling head back.

[0038] In some embodiments, the behavioral unit is a sound made by the individual being assessed or by an individual interacting with the individual being assessed, hi some embodiments, the sound is made by the individual's control of their mouth, body, or an object.

[0039] Non-limiting examples of sounds produced from the mouth that constitute behavioral units include verbal communication, clearly audible and understandable speech, slurred speech, whispering, mumbling, speech into or away from a recording device, talking, shouting, singing, humming, high-pitched speech, speech with modulated volume or tone, speech with a trembling voice, rapid speech, slow speech, babbling, crying, coughing, sneezing, snoring, burping, moaning, laughing, gasping, hiccuping, audible exhalation or inhalation, clicking the tongue, whistling, wheezing, imitating noises, imitating flatulence, imitating animal sounds, musical or rhythmic noises, speaking in various languages, snoring, sighing, slurping, or yawning. In some embodiments, the sounds emanating from the body include clapping, lightly or firmly tapping a part of the body or an object with one or both hands, cracking bones such as the fingers or neck, tapping fingers or legs, or tapping arms.

[0040] In some embodiments, sounds, words, or phrases uttered by an individual are used to determine the degree of speech and / or language development. Such sounds may include speech sounds or phonemes that constitute units of words. The sounds may be detected and / or analyzed based on a particular language environment (e.g., sound detection is customized for the language). Non-limiting examples of speech sounds or phonemes in English include / b / , / pbp / , / m / , / n / , / t / , / d / , / k / , / g / , / f / , / s / , / y / , / ae / , / e / , and / oυ / . English may include 24 consonant phonemes and 20 vowel phonemes. Alternatively, non-English language sounds or phonemes are used to determine the degree of speech and / or language development.

[0041] Non-limiting examples of sounds made by one or more objects under the control of an individual that constitute behavioral units include hitting an object with another object, entity, or individual, or causing an object to make a noise. In some embodiments, a behavioral unit is a qualitative observation of a sound made by an individual, such as the presence of stuttering, slurred speech, tongue twisting, monotonous speech, or abnormal intonation.

[0042] In some embodiments, a behavioral unit includes a facial movement, a body movement, or a lack of sound by an individual. For example, if an individual is asked a direct question and then does not respond with a facial movement, body movement, or sound, non-response to the stimulus can be a behavioral unit.

[0043] Identifying behavioral units Behavioral units are identified from an observable analysis session with an individual being evaluated for a condition. The observable analysis session may include one or more questioning sessions in which diagnostic questions are posed to the individual being evaluated. The observable analysis session may include one or more sessions in which the individual being evaluated is observed in interaction with another individual. In some cases, the observable analysis session includes an interactive activity (e.g., a storytelling activity) during which the individual is guided through the activity and prompted or otherwise guided to provide feedback. This activity serves as input data that is analyzed to identify behavioral units.

[0044] Observation may be of a live or real-time session, or of a recording of an observable session. For example, a human may observe a live session. For example, a machine may perform real-time video and / or audio analysis of the session using video or audio recording techniques, where video and / or audio input to the machine via the video or audio recording techniques is analyzed in real-time by a software module, which is a component of the machine. Similarly, a human or machine may be used to analyze a recorded video or audio session that was recorded at another time.

[0045] Analysis of the observable session includes identification of one or more behavioral units. In some embodiments, the one or more behavioral units are used to determine the presence or absence of a behavioral disorder, developmental delay, or nervous system dysfunction.

[0046] In some embodiments, behavior units are identified by a person (referred to as a "person identifier"). In some embodiments, behavior units are created by a person identifying facial movements, body movements, or sounds in an individual's video and audio data. In some embodiments, a person analyzes images of an individual and identifies the location of behavior units in each image. In some embodiments, a person watches videos of an individual and identifies the location of behavior units in the videos in terms of location within a frame and time of occurrence. In some embodiments, image analysis software is used by a person to provide clearer image data on which to base behavior units. In some embodiments, a human analyzes the sounds of an individual and identifies periods during which behavior units are present. In some embodiments, audio extraction and analysis software is used to help filter out background noise and irrelevant noise to provide clearer behavior unit data.

[0047] In some embodiments, the behavioral units are identified by a software module. In some embodiments, the behavioral units are created by the software module identifying facial movements, body movements, or sounds in the individual's video and audio data. In some embodiments, the software module analyzes images of the individual and identifies the location of the behavioral units in each image. In some embodiments, the software module analyzes images of the individual and identifies the location of the behavioral units in each sound. In some embodiments, the software module utilizes recognition software to determine the individual's face and body. In some embodiments, identifying the multiple behavioral units includes analyzing the video information using facial recognition to detect one or more facial movements. In some embodiments, identifying the multiple behavioral units includes analyzing the audio information using speech recognition to detect one or more sounds, words, or phrases.

[0048] In some embodiments, a behavioral unit is any visible or audible observation that can be determined by a computerized algorithm that analyzes the video data or audible. In some embodiments, the video or audio data has a temporal component, and such video or audio frames are time-stamped and analyzed in relation to the video and audio data within a particular time proximity of the time-stamped video or audio data. In some embodiments, a behavioral unit may be a bodily movement or a micro-expression that is imperceptible to an observer, but is observable by an algorithm that analyzes the video or audio data over time. For example, a computer analysis of a video of an individual is performed in which blinking is identified as a behavioral unit. The observer does not identify an eye abnormality, but the computer algorithm indicates that the individual blinks at an abnormal rate or has an increased rate of blinking when asked a question. Alternatively, in some embodiments, one or more behavioral units in the audio or video data are analyzed with each image or audio clip, without regard to other images or audio clips. For example, facial movements that perform a particular frown have previously been identified as a behavioral unit associated with individuals with autistic disorders. Computer analysis of an individual's images will show the presence or absence of the behavioral unit regardless of the time the frown occurs, and the data will be used for subsequent analysis or individual to determine a diagnosis.

[0049] In some embodiments, behavioral units are identified by both human and software module identification. In some embodiments, behavioral units are identified by software or human identifiers, and the identified behavioral units are then used as training data for training a machine learning software module, which is then used to identify behavioral units in new images and audio. For example, in some embodiments, software (or a human) is utilized to identify an individual's morphology in a video or image and associate the identified behavioral units with locations on the individual's body. For example, software (or a human) may be used to determine whether a human face is present in the video or image being analyzed. If a human face is present and a behavioral unit is identified within an area identified as a human face, the behavioral unit may be mapped to a location on the face. For example, software (or a human) identifies eyebrows as a behavioral unit across many video images using software that selects locations in the video images where eyebrow movement is present. The software (or a human) determines that the image contains a human face and maps the location of the eyebrows to the human face. The identified and mapped images are used to train a machine learning algorithm. Because eyebrows must be mapped to a location on the face, the algorithm can better determine whether a new video image with objects that appear to resemble eyebrows should be recognized as eyebrows.

[0050] In some embodiments, identifying the behavioral units includes analyzing the video information using facial recognition software to detect one or more facial movements. In some embodiments, identifying the behavioral units includes analyzing the audio information using speech recognition software to detect one or more sounds, words, or phrases. In some embodiments, the speech recognition software analyzes the audio information to detect sounds, words, or phrases corresponding to speech activity. A speech detection algorithm may be utilized to detect speech activity by performing one or more steps such as pre-processing the audio information to remove background noise (e.g., spectral subtraction), identifying behavioral units corresponding to sounds, words, or phrases indicative of speech activity, or classifying portions of the audio information as speech or non-speech activity. In some embodiments, a speech detection algorithm detects the phonemes that make up spoken language and analyzes sequences of phonemes to identify words and phrases to identify the presence or absence of a speech and / or language disorder or deficit, for example, by evaluation of speech and / or language aspects such as speech articulation, fluency, voice, language comprehension and decoding, and language production and use. In some embodiments, the detected speech is parsed into constituent units, compared to a baseline or reference standard, and / or evaluated using one or more speech or language metrics to generate an assessment and determine the subject's speech or language level.

[0051] Evaluation of an individual in a specific condition using one or more of the identified behavioral units In some embodiments, the software module uses the one or more behavioral units identified to determine whether one or more conditions may be present in the individual being evaluated. The conditions determined to be present may include one or more of a behavioral disorder, a developmental delay, and a nervous system dysfunction.

[0052] In some embodiments, the software module is configured to identify the possible presence of one or more behavioral disorders, including Attention Deficit Hyperactivity Disorder (ADHD), Oppositional Defiant Disorder (ODD), Autism Spectrum Disorder (ASD), anxiety disorder, depression, bipolar disorder, learning disability, or conduct disorder. In some embodiments, the Attention Deficit Hyperactivity Disorder comprises Predominantly Inattentive ADHD, Predominantly Hyperactive-Impulsive type ADHD, or Combined Hyperactive-Impulsive and Inattentive ADHD. In some embodiments, the Autism Spectrum Disorder comprises Autistic Disorder (Classic Autism), Asperger's Syndrome, Pervasive Developmental Disorder (Atypical Autism), or Childhood Disintegrative Disorder. In some embodiments, the Anxiety Disorder comprises Panic Disorder, Phobia, Social Anxiety Disorder, Obsessive-Compulsive Disorder, Separation Anxiety Disorder, Illness Anxiety Disorder (Hypochondriasis), or Post-Traumatic Stress Disorder. In some embodiments, the depression comprises major depression, persistent depressive disorder, bipolar disorder, seasonal affective disorder, psychotic depression, postpartum depression, premenstrual dysphoric disorder, "situational" depression, or atypical depression. In some embodiments, the bipolar disorder comprises bipolar disorder I, bipolar disorder II, cyclothymic disorder, or bipolar disorder due to another medical or substance abuse disorder. In some embodiments, the learning disorder comprises dyslexia, dyscalculia, dysgraphia, dyspraxia (sensory integration disorder), dysphagia / aphasia, auditory processing disorder, or visual processing disorder. In some embodiments, the behavioral disorder is a disorder defined in the Diagnostic and Statistical Manual of Mental Disorders (DSM), full edition.

[0053] In some embodiments, the software module is configured to determine the possible presence or absence of one or more developmental delays, including autism spectrum disorder, mental retardation, cerebral palsy, Down syndrome, failure to thrive, muscular dystrophy, hydrocephalus, developmental coordination disorder, cystic fibrosis, fetal alcohol syndrome, homocystinuria, tuberous sclerosis, abetalipoproteinemia, phenylketonuria, speech delay, gross motor delay, fine motor delay, social delay, emotional delay, behavioral delay, or cognitive delay. In some embodiments, the mental retardation comprises adrenoleukodystrophy, Ito syndrome, acroosteoporosis, Huntington's disease, Arskog syndrome, Aicardi syndrome, or Tay-Sachs disease. In some embodiments, the cerebral palsy comprises spastic cerebral palsy, dyskinetic cerebral palsy, hypotonic cerebral palsy, ataxic cerebral palsy, or mixed cerebral palsy. In some embodiments, the autism spectrum disorder comprises autistic disorder (classic autism), Asperger syndrome, pervasive developmental disorder (atypical autism), or childhood disintegrative disorder. In some embodiments, the Down syndrome comprises trisomy 21, mosaicism, or a translocation. In some embodiments, the muscular dystrophy comprises Duchenne muscular dystrophy, Becker muscular dystrophy, congenital muscular dystrophy, myotonic dystrophy, facioscapulohumeral muscular dystrophy, oculopharyngeal muscular dystrophy, distal muscular dystrophy, or Emery-Dreifuss muscular dystrophy.

[0054] In some embodiments, the software module is configured to determine the likely presence or absence of one or more nervous system dysfunctions including amyotrophic lateral sclerosis, arteriovenous malformation, cerebral aneurysm, brain tumor, dural arteriovenous fistula, epilepsy, headache, memory impairment, multiple sclerosis, Parkinson's disease, peripheral neuropathy, post-herpetic neuralgia, spinal cord tumor, stroke, Alzheimer's disease, corticobasal degeneration, Creutzfeldt-Jakob disease, frontotemporal dementia, dementia with Lewy bodies, mild cognitive impairment, progressive supranuclear palsy, or vascular dementia.

[0055] In some embodiments, the behavioral disorder, developmental delay, or nervous system dysfunction comprises pervasive developmental disorder (PDD), autism spectrum disorder (ASD), social communication disorder, repetitive restriction of behaviors, interests, and activities (RRB), autism (classic autism), Asperger's syndrome (high functioning autism), pervasive developmental disorder not otherwise specified (PDD-NOS, "atypical autism"), attention deficit disorder (ADD), attention deficit hyperactivity disorder (ADHD), speech and language delay, obsessive-compulsive disorder (OCD), depression, schizophrenia, Alzheimer's disease, dementia, intellectual disability, or learning disability. In some embodiments, the behavioral disorder, developmental delay, or nervous system dysfunction is autism spectrum disorder or autism.

[0056] FIG. 1 illustrates a pictorial representation from which behavioral units are extracted. An interrogator or caregiver (101) interacts with an individual under examination (102). The interrogator or caregiver can ask the individual to interact with an object (103) or to accomplish a task. Alternatively, the interrogator or caregiver can ask for a response from the subject. The movements or sounds of the individual are registered as behavioral units, as is a software module capable of detecting and clinically relevant to the possible presence or absence of behavioral disorders, developmental delays, or neurological dysfunction. For example, the child's eye movements, the child's verbal communication, and the child's facial and hand movements can be behavioral units. In some cases, the child provides a response through an automated assessment and / or treatment process. For example, the child may be provided with a digital device with an interactive module (e.g., storytime) that provides an interactive digital experience to elicit and / or induce a response or feedback (e.g., video, audio, or other user input via buttons / touch screen) from the child. This can be analyzed to identify behavioral units that are used to determine higher order behaviors and / or assess the child's status or progress (e.g., level of speech or language development in relation to expected development at a given age).

[0057] Higher order behavior In some embodiments, the higher order behaviors are determined by compiling a plurality of behavioral units. In some embodiments, the higher order behaviors identified by the methods, devices, systems, software, and platforms described herein are output that are provided and function as at least a portion of the assessment generated for the individual being assessed. In some embodiments, identifying higher order behaviors from a collection of observed behavioral units provides a way to summarize, contextualize, and / or better assess individual behavioral units.

[0058] In some embodiments, the higher order behavior includes verbal communication, non-verbal communication, lack of communication, display of thoughts or emotions, control of movement, direction of gaze or attention, or direction of verbal communication. For example, the video of the individual may include behavioral units of smiling, laughing, and clapping. The displayed higher order behavior may be happiness or joy. In some embodiments, the verbal communication may include signs or symptoms of speech or language impairment, delay, or disorder. Examples include impairments or deficiencies in speech sound articulation, fluency, voice, language comprehension and decoding, and language production and use. In some embodiments, the higher order behavior including verbal communication of the individual is analyzed using an algorithm (e.g., a machine learning algorithm) to determine whether the individual has one or more behavioral disorders, developmental delays, and nervous system dysfunctions, or signs or symptoms thereof, such as speech or language deficits or delays associated with the individual's expected developmental level (e.g., based on age). A determination of an individual's actual developmental level (e.g., assessed using speech and / or language metrics) can be compared to the expected developmental level to determine whether the individual has one or more behavioral disorders, developmental delays, and neurological dysfunction, e.g., speech and / or language delays.

[0059] In some embodiments, linguistic communication includes recognizable words and phrases, nearly recognizable words and phrases, incoherent speech, rambling, laughter, grunts, crying, or any audible communication that conveys emotions, thoughts, intentions, or requests. In some embodiments, behavioral units responsive to speech are analyzed to determine metrics of speech and / or language development. For example, these metrics may include speech metrics such as articulation, fluency, and voice of speech sounds, and linguistic metrics such as receptive skills (e.g., the ability to understand and decode language) and expressive skills (e.g., the ability to produce and use language). As an illustrative example, speech may be parsed to identify specific speech sounds (e.g., individual phonemes or sets of phonemes that make up words) that constitute linguistic input or feedback from the subject, and the parsed speech sounds may be compared to baseline or reference standard speech sounds to determine a score. As another illustrative example, sets of words may be evaluated for acoustic features such as the presence of pauses.

[0060] In some embodiments, non-verbal communication includes directing the attention of others, touching a person or object, waving, clapping, grabbing and directing the hand or body of another, making noises with an object, showing emotion or thought through facial expressions or body movements, or eye contact, hi some embodiments, non-verbal communication includes facial expressions, posture, gestures, eye contact, or touch.

[0061] In some embodiments, the display of thoughts or emotions is through the use of body language and facial expressions. In some embodiments, emotions include anger, anticipation, anxiety, indifference, annoyance, boredom, unhappiness, calm, ease, cheerfulness, nervousness, comfort, sympathy, concentration, confidence, satisfaction, curiosity, joy, desire, despair, disappointment, determination, dissatisfaction, dislike, dejection, apprehension, confusion, jealousy, excitement, fear, frustration, jubilation, gratitude, distress, guilt, happiness, hate, hope, impatience, irritation, envy, enjoyment, loneliness, love, overwhelm, panic, pain, delusion, pride, rage, regret, relief, unwillingness, remorse, indignation, sadness, contentment, self-pity, shame, shock, self-conceit, doubt, doubt, or worry.

[0062] In some embodiments, the higher order behavior includes movement control. In some embodiments, movement control is determined by analyzing the individual's movements and providing a probability that the individual's movements are abnormal within the context of the video. In some embodiments, movement control is determined by the presence or absence of behavioral units related to movement control. In some embodiments, movement control includes contracting movements, stumbling movements, stumbling movements, hitting objects or people with limbs or body, dropping objects, loss of balance, surprised facial expressions, frustrated facial expressions, angry facial expressions, concentrated facial expressions, painful facial expressions, embarrassed facial expressions, crying, screaming, or verbal communication related to lack of movement control. In some embodiments, the individual's age, disability, or injury is taken into account in determining the presence or absence of movement control. In some embodiments, the individual's movement control is determined as a percentage, for example, the individual may be in a state where about 100%, 99%, 95%, 80%, 75%, 50%, or 25% of their movements are controlled.

[0063] In some embodiments, the gaze or direction of gaze includes evaluating the direction and length of gaze of the subject. In some embodiments, the direction and length of gaze is relative to other movements, objects with which the individual is interacting, or people with whom the individual is interacting. For example, a video of an adult posing a question to an individual and the individual responding is evaluated. The direction of the individual's gaze is evaluated to determine whether and for how long the individual maintains eye contact with the questioner or caregiver. If an additional stimulus is introduced, for example, another person or animal approaches the individual, and the individual directs their gaze toward the additional stimulus, the analysis takes these into account. In some embodiments, the ability of the individual to maintain eye contact with the questioner or caregiver is a higher order behavior.

[0064] In some embodiments, the direction of the verbal communication includes evaluating the direction of the subject's verbal communication. In some embodiments, the direction and context of the verbal communication is relative to other movements, objects the individual is interacting with, or people the individual is interacting with. For example, the amount of verbal communication of an individual may fluctuate due to the individual directing their voice in multiple directions. If the intended reception of communication is static, the individual may not face the questioner or caregiver while responding. The individual may be wandering or walking or turning away while answering a question. In some embodiments, maintaining a particular direction while engaging in verbal communication is a higher order behavior.

[0065] In some embodiments, machine learning algorithms are used to identify higher order behaviors.

[0066] Behavioral patterns One or more higher order behaviors and / or behavioral units observed over time can together constitute one or more behavioral patterns. In some embodiments, the behavioral patterns identified by the methods, devices, systems, software, and platforms described herein are output provided and function as at least a portion of the assessment generated for the individual being assessed. In some embodiments, identifying behavioral patterns from a collection of higher order behaviors and / or behavioral units provides a way to summarize, contextualize, and / or better assess individual higher order behaviors and behavioral units.

[0067] In some embodiments, the behavioral patterns include behavioral units or higher order behaviors that are mapped over time. In some embodiments, the behavioral patterns include two or more higher order behaviors that are related to each other. For example, in response to a question, an individual responds verbally while moving their eyes continuously and walking around the room. The individual's response initially addresses the question, but the response becomes smaller and smaller in volume. Higher order behaviors are identified as avoiding eye contact when responding, avoiding turning to face the questioner or caregiver when responding, and failing to answer the question. The identified behavioral patterns may indicate that the individual is unable to concentrate sufficiently for verbal communication long enough to answer the question. The behavioral patterns or higher order behaviors may be associated with both ADHD and a form of autism. However, due to differences in the specific behavioral units of individuals, the machine learning algorithm described herein separates each condition and provides a probability score for each diagnosis.

[0068] In some embodiments, behavioral patterns include the ability to respond to stimuli, the ability to perform assigned tasks, appropriate speech responses, appropriate movement, the ability to concentrate, or the ability to accurately perceive everything.

[0069] In some embodiments, machine learning algorithms are used to identify behavioral patterns.

[0070] FIG. 2 is a diagram illustrating a representation of how behavioral units (205) and higher order behaviors (210) are mapped (215) over time to generate behavioral patterns (220). A portable video and audio capture device (201) records video of an individual. A trained machine learning algorithm recognizes each behavioral unit and maps (215) their presence over time, exemplified as speaking, smiling, pointing, shrugging, standing, laughing, sitting, and eye contact. Higher order behaviors are constructed from behavioral units (210), exemplified as vocabulary size, voice intonation, conversational intimacy, quality of eye contact, facial expression, and pointing at objects. The machine learning algorithm utilizes the time-mapped behavioral units and higher order behaviors to generate behavioral patterns (220), exemplified as levels of immersion, attention span, and enjoyment from sharing.

[0071] Methods for assessing behavioral disorders, developmental delays, and neurological dysfunction In some embodiments, the individual is assessed by recording video and / or audio data of the individual, and the assessment of the video and / or audio data is performed by machine learning algorithms described herein. In some embodiments, the individual is recorded answering questions asked by an interrogator or caregiver. In some embodiments, the individual remains seated while responding to the interrogator or caregiver. In some embodiments, the individual is allowed to respond to the interrogator or caregiver while sitting or moving freely around the room. In some embodiments, the individual remains within the frame of the recording device. In some embodiments, the individual remains within audible distance of the recording device. In some embodiments, the audible distance is within 25, 10, 5, or 3 meters of the recording device. In some embodiments, the individual is assessed by responding to questions using verbal communication. In some embodiments, the individual is assessed by responding to questions using non-verbal communication. For example, the interrogator can ask the individual to make a facial expression, such as a smile, or to express an emotion, such as excitement. Responses to such questions can assess whether the individual is able to perceive the behavior of others and appropriately replicate this behavior. In some embodiments, the interrogator interacts with the individual to elicit a desired response. For example, the interrogator may feed the individual a favorite food or compliment the individual. The interrogator may also pose questions to make the individual feel uncomfortable or emotional. In some embodiments, there is no interrogator and the individual is recorded without prompting or prompting from a third party. In some embodiments, recording is done with guidance to the recording device. For example, guidance may be provided to the recording device to record the individual's sleep, play, eating, communication, or the arousal of certain emotions.

[0072] In some embodiments, the video and / or audio recording is performed by a mobile device. In some embodiments, the mobile device is a smartphone, tablet, smartwatch, or device with mobile camera or recording capabilities. In some embodiments, the video and / or audio recording is performed by a fixed camera and / or microphone. For example, an individual may be asked questions in a clinician's office and their responses recorded by a camera on a tripod with a microphone attached. In some embodiments, the camera is a high-definition camera.

[0073] In some embodiments, the methods disclosed herein are used to aid in the diagnosis of behavioral disorders, developmental delays, or nervous system dysfunction.

[0074] In some embodiments, the methods disclosed herein are used to aid in the diagnosis of behavioral disorders. In some embodiments, the behavioral disorder comprises attention deficit hyperactivity disorder (ADHD), oppositional defiant disorder (ODD), autism spectrum disorder (ASD), anxiety disorder, depression, bipolar disorder, learning disability, or conduct disorder. In some embodiments, the attention deficit hyperactivity disorder comprises predominantly inattentive ADHD, predominantly hyperactive-impulsive ADHD, or combined hyperactive-impulsive and inattentive ADHD. In some embodiments, the autism spectrum disorder comprises autistic disorder (classic autism), Asperger syndrome, pervasive developmental disorder (atypical autism), or childhood disintegrative disorder. In some embodiments, the anxiety disorder comprises panic disorder, phobia, social anxiety disorder, obsessive-compulsive disorder, separation anxiety disorder, illness anxiety disorder (hypochondriasis), or post-traumatic stress disorder. In some embodiments, the depression comprises major depression, persistent depressive disorder, bipolar disorder, seasonal affective disorder, psychotic depression, postpartum depression, premenstrual dysphoric disorder, "situational" depression, or atypical depression. In some embodiments, the bipolar disorder comprises bipolar disorder I, bipolar disorder II, cyclothymic disorder, or bipolar disorder due to another medical or substance abuse disorder. In some embodiments, the learning disorder comprises dyslexia, dyscalculia, dysgraphia, dyspraxia (sensory integration disorder), dysphagia / aphasia, auditory processing disorder, or visual processing disorder. In some embodiments, the behavioral disorder is a disorder defined in the Diagnostic and Statistical Manual of Mental Disorders (DSM), full edition.

[0075] In some embodiments, the methods disclosed herein are used to aid in the diagnosis of developmental delay. In some embodiments, the developmental delay comprises autism spectrum disorder, mental retardation, cerebral palsy, Down syndrome, failure to thrive, muscular dystrophy, hydrocephalus, developmental coordination disorder, cystic fibrosis, fetal alcohol syndrome, homocystinuria, tuberous sclerosis, abetalipoproteinemia, phenylketonuria, Aase syndrome, speech delay, gross motor delay, fine motor delay, social delay, emotional delay, behavioral delay, or cognitive delay. In some embodiments, the mental retardation comprises adrenoleukodystrophy, Ito syndrome, acroosteoporosis, Huntington's disease, Arskog syndrome, Aicardi syndrome, or Tay-Sachs disease. In some embodiments, the cerebral palsy comprises spastic cerebral palsy, dyskinetic cerebral palsy, hypotonic cerebral palsy, ataxic cerebral palsy, or mixed cerebral palsy. In some embodiments, the autism spectrum disorder comprises autistic disorder (classic autism), Asperger syndrome, pervasive developmental disorder (atypical autism), or childhood disintegrative disorder. In some embodiments, the Down syndrome comprises trisomy 21, mosaicism, or a translocation. In some embodiments, the muscular dystrophy comprises Duchenne muscular dystrophy, Becker muscular dystrophy, congenital muscular dystrophy, myotonic dystrophy, facioscapulohumeral muscular dystrophy, oculopharyngeal muscular dystrophy, distal muscular dystrophy, or Emery-Dreifuss muscular dystrophy.

[0076] In some embodiments, the methods disclosed herein are used to aid in the diagnosis of nervous system dysfunction, including amyotrophic lateral sclerosis, arteriovenous malformation, cerebral aneurysm, brain tumor, dural arteriovenous fistula, epilepsy, headache, memory disorder, multiple sclerosis, Parkinson's disease, peripheral neuropathy, postherpetic neuralgia, spinal tumor, stroke, Alzheimer's disease, corticobasal degeneration, Creutzfeldt-Jakob disease, frontotemporal dementia, dementia with Lewy bodies, mild cognitive impairment, progressive supranuclear palsy, or vascular dementia.

[0077] In some embodiments, the methods disclosed herein are used to aid in the diagnosis of behavioral disorders, developmental delays, or nervous system dysfunction. In some embodiments, the methods disclosed herein are used in conjunction with known techniques for diagnosing behavioral disorders, developmental delays, or nervous system dysfunction. In some embodiments, the methods disclosed herein are used in conjunction with the provision of questionnaires or video observations by a clinician. In some embodiments, the methods disclosed herein improve the accuracy of known diagnostic methods or reduce the time or retries required for an accurate diagnosis.

[0078] In some embodiments, the methods disclosed herein are used to monitor the progression or regression of behavioral disorders, developmental delays, and nervous system dysfunction. In some embodiments, the individual has been diagnosed with a behavioral disorder, developmental delay, or nervous system dysfunction and is receiving treatment. In some embodiments, the methods disclosed herein are used to evaluate the effectiveness of the treatment. In some embodiments, monitoring or evaluating the individual's condition includes analyzing the number and / or frequency of related behavioral units, higher order behaviors, and behavioral patterns detected over the course of multiple analysis sessions. In some embodiments, the machine learning module disclosed herein is used to analyze the recordings of each analysis session and provide a progress value. In some embodiments, the progress value is derived by comparing the probability scores (see below) of each analysis session over time. For example, a child receiving treatment for autism may be evaluated for autism over the course of three analysis sessions. Each evaluation by the machine learning module generates a probability score that is used to indicate the likelihood that the individual has autism. The probability score decreases over the three sessions. These results confirmed that the prescribed treatment was effective and the initial diagnosis was correct. The machine learning algorithm provides an indication that the treatment is being continued. In some embodiments, the progress value is obtained from a machine learning module trained on the progress data. In some embodiments, the progress data is a record of an analysis session of an individual patient in which behavioral disorders, developmental delays, and nervous system dysfunctions have regressed or progressed as determined by a clinician. The clinician's determination is included in the progress data. For example, the machine learning module is trained on a series of records in which an individual has received treatment for ADHD, and the training data includes clinical determinations regarding whether the disease is progressing or regressing. The machine learning module analyzes a series of records of an individual diagnosed with ADHD to determine whether a given treatment regimen is effective. The machine learning algorithm determines that the disease is progressing. The machine learning algorithm recommends that the treatment for the individual be changed and / or that the ADHD diagnosis be reevaluated.

[0079] In some embodiments, behavioral patterns are monitored over time, for example, individuals with ADHD may be assessed for energy levels, sociability, and vocabulary over multiple years.

[0080] Interactive modules for assessing and treating behavioral disorders, developmental delays, and neurological dysfunction In some embodiments, the systems and methods disclosed herein include one or more interactive modules for assessing and / or treating behavioral disorders, developmental delays, and nervous system dysfunction. The interactive modules can provide an individual with one or more activities for assessing or evaluating and / or treating behavioral disorders, developmental delays, or nervous system dysfunction, or symptoms or cognitive functions associated therewith. The activities can be interactive activities in which stimuli and corresponding prompts, such as a portion of a story and a question associated therewith, are presented to the individual. For example, FIG. 5A illustrates an example of a GUI display in which a virtual character provides narrative guidance. A story sequence then begins (see FIG. 5B). Once the story sequence has ended (an example of a story sequence shown in FIG. 6B), the virtual character can prompt the individual or user to answer a question (see FIG. 5C) or provide instructions (see FIG. 6A). Feedback or responses by the individual prompted by the activities can be detected and stored, for example, as audio and / or video data detected by a microphone and / or camera of the computing device. The detection of the individual's audio and / or video information, the processing of the information to identify behavioral units, and the analysis of the behavioral units to determine higher-level actions and / or behavioral patterns can be performed as described throughout this specification. Another illustrative example of an interactive activity is an emotion guessing game in which a user is prompted to guess the emotion of one or more characters in an image by providing a verbal response that can be recorded as audio information. Another illustrative example of an interactive activity is a search game in which a user is prompted to point a camera at various objects and describe them. Thus, the platform, device, and method of the present invention can incorporate a variety of interactive activities.

[0081] In some embodiments, the interactive module is configured to dynamically prompt the user to adjust the context until it is appropriate for evaluation, for example, the interactive module automatically prompts the user to move their face until they are visible to the camera, to speak louder if they cannot be heard, or to turn on the lights if the video / camera image is too dark.

[0082] In addition to the examples disclosed herein, various activities can be used to provide digital therapy and / or assessment / evaluation of an individual. For example, a non-limiting example activity is to display photos and prompt the individual to come up with a story or narrative that describes the photos (see Figures 7A-7B). Another example is to provide a series of photos or images and prompt the individual to arrange them in a sequence or appropriate order (see Figure 8). Yet another example is to provide a GUI tool that allows the individual to create their own narrative using graphic and / or audio elements (see Figure 9). Yet another example is to provide an embedded intervention via social stories (see Figure 10) or game-based life skills training (see Figure 11).

[0083] A notable advantage of the interactive modules disclosed herein is that they can facilitate a dynamic process in which an individual is treated and reevaluated or assessed over the course of an activity. This allows the interactive module to be adjusted or modified in real time in response to input, feedback, or other collected user information. The interactive module can be configured to modify the difficulty of one or more activities based on ongoing evaluation of the user, for example, increasing the difficulty level if speech and / or language proficiency metrics (e.g., measures of speech articulation and fluency) indicate a child's proficiency or decreasing the difficulty level if the child's proficiency falls below a minimum threshold. For example, if the difficulty level of a current activity is determined to be too high (e.g., a child has problems articulating answers to questions or verbal responses, resulting in an error rate of more than 50% in speech articulation), the difficulty level can be adjusted downward during the same session while the individual is still engaged in the activity. Alternatively, or in combination, the activity is modified or adjusted after the end of a session to incorporate the changes, such as difficulty level, into the next session. This process can be repeated (e.g., periodically or continuously) between activities or during the same activity. In some embodiments, the interactive module modifies or adjusts the activity over time based on an evaluation of user information collected during the activity (i.e., information possibly collected from follow-up testing or assessment after completion of the activity). Modification of the activity may be based on an individual's progress. For example, an individual may successfully meet a target metric for speech and / or language proficiency (e.g., a success rate threshold for vocalization of sounds, etc.), which may result in the activity or a next activity being adjusted to increase the difficulty or challenge for the individual user by adjusting the difficulty for the individual user.

[0084] Speech assessments performed using the interactive module may include a variety of conditions. Non-limiting examples of conditions that can be assessed for speech assessment include apraxia dysarthria, and articulation for sound. Non-limiting examples of conditions that can be assessed for speech assessment include phonology and phonemes for phonetics. Non-limiting examples of conditions that can be assessed for speech assessment include stuttering, extensions, repetitions, interruptions, and speed for fluency. Non-limiting examples of conditions that can be assessed for speech assessment include structural, neurological, and functional for voice.

[0085] In some embodiments, the interactive module is configured to assess and / or treat behavioral disorders, developmental delays, or nervous system dysfunction. For example, autism spectrum disorder has core characteristics such as communication and social pragmatic deficits. These characteristics include reduced frequency of narration in conversation, reduced coherence, difficulty referring to mental states in stories, and difficulty using causal language. Attention deficit hyperactivity disorder is frequently characterized by language dysfunction, including, for example, short speech, poorly organized speech, many sequence errors, and misunderstandings and word substitutions. Thus, the interactive module provides activities that act as digital therapy that can help individuals improve these characteristics.

[0086] The systems and methods disclosed herein provide an improvement over traditional assessments including SSR, which includes parent interviews, observations of reading sessions, and video recordings of individuals, and clinical assessments, which include parent interviews, standardized tests (Wh-questions, vocabulary, picture books), and informal probes. These traditional assessments are time-consuming and costly. In contrast, systems and methods are disclosed herein that provide automated activities, such as automated storytelling activities or modes, in which the individual is guided through a story while feedback or other data is assessed to assess various speech and / or language metrics, including, for example, story comprehension, story recall or retelling, and picture-associated stories. In some cases, parent questionnaires may be provided to obtain additional relevant input data.

[0087] In some embodiments, the interactive module provides the activity and receives user input or responses to the activity. For example, the user input or responses may include answers to questions or prompts, selection of available options or modifications to the activity, audio / video information obtained by microphone / camera recording of the individual during the activity, and other forms of information that may be collected during the activity. The input or response may be collected dynamically, e.g., the user is prompted by the module to provide an input or response, which may include a written or typed response via a graphical user interface of the user device, or via speech (optionally including video) detected by the microphone and / or camera of the user device. Alternatively, the input or response may be collected passively, e.g., audio and video information is collected without the need for active direction or prompting of the individual. Examples of collected data include spoken words, facial expressions, or other data that may be used to determine behavioral units and / or patterns that are automatically collected by the microphone and / or camera. In some embodiments, the activity is dynamic depending on the user's engagement or input / feedback. As an example, an activity is a storytelling mode that actively asks questions and receives user input while guiding an individual or user through a story. This activity can involve active participation by an individual, for example, an individual can take on the role or aspect of a character in a story in which words or sounds uttered by the individual are recorded.

[0088] In some embodiments, the software module is configured to evaluate user input, responses, feedback, or other collected user data during the course of an activity. In some embodiments, the data is analyzed to determine or evaluate behavioral disorders, developmental delays, cognitive impairments, or some related signs, symptoms, or cognitive functions. In some embodiments, behavioral units and / or higher order actions or behavioral patterns are identified or evaluated. The behavioral units may include speech sounds, words, or phrases that are identified as utterances (or attempts at speech). In some embodiments, the behavioral units include speech elements such as speech sounds or phonemes that are analyzed to identify words or phrases that make up the utterance.

[0089] In some embodiments, the systems and methods disclosed herein include an interaction module configured to collect data (e.g., audio of a user's speech) and a diagnostic or assessment module configured to analyze the data to detect a speech or language delay or disorder, or a sign or symptom thereof. In some embodiments, the assessment module is configured to determine a severity level of one or more developmental delays, or a symptom or cognitive function associated with one or more developmental delays (e.g., speech or language delays). Non-limiting examples include levels of speech and / or language, emotion recognition, non-verbal communication (e.g., degree of voice development deficit), reading comprehension, word acquisition, and other signs, symptoms, or cognitive functions. The severity level may be a score, a category (e.g., low, moderate, or high severity of the impairment or deficit), or other metric suitable for assessing these signs, symptoms, or cognitive functions. In some embodiments, the speech disorder or delay includes a specific symptom or cognitive metric. For example, speech disorders may encompass articulation of speech sounds (e.g., how well a child can articulate certain speech sounds), fluency (e.g., stuttering indicates a lack of fluency), and voice. Articulation errors include substitution errors, where a sound or phoneme that a child can produce is replaced with a sound or phoneme that the child cannot yet produce, omission errors, where a sound that is too difficult to produce is dropped (e.g., the consonant "r" is dropped from "red"), distortion errors, where a sound or phoneme is used but not articulated correctly, and addition errors, where an extra sound or phoneme is added.

[0090] In some embodiments, the systems and methods disclosed herein provide interactive modules that include interactive activities. The interactive activities can be individualized or customized based on target demographics (e.g., the age of the individual). For example, the activities may have difficulty levels that are adjusted according to the age of the individual (e.g., the difficulty or complexity of the activities can be categorized into various ages, such as 3, 4, 5, 6, 7, or 8 years old). In some embodiments, the input or response of the individual is obtained, for example, a voice clip (audio data) collected from the individual.

[0091] In some embodiments, the interactive module provides the individual or user with the results of the assessment. The results of the assessment may include assessment or performance metrics or scores corresponding to fine motor, cognitive, gross motor, speech and language, social and emotional, and behavioral categories (FIG. 11A). Categories are selectable within the interactive module's graphical user interface to reveal additional subcategories, such as speech, fluency, voice, and language for the speech and language category (FIG. 11B). An explanation of the individual's assessment results can be provided (see FIG. 11C).

[0092] Speech and language analysis In some embodiments of the methods, devices, systems, software, and platforms described herein, a software module is utilized to analyze an individual's audio information to identify speech and / or language related behavioral units, higher order actions, and / or behavioral patterns. In some embodiments, the software module utilizes one or more machine learning algorithms or models to engage in automated speech recognition and speech / language analysis using the audio data obtained for the individual.

[0093] In some embodiments, the audio data is acquired using interactive activities, such as those provided by an interactive module as described herein. The interactive module can be provided through a computing device, such as a smartphone or tablet computer. The interactive activities may include story and picture description tasks that elicit speech and language production from the child, with minimal assistance from the parents. The speech and language data may then be analyzed by a software module, for example, for feature extraction and automated diagnostic assessment of speech and / or language proficiency. The audio data may be processed to identify speech elements. For example, audio clips may be processed to identify phonemes, which represent distinct sound units in a particular language (e.g., English).

[0094] In some embodiments, video or camera data is also acquired and used alone or in combination with audio data for feature extraction and automated diagnostic assessment. The video or camera data can be used to detect facial or other features via the built-in camera of a smartphone or tablet.

[0095] In some embodiments, the audio data is processed to identify sounds responsive to speech. In some embodiments, the audio data is processed or analyzed to identify speech sounds, words, or phrases. The uttered sounds, words, or phrases can be analyzed to determine one or more performance metrics or parameters related to speech and / or language. In some embodiments, the software module identifies acoustic features related to speech and / or language. For example, the acoustic features may include features related to pauses in speech, such as the number of short and / or long pauses, the average length of the pauses, the variability of their length, and other similar statistics related to uninterrupted speech. In some embodiments, the software module engages in automatic speech analysis, prosodic analysis, and morphosyntactic analysis. For example, the analysis may include automatic measurements of dialogue structure, differential syntactic analysis based on speech recognition, and automatic measurements of emotional valence and emotional arousal in the speech. In some embodiments, the software module determines a semantic similarity measure, e.g., a word overlap measure that corresponds to a simple word overlap measure between a pair of stories defined as the magnitude of word crossover in the stories. Such measurements may be useful, for example, to determine how closely an individual's retelling of a story during an interactive activity compares to the original story, thereby providing a metric regarding proficiency in story retelling.

[0096] The speech and / or language data is analyzed using the algorithms or models disclosed herein to determine the individual's speech and language proficiency. Speech proficiency can be measured in terms of articulation (e.g., production of age-appropriate speech sounds), fluency (e.g., production of speech with age-appropriate continuity, measurable using the number and duration of unnecessary pauses), and voice (e.g., production of normal voice quality). Language proficiency can be measured in terms of receptive and expressive language (e.g., whether a child can understand spoken language and produce communicative speech), and narrative ability (e.g., whether a child can use language to understand, retell, and create a coherent story).

[0097] Machine Learning Software Module In some embodiments of the methods, devices, systems, software, and platforms described herein, a machine learning software module is utilized to identify one or more behavioral units, and / or higher order behaviors, and / or behavioral patterns, and / or indicators of the likelihood of whether a particular disease is present in the individual being evaluated. In some embodiments, the machine learning software module includes a machine learning model (or data model). It should be understood that machine learning encompasses many data architectures and configurations, and the teachings herein are not limited to one type of machine learning.

[0098] The machine learning software modules described herein are typically trained using video and / or audio datasets. In some embodiments, the video and / or audio datasets include previously identified behavioral units. In some embodiments, the video and / or audio datasets include previously identified behavioral units and a diagnosis of a behavioral disorder, developmental delay, or nervous system dysfunction. In some embodiments, the video and / or audio datasets include previously identified higher order behaviors. In some embodiments, the video and / or audio datasets include previously identified higher order behaviors and a diagnosis of a behavioral disorder, developmental delay, or nervous system dysfunction. In some embodiments, the video and / or audio datasets include previously identified behavioral patterns and a diagnosis of a behavioral disorder, developmental delay, or nervous system dysfunction. In some embodiments, the video and / or audio datasets include previously identified behavioral units and previously identified higher order behaviors. In some embodiments, the video and / or audio datasets include previously identified behavioral units, previously identified higher order behaviors, and a diagnosis of a behavioral disorder, developmental delay, or nervous system dysfunction. In some embodiments, the video and / or audio dataset includes previously identified higher order behaviors and previously identified behavioral patterns. In some embodiments, the video and / or audio dataset includes previously identified higher order behaviors, previously identified behavioral patterns, and a diagnosis of a behavioral disorder, developmental delay, or nervous system dysfunction. In some embodiments, the video and / or audio dataset includes previously identified behavioral units and previously identified behavioral patterns. In some embodiments, the video and / or audio dataset includes previously identified behavioral units, previously identified behavioral patterns, and a diagnosis of a behavioral disorder, developmental delay, or nervous system dysfunction. In some embodiments, the video and / or audio dataset includes previously identified behavioral units, previously identified higher order behaviors, and previously identified behavioral patterns.In some embodiments, the video and / or audio dataset includes previously identified behavioral units, previously identified higher order behaviors, previously identified behavioral patterns, and diagnoses of behavioral disorders, developmental delays, or neurological dysfunction.

[0099] In some embodiments, the trained machine learning software module analyzes new video and / or audio data not previously associated with a behavioral unit, higher order behavior, behavioral pattern, or with a particular behavioral disorder, developmental delay, or nervous system dysfunction. In some embodiments, the machine learning software module identifies one or more behavioral units, higher order behaviors, or behavioral patterns present in the new video and / or audio data. In some embodiments, the machine learning software module identifies one or more behavioral units present in the new video and / or audio data. In some embodiments, the machine learning software module identifies one or more higher order behaviors present in the new video and / or audio data. In some embodiments, the machine learning software module identifies one or more behavioral patterns present in the new video and / or audio data. In some embodiments, the machine learning algorithm analyzes the new video and / or audio data and provides a likelihood that one or more behavioral disorders, developmental delays, or nervous system dysfunctions are present in the individual being evaluated.

[0100] In some embodiments, one or more machine learning software modules are utilized to identify behavioral units, identify higher order behaviors, identify behavioral patterns, and provide a diagnosis of behavioral disorders, developmental delays, or nervous system dysfunction. In some embodiments, the machine learning software module for identifying behavioral units is a feature detection algorithm. In some embodiments, the machine learning software module for identifying higher order behaviors is a detection algorithm of higher order behaviors. In some embodiments, the machine learning software module for identifying behavioral patterns is a behavioral pattern detection algorithm. In some embodiments, the machine learning software module for diagnosing behavioral disorders, developmental delays, or nervous system dysfunctions is a behavioral assessment algorithm.

[0101] In some embodiments, the machine learning software module is a supervised learning algorithm, hi some embodiments, the machine learning software module is selected from nearest neighbor, naive Bayes, decision tree, linear regression, support vector machine, or neural network.

[0102] In some embodiments, the machine learning software module provides a probabilistic diagnosis without considering contextual indicators. In some embodiments, the machine learning software module provides a probabilistic diagnosis through analysis of video or audio data alone. In some embodiments, the machine learning software module provides a probabilistic diagnosis through analysis of video or audio data and contextual indicators. For example, a user provides video and audio data of a 5-year-old boy responding to questions designed to evaluate the boy for autism. In one embodiment, the machine learning software module analyzes the video and audio data, including data input by the user related to contextual indicators such as the child's age, gender, suspected diagnosis, environment, and predictions of verbal responses to the questions. In another embodiment, the machine learning software module analyzes the video and audio data without the contextual indicators illustrated above.

[0103] In some embodiments, the methods disclosed herein utilize a probability score to recommend a diagnosis. For example, a machine learning software module analyzes a data set, determines that a number of indicators exist, and assigns various scores that reflect the degree to which the behavioral units, higher order behaviors, or behavioral patterns in the data set overlap with those identified in the data set used to train the machine learning software module for each indicator. In some embodiments, the probability score is a reflection of how well the data fits each modeled indicator. In some embodiments, one or more behavioral disorders, developmental delays, or nervous system dysfunctions are given a probability score. In some embodiments, the probability score is termed "prediction".

[0104] In some embodiments, a probability score threshold can be used in conjunction with the probability score to determine whether a diagnosis is recommended, for example, if the probability score is too low, no diagnosis is recommended for the indication.

[0105] In some embodiments, the probability threshold is used to adjust the sensitivity of the trained machine learning software module. For example, the probability threshold may be 1%, 2%, 5%, 10%, 15%, 20%, 25%, 30%, 35%, 40%, 45%, 50%, 55%, 60%, 65%, 70%, 75%, 80%, 85%, 90%, 95%, 98%, or 99%. In some embodiments, if the accuracy, sensitivity, or specificity is below a predetermined adjustment threshold, the probability threshold is adjusted. In some embodiments, the adjustment threshold is used to determine the parameters of the training period. For example, if the accuracy of the probability threshold is below the adjustment threshold, the system extends the training period and / or requires additional voice recordings and / or identity data. In some embodiments, the additional voice recordings and / or identity data can be included into the training data. In some embodiments, the additional recordings and / or identity data can be used to refine the training data set.

[0106] FIG. 3 is a schematic diagram of an exemplary data processing module (300) for providing the machine learning algorithms and methods described herein. The data processing module (300) generally includes a pre-processing module (305), a training module (310), and a prediction module (320). The data processing module can extract training data (350) from a database or obtain new data (355) using a user interface (330). The pre-processing module can apply one or more transformations to standardize the training data or new data for the training module or the prediction module. The pre-processed training data can be passed to the training module, which can build a trained machine learning algorithm (360) based on the training data. The training module includes a validation module (315) configured to validate the trained machine learning algorithm using any suitable validation algorithm (e.g., stratified k-fold cross-validation). The pre-processed new data can be passed to a prediction module, which can match the new data to a machine learning algorithm built using the training module to output a prediction (370) of whether a particular condition (e.g., behavioral disorder, developmental delay, or nervous system dysfunction) is present in the individual.

[0107] The training data (350) used by the training module to build the machine learning model may include multiple data sets from multiple subjects, with each subject's data set including a set of behavioral units, higher order behaviors, or behavioral patterns, and a classification of the subject's behavioral disorder, developmental delay, or nervous system dysfunction. The behavioral units may include movements or sounds of the subject that are machine detectable and clinically relevant to the possible presence or absence of a behavioral disorder, developmental delay, or nervous system dysfunction. Each of the behavioral units, higher order behaviors, or behavioral patterns may be associated with the identification of one or more developmental disorders or diseases, and each of the corresponding behavioral units, higher order behaviors, or behavioral patterns may indicate the presence of the disorder in a particular subject. For example, the behavioral pattern may be the subject's ability to engage in imaginative or imaginary play, and the value for the particular subject may be a score corresponding to the presence of the behavioral pattern in the subject. The behavioral pattern may be observed in the subject, for example, using a video of the subject engaged in a certain behavior, and the behavioral pattern identified by analyzing the video recording with a machine learning algorithm. In addition, each subject's data set in the training data also includes a classification that the subject has a behavioral disorder, developmental delay, or nervous system dysfunction. For example, the classification may be autism, autism spectrum disorder (ASD), or non-spectrum. Preferably, the classification includes a clinical diagnosis assigned by a qualified individual, such as a licensed clinical psychologist, to improve the predictive accuracy of the machine learning algorithm. The training data may include datasets available from large data repositories, such as the Autism Diagnostic Interview-Revised (ADI-R) data and / or the Autism Diagnostic Observation Schedule (ADOS) data available from the Autism Genetic Resource Exchange (AGRE), or other suitable data repositories (e.g., Boston Autism Consortium (AC), Simons Foundation, National Autism Research Database, etc.).Alternatively, or in combination, the training data may include large self-reported datasets, which can be crowdsourced from users (e.g., via a website, mobile application, etc.).

[0108] The pre-processing module (305) can be configured to apply one or more transformations to the extracted training data, for example, to clean and normalize the data. The pre-processing module can be configured to discard data with spurious metadata or with very few observations. The pre-processing module can be further configured to standardize the coding of behavioral units.

[0109] The training module (310) is used to train a machine learning algorithm. The machine learning algorithm can be constructed to capture statistical associations, if any, between a given behavioral unit, higher order behavior, or behavioral pattern and a particular behavioral disorder, developmental delay, or nervous system dysfunction based on the training data. The machine learning algorithm may, for example, include statistical correlations between a plurality of clinical features and clinical diagnoses of one or more behavioral disorders, developmental delays, or nervous system dysfunctions. A given behavioral unit may have different predictive utility for classifying each of the plurality of behavioral disorders, developmental delays, or nervous system dysfunctions being evaluated. A likelihood score describing the likelihood of a particular behavioral disorder, developmental delay, or nervous system dysfunction may be extracted to predict each of the plurality of behavioral disorders, developmental delays, or nervous system dysfunctions. The machine learning algorithm can be used to extract these statistical associations from the training data to construct a model that can accurately predict the disorder when a dataset including one or more behavioral disorders, developmental delays, or nervous system dysfunctions is fitted to the model.

[0110] One or more machine learning algorithms may be used to build machine learning algorithms such as support vector machines that stepwise arrange the selection and / or graphic model of behavioral units backwards. Both of these may have the advantage of inferring interactions between behavioral units. Machine learning algorithms or other statistical algorithms may be used, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), long short term memory networks (LSTMs), alternating decision trees (ADTrees), decision stumps, feature trees (FTs), logistic model trees (LMTs), logistic regression, random forests, linear classifiers, or any machine learning or statistical algorithms known in the art. One or more algorithms may be used together to create an ensemble method, which may be optimized using a machine learning ensemble meta-algorithm such as boosting (e.g., AdaBoost, LPBoost, TotalBoost, BrownBoost, MadaBoost, LogitBoost, etc.) to reduce bias and / or variance. Once the machine learning algorithm is derived from the training data, the model can be used as a predictive tool to assess the risk of a subject having one or more disorders. Machine learning analysis can be performed using one or more of many programming languages ​​and platforms known in the art, such as TensorFlow, Keras, R, Weka, Python, and / or Matlab.

[0111] Alternatively, or in combination, a behavioral unit, higher order behavior, or behavioral pattern of interest in a subject can be evaluated through a structured interaction with the subject. For example, the subject can be asked to play a game, such as a computer game, and the subject's performance on the game can be used to evaluate one or more behavioral units, higher order behaviors, or behavioral patterns of the subject. The subject can be presented with one or more stimuli (e.g., visual stimuli presented to the subject via a display), and the subject's response to the stimuli can be used to evaluate the subject's behavioral unit, higher order behavior, or behavioral pattern. The subject can be asked to perform a particular task (e.g., the subject can be asked to pop a bubble with his / her finger), and the subject's response to the request, or the subject's ability to perform the requested task, can be used to evaluate the subject's behavioral unit, higher order behavior, or behavioral pattern.

[0112] Systems and Devices The present disclosure provides a computer controlled device programmed to implement the method of the present disclosure. Figure 4 illustrates a computer device (401) suitable for incorporation into the platform, device, and method described herein. The computer device (401) can process various aspects of the information of the present disclosure, such as questions, answers, responses, statistical analysis, etc. The computer device (401) is an electronic device of a user or a computer device that is remotely located relative to the electronic device. The electronic device may be a mobile electronic device.

[0113] The computing device (401) includes a central processing unit (CPU, herein "processor" and "computer processor") (405), which may be a single-core or multi-core processor, or multiple processors for parallel processing. The computing device (401) also includes memory or storage locations (410) (e.g., random access memory, read-only memory, flash memory), electronic storage (415) (e.g., hard disk), communication interfaces (420) (e.g., network adapters) for communicating with one or more other devices, and peripherals (425) such as cache, other memory, data storage, and / or electronic display adapters. The memory (410), storage (415), interface (420), and peripherals (425) communicate with the CPU (405) through a communication bus (solid lines), such as a motherboard. The storage (415) may be a data storage device (or data repository) for storing data. The computing device (401) may be operatively coupled to a computer network ("network") (430) with the aid of a communication interface (420). The network (430) may be the Internet and / or an extranet, or an intranet and / or an extranet in communication with the Internet. In some cases, the network (430) is a telecommunications and / or data network. The network (430) may include one or more computer servers enabling distributed computing, such as cloud computing. The network (430), optionally with the aid of the computing device (401), may implement a peer-to-peer network, which may enable devices coupled to the computing device (401) to act as clients or servers.

[0114] The CPU (405) may execute sequences of machine-readable instructions, which may be embedded in a program or software. The instructions may be stored in a memory location, such as the memory (410). The instructions may be directed to the CPU (405), which may then program or otherwise configure the CPU (405) to perform the methods of the present disclosure. Examples of operations performed by the CPU (405) may include fetch, decode, execute, and writeback.

[0115] The CPU (405) may be part of a circuit, such as an integrated circuit. One or more other components of the device (401) may be included in the circuit. In some cases, the circuit is an application specific integrated circuit (ASIC).

[0116] The storage device (415) can store files such as drivers, libraries, and saved programs. The storage device (415) can store user data, such as user preferences and user programs. The computing device (401) may optionally include one or more additional data storage devices external to the computing device (401), such as located on a remote server in communication with the computing device (401) over an intranet or the Internet.

[0117] The computing device (401) can communicate with one or more remote computing devices via the network (430). For example, the computing device (401) can communicate with a remote computing device of a user (e.g., a parent). Examples of remote computing devices and mobile communication devices include a personal computer (e.g., a portable PC), a slate or tablet PC (e.g., Apple® iPad®, Samsung® Galaxy Tab), a phone, a smartphone (e.g., Apple® iPhone®, Android-enabled device, Blackberry®), or a personal digital assistant. A user can access the computing device (401) using the network (430).

[0118] Methods as described herein can be implemented as machine (e.g., computer processing unit) executable code stored in an electronic storage location of the computing device (401), such as memory (410) or electronic storage (415). The machine executable or machine readable code can be provided in the form of software. In use, the code may be executed by the processor (405). In some cases, the code can be retrieved from the electronic storage (415) and stored in the memory (410) for easy access by the processor (405). In some circumstances, the electronic storage (415) may be omitted and the machine executable instructions are stored in the memory (410).

[0119] The code may be pre-compiled and configured for use with a machine having a processor adapted to execute the code, or may be compiled on-the-fly. The code may be provided in a selectable programming language such that the code is executable in a pre-compiled or as-compiled fashion.

[0120] Aspects of the devices and methods provided herein, such as the computing device (401), may be embodied in programming. Various aspects of the technology may be considered as an "article of manufacture" or "article of manufacture" that is generally in the form of machine (or processor) executable code and / or associated data carried on or embedded in a type of machine-readable medium. The machine executable code may be stored in electronic storage such as memory (e.g., read-only memory, random access memory, flash memory) or hard disk. A "storage" type medium may include any or all of the tangible memory of a computer, such as various semiconductor memories, tape drives, disk drives, etc., or processors, or associated modules thereof, which may provide non-transitory storage at any time for programming of the software. All or portions of the software may be communicated from time to time over various telecommunications networks, such as the Internet. Such communication may enable, for example, loading of the software from one computer or processor to another, such as from a management server or host computer to a computer platform of an application server. Thus, other types of media that may bear software elements include light waves, radio waves, and electromagnetic waves, such as those used across physical interfaces between local devices over wired and optical landline networks, as well as over various air-links. Physical elements such as wired, wireless, and optical links that carry such waves may also be considered media bearing the software. As used herein, unless limited to non-transitory, tangible "storage" media, terms such as computer or machine "readable medium" refer to media that participate in sending instructions to a processor for execution.

[0121] Thus, a machine-readable medium such as a computer-executable code may take many forms, including but not limited to a tangible storage medium, a carrier wave medium, or a physical transmission medium. Non-volatile storage media include optical and magnetic disks, such as any of the storage devices in a computer, such as those that can be used to implement the databases shown in the figures. Volatile storage media include dynamic memory, such as the main memory of such a computer platform. Tangible transmission media include coaxial cables, copper wire, and optical fibers, including, for example, the wires that comprise a bus within a computing device. Carrier wave transmission media may take the form of electric, electromagnetic, acoustic, or light waves, such as those generated during radio frequency (RF) and infrared (IR) data communications. Thus, common forms of computer readable media include, for example, floppy disks, flexible disks, hard disks, magnetic tapes, other magnetic media, CD-ROMs, DVDs or DVD-ROMs, other optical media, punch cards, paper tapes, other physical storage media with patterns of holes, RAM, ROM, PROMs and EPROMs, FLASH-EPROMs, other memory chips or cartridges, carrier waves carrying data or instructions, cables or links which transmit such carrier waves, or other media from which programming code and / or data can be read by a computer. Many of these forms of computer readable media may require the delivery of one or more sequential instructions to one or more processors for execution.

[0122] The computing device (401) may include or be in communication with an electronic display (435) that includes a user interface (UI) for providing, for example, questions and answers, analysis results, and recommendations. Examples of UIs include, but are not limited to, graphical user interfaces (GUIs) and web-based user interfaces.

[0123] The disclosed platform, device, and method can be implemented as one or more algorithms with instructions provided by one or more processors as disclosed herein. The algorithm can be implemented as software after execution by the central processing unit (405). The algorithm can be, for example, a random forest, a graphic model, a support vector machine, or others.

[0124] The above steps illustrate a method of the device according to one example, but one of ordinary skill in the art would recognize many variations based on the teachings provided herein. Steps may be completed in different orders. Steps may be added or removed. Some of the steps may include sub-steps. Many of the steps may be repeated as often as beneficial to the platform.

[0125] Many of the examples as described herein can be combined with one or more other examples. Additionally, one or more components of one or more examples can be combined with other examples.

[0126] treatment The platform, device and method disclosed herein can incorporate pharmaceutical treatment. In some embodiments, drugs for treating behavioral disorders, developmental delays, and nervous system dysfunctions disclosed herein are used. In some embodiments, the effectiveness of drug treatment is monitored or evaluated. In some embodiments, a method of administering a drug to a subject includes detecting a behavioral disorder, developmental delay, or nervous system dysfunction of a subject using a machine learning algorithm disclosed herein, and administering a drug to the subject according to the detected behavioral disorder, developmental delay, or nervous system dysfunction. Behavioral disorders, developmental delays, or neurological dysfunction include Attention Deficit Disorder (ADD), Obsessive-Compulsive Disorder, Acute Stress Disorder, Adjustment Disorder, Agnostic Phobias, Alzheimer's Disease, Anorexia Nervosa, Anxiety Disorder, Bereavement, Bipolar Disorder, Body Dysmorphic Disorder, Brief Psychotic Disorder, Hyperorexia Nervosa, Conduct Disorder, Delusional Disorder, Depersonalization Disorder, Depression, Dissociative Amnesia, Dissociative Disorder, Dissociative Wandering Disorder, Dysthymic Disorder, Eating Disorder, Gender Dysphoria, General may include sexual anxiety disorder, hoarding disorder, intermittent explosive disorder, kleptomania, panic disorder, Parkinson's disease, gambling disorder, postpartum depression, posttraumatic stress disorder, premenstrual dysphoric disorder, emotional dysregulation disorder, pyromania, schizoaffective disorder, schizophrenia, schizophreniform disorder, seasonal affective disorder, shared psychotic disorder, social phobia, specific phobia, stereotypic movement disorder, Tourette's disorder, episodic tic disorder, or trichotillomania.

[0127] The behavioral disorder, developmental delay, or nervous system dysfunction may comprise an autism spectrum disorder and the medication may be selected from the group consisting of risperidone, quetiapine, amphetamine, dextroamphetamine, methylphenidate, methamphetamine, dextroamphetamine, dextromethylphenidate, guanfacine, atomoxetine, lisdexamfetamine, clonidine, and aripiprazole; or the behavioral disorder, developmental delay, or nervous system dysfunction comprises attention deficit disorder (ADD); The drug may be selected from the group consisting of amphetamine, dextroamphetamine, methylphenidate, methamphetamine, dextroamphetamine, dextromethylphenidate, guanfacine, atomoxetine, lisdexamfetamine, clonidine, and modafinil, or the behavioral disorder, developmental delay, or nervous system dysfunction comprises obsessive-compulsive disorder and the drug is selected from the group consisting of buspirone, sertraline, escitalopram, citalopram, fluoxetine, paroxetine, venlafaxine, clomipramine, and fluvoxamine. or the behavioral disorder, developmental delay, or nervous system dysfunction comprises acute stress disorder and the medication may be selected from the group consisting of propranolol, citalopram, escitalopram, sertraline, paroxetine, fluoxetine, venlafaxine, mirtazapine, nefazodone, carbamazepine, divalproex, lamotrigine, topiramate, prazosin, phenelzine, imipramine, diazepam, clonazepam, lorazepam, and alprazolam, ... an indication and the behavioral disorder, developmental delay, or nervous system dysfunction comprises acrophobia and the drug may be selected from the group consisting of diazepam, clonazepam, lorazepam, alprazolam, citalopram, escitalopram, sertraline, paroxetine, fluoxetine, diazepam, clonazepam, lorazepam, and alprazolam, or the behavioral disorder, developmental delay, or nervous system dysfunction comprises acrophobia and the drug may be selected from the group consisting of diazepam, clonazepam, lorazepam, alprazolam, citalopram, escitalopram, sertraline, paroxetine, fluoxetine, and buspirone, or the behavioral disorder, developmental delay, or nervous system dysfunction comprisesThe disorder includes Alzheimer's disease, and the drug may be selected from the group consisting of donepezil, galantamine, memantine, and rivastigmine; or the behavioral disorder, developmental delay, or nervous system dysfunction includes anorexia nervosa, and the drug may be selected from the group consisting of olanzapine, citalopram, escitalopram, sertraline, paroxetine, and fluoxetine; or the behavioral disorder, developmental delay, or nervous system dysfunction includes anxiety disorder, and the drug may be selected from the group consisting of sertraline, escitalopram, citalopram, fluoxetine, diazepam, buspirone. , venlafaxine, duloxetine, imipramine, desipramine, clomipramine, lorazepam, clonazepam, and pregabalin; or the behavioral disorder, developmental delay, or nervous system dysfunction comprises bereavement and the medication may be selected from the group consisting of citalopram, duloxetine, and doxepin; or the behavioral disorder, developmental delay, or nervous system dysfunction comprises binge eating disorder and the medication may be selected from the group consisting of lisdexamfetamine; or the behavioral disorder, developmental delay, or nervous system dysfunction comprises body dysmorphic disorder and the medication may be selected from the group consisting of sertraline, escitalopram, and citalopram; or the behavioral disorder, developmental delay, or nervous system dysfunction comprises brief psychotic disorder and the medication may be selected from the group consisting of clozapine, asenapine, olanzapine, and or the behavioral disorder, developmental delay, or nervous system dysfunction comprises hyperuricemia and the medication may be selected from the group consisting of sertraline and fluoxetine, or the behavioral disorder, developmental delay, or nervous system dysfunction comprises conduct disorder and the medication may be selected from the group consisting of lorazepam, diazepam, and clobazam, or the behavioral disorder, developmental delay, or nervous system dysfunction comprises delusional disorder and the medication may be selected from the group consisting of clozapine, asenapine, risperidone, venlafaxine, bupropion,and buspirone; the behavioral disorder, developmental delay, or nervous system dysfunction comprises depersonalization disorder and the medication may be selected from the group consisting of sertraline, fluoxetine, alprazolam, diazepam, and citalopram; or the behavioral disorder, developmental delay, or nervous system dysfunction comprises depression and the medication may be selected from the group consisting of sertraline, fluoxetine, citalopram, bupropion, escitalopram, venlafaxine, aripiprazole, buspirone, vortioxetine, and vilazodone. or the behavioral disorder, developmental delay, or nervous system dysfunction comprises severe mood dysregulation disorder and the medication may be selected from the group consisting of quetiapine, clozapine, asenapine, and pimavanserin; or the behavioral disorder, developmental delay, or nervous system dysfunction comprises dissociative amnesia and the medication may be selected from the group consisting of alprazolam, diazepam, lorazepam, and chlordiazepoxide; or the behavioral disorder, developmental delay, or nervous system dysfunction comprises a dissociative disorder and the medication may be selected from the group consisting of bupropion, vortioxetine, and vilazodone. Alternatively, the behavioral disorder, developmental delay, or nervous system dysfunction comprises dissociative wandering and the medication may be selected from the group consisting of amobarbital, aprobarbital, butabarbital, and methohexital; or the behavioral disorder, developmental delay, or nervous system dysfunction comprises dysthymic disorder and the medication may be selected from the group consisting of bupropion, venlafaxine, sertraline, and citalopram; or the behavioral disorder, developmental delay, or nervous system dysfunction comprises an eating disorder and the medication may be selected from the group consisting of olanzapine, citalopram, escitabine, sirolimus ... or the behavioral disorder, developmental delay, or nervous system dysfunction comprises gender dysphoria and the medication may be selected from the group consisting of estrogen, prostogen, and testosterone; or the behavioral disorder, developmental delay, or nervous system dysfunction comprises generalized anxiety disorder and the medication may be selected from the group consisting of venlafaxine, duloxetine, buspirone, sertraline, and fluoxetine; or the behavioral disorder, developmental delay, or nervous system dysfunction comprises generalized anxiety disorder and the medication may be selected from the group consisting of venlafaxine, duloxetine, buspirone, sertraline, and fluoxetine.The developmental delay, or nervous system dysfunction may comprise a hoarding disorder and the medication may be selected from the group consisting of buspirone, sertraline, escitalopram, citalopram, fluoxetine, paroxetine, venlafaxine, and clomipramine; or the behavioral disorder, developmental delay, or nervous system dysfunction may comprise an intermittent explosive disorder and the medication may be selected from the group consisting of asenapine, clozapine, olanzapine, and pimavanserin; or the behavioral disorder, developmental delay, or nervous system dysfunction may comprise kleptomania and the medication may be selected from the group consisting of espirone, sertraline, escitalopram, citalopram, fluoxetine, paroxetine, venlafaxine, and clomipramine. Alternatively, the behavioral disorder, developmental delay, or nervous system dysfunction comprises panic disorder and the drug is selected from the group consisting of bupropion, vilazodone, and vortioxetine; or the behavioral disorder, developmental delay, or nervous system dysfunction comprises Parkinson's disease and the drug is selected from the group consisting of rivastigmine, selegiline, rasagiline, bromocriptine, amantadine, cabergoline, and benztropine. or the behavioral disorder, developmental delay, or nervous system dysfunction comprises gambling addiction and the medication may be selected from the group consisting of bupropion, vilazodone, and vortioxetine, or the behavioral disorder, developmental delay, or nervous system dysfunction comprises postpartum depression and the medication may be selected from the group consisting of sertraline, fluoxetine, citalopram, bupropion, escitalopram, venlafaxine, aripiprazole, buspirone, vortioxetine, and vilazodone, or The behavioral disorder, developmental delay, or nervous system dysfunction comprises post-traumatic stress disorder and the medication may be selected from the group consisting of sertraline, fluoxetine, and paroxetine; or the behavioral disorder, developmental delay, or nervous system dysfunction comprises premenstrual dysphoric disorder and the medication may be selected from the group consisting of estradiol, drospirenone, sertraline, citalopram, fluoxetine, and buspirone; or the behavioral disorder, developmental delay, or nervous system dysfunction comprises emotional dysregulation disorder and the medication may be selected from the group consisting ofor the behavioral disorder, developmental delay, or nervous system dysfunction comprises pyromania and the medication may be selected from the group consisting of clozapine, asenapine, olanzapine, paliperidone, and quetiapine; or the behavioral disorder, developmental delay, or nervous system dysfunction comprises schizoaffective disorder and the medication is selected from the group consisting of sertraline, carbamazepine, oxcarbazepine, valproate, haloperidol, olanzapine, and loxapine. or the behavioral disorder, developmental delay, or nervous system dysfunction comprises schizophrenia and the medication may be selected from the group consisting of chlorpromazine, haloperidol, fluphenazine, risperidone, quetiapine, ziprasidone, olanzapine, perphenazine, aripiprazole, and procurarperazine; or the behavioral disorder, developmental delay, or nervous system dysfunction comprises schizophreniform disorder and the medication may be selected from the group consisting of paliperidone, clozapine, and risperidone; The disorder may include seasonal affective disorder and the medication may be selected from the group consisting of sertraline and fluoxetine; or the behavioral disorder, developmental delay, or nervous system dysfunction comprises a shared psychotic disorder and the medication may be selected from the group consisting of clozapine, pimavanserin, risperidone, and lurasidone; or the behavioral disorder, developmental delay, or nervous system dysfunction comprises social phobia and the medication may be selected from the group consisting of amitriptyline, bupropion, citalopram, fluoxetine, sertraline, and venlafaxine. or the behavioral disorder, developmental delay, or nervous system dysfunction comprises a specific phobia and the medication may be selected from the group consisting of diazepam, estazolam, quazepam, and alprazolam; or the behavioral disorder, developmental delay, or nervous system dysfunction comprises a stereotypic movement disorder and the medication may be selected from the group consisting of risperidone and clozapine; or the behavioral disorder, developmental delay, or nervous system dysfunction comprises Tourette's disorder and the medication may be selected from the group consisting of haloperidol, fluphenazine, risperidone, ziprasidone, pimozide, perphenazine,and aripiprazole, or the behavioral disorder, developmental delay, or nervous system dysfunction comprises a temporary tic disorder and the drug may be selected from the group consisting of guanfacine, clonidine, pimozide, risperidone, citalopram, escitalopram, sertraline, paroxetine, and fluoxetine, or the behavioral disorder, developmental delay, or The nervous system dysfunction may include trichotillomania and the drug may be selected from the group consisting of sertraline, fluoxetine, paroxetine, desipramine, and clomipramine.

[0128] Devices and systems for assessing behavioral disorders, developmental delays, and nervous system dysfunction Described herein is a platform, device and method for determining the developmental progress of a subject.For example, the described platform, device and method can identify a subject as having progress in one or more developmental domains or cognitive decline in one or more cognitive functions, or as having delayed development or at risk of one or more behavioral disorders, developmental delay, or nervous system dysfunction.The disclosed platform, device and method can assess the progress of a subject by evaluating a plurality of behavioral units of the subject based on an evaluation model, and the evaluation model can be generated from a large data set of a relevant subject population using machine learning techniques.

[0129] The platform, device, and method are described herein in terms of identifying one or more behavioral disorders, developmental delays, or nervous system dysfunctions in a subject. For example, the platform, device, and method can be used to identify a subject as being developmentally advanced by identifying one or more areas of development in which the subject is advanced. To identify one or more areas of advanced development, the platform, device, and method can be configured to evaluate one or more characteristics or traits of the subject, for example, related to advanced or gifted behavior. The platform, device, and method as described can also be used to identify a subject as being cognitively impaired in one or more cognitive functions by evaluating one or more cognitive functions of the subject.

[0130] Described herein are platforms, devices and methods for diagnosing or evaluating the behavioral units, behavioral functions, and / or higher order behaviors or behavioral patterns associated with one or more behavioral disorders, developmental delays, or nervous system dysfunctions of a subject.This process may include the evaluation of the cognitive functions, features, or characteristics associated with one or more behavioral disorders, developmental delays, or nervous system dysfunctions.For example, a person can be evaluated for speech and / or language proficiency based on the behavioral units, higher order behaviors, and / or behavioral patterns disclosed herein.

[0131] Described herein are platforms, devices, and methods for diagnosing or assessing the risk of one or more behavioral disorders, developmental delays, or nervous system dysfunction in a subject. The method may include providing a data processing module that can be used to construct and implement an assessment procedure for screening a subject for one or more of a plurality of developmental disorders or diseases. The assessment procedure can assess a plurality of characteristics or traits of the subject through the use of machine learning algorithms, and each characteristic can be related to the likelihood that the subject has at least one of a plurality of disorders that can be screened by the procedure. Each characteristic can be related to the likelihood that the subject has two or more related disorders, and the two or more related disorders can have one or more associated symptoms. The characteristics can be assessed in many ways. For example, the characteristics can be assessed through the subject's response to a question, through the observation of the subject, or through the results of a structured dialogue with the subject, as described herein.

[0132] In some embodiments, the systems and devices disclosed herein include a recording device. In some embodiments, the video and / or audio recording is performed using a mobile device. In some embodiments, the mobile device is a smartphone, tablet, smartwatch, or other device with a mobile camera or recording capabilities. In some embodiments, the video and / or audio recording is performed using a fixed camera and / or microphone. For example, an individual may be posed a question in a clinician's office and their response may be recorded by a camera on a tripod with a microphone attached. In some embodiments, the camera is a high-definition camera. In some embodiments, the individual is prompted to provide a response via the device interface, e.g., to select or type an answer or response to a question or prompt within the touch screen interface of the device. In some embodiments, the audio itself is recorded and analyzed to accurately measure behavior, e.g., to determine speech and language levels. Information obtained based on the analysis of these inputs (e.g., audio, video, responses, etc.) may be evaluated to determine appropriate treatment. The digital therapy may be an interactive activity, where the activity is dynamically modified or changed while the individual is engaged in the activity, based on the analysis of the inputs. A variety of modalities may be utilized in which an individual remains engaged in an activity and is monitored during the activity, and the activity is dynamically modified or adjusted based on real-time analysis of the individual's input or responses. As an example, a storytelling activity may guide an individual through an interactive narrative that may be separated into various storylines or threads of varying difficulty, where an appropriate thread is selected based on an assessment of the individual's speech and language level associated with each of the available thread difficulties.

[0133] In some cases, the device operates at a frame rate of up to 15-20 FPS, enabling facial expression recognition within 100 ms. The device may operate at frame rates of 10 FPS-100 FPS. The device may operate at frame rates of 1 FPS-10 FPS, 1 FPS-20 FPS, 1 FPS-30 FPS, 1 FPS-40 FPS, 1 FPS-50 FPS, 1 FPS-100 FPS, 10 FPS-15 FPS, 10 FPS-20 FPS, 10 FPS-25 FPS, 10 FPS-30 FPS, 10 FPS-35 FPS, 10 FPS-40 FPS, 10 FPS-45 FPS, 10 FPS-50 FPS, 10 FPS-60 FPS, 10 FPS-80 FPS, 10 FPS-100 FPS, 15 FPS-20 FPS, 15 FPS- 25FPS, 15FPS~30FPS, 15FPS~35FPS, 15FPS~40FPS, 15FPS~45FPS, 15FPS~50FPS, 15FPS~60FPS, 15FPS~80FPS, 15FPS~100FPS, 20FPS~25F PS, 20FPS~30FPS, 20FPS~35FPS, 20FPS~40FPS, 20FPS~45FPS, 20FPS~50FPS, 20FPS~60FPS, 20FPS~80FPS, 20FPS~100FPS, 25FPS~30FPS, 2 5FPS~35FPS, 25FPS~40FPS, 25FPS~45FPS, 25FPS~50FPS, 25FPS~60FPS, 25FPS~80FPS, 25FPS~100FPS, 30FPS~35FPS, 30FPS~40FPS, 30FP S~45FPS, 30FPS~50FPS, 30FPS~60FPS, 30FPS~80FPS, 30FPS~100FPS, 35FPS~40FPS, 35FPS~45FPS, 35FPS~50FPS, 35FPS~60FPS, 35FPS~8 It can operate at frame rates of 0FPS, 35FPS-100FPS, 40FPS-45FPS, 40FPS-50FPS, 40FPS-60FPS, 40FPS-80FPS, 40FPS-100FPS, 45FPS-50FPS, 45FPS-60FPS, 45FPS-80FPS, 45FPS-100FPS, 50FPS-60FPS, 50FPS-80FPS, 50FPS-100FPS, 60FPS-80FPS, 60FPS-100FPS, or 80FPS-100FPS.The device may operate at a frame rate of 10FPS, 15FPS, 20FPS, 25FPS, 30FPS, 35FPS, 40FPS, 45FPS, 50FPS, 60FPS, 80FPS, or 100FPS. The device may operate at a frame rate of at least 1FPS, 5FPS, 10FPS, 15FPS, 20FPS, 25FPS, 30FPS, 35FPS, 40FPS, 45FPS, 50FPS, 60FPS, or 80FPS. The device may operate at a frame rate of at most 15FPS, 20FPS, 25FPS, 30FPS, 35FPS, 40FPS, 45FPS, 50FPS, 60FPS, 80FPS, or 100FPS.

[0134] In some cases, the device is capable of detecting facial expressions or movements within 10ms to 200ms. 0ms~90ms, 20ms~100ms, 20ms~150ms, 20ms~200ms, 30ms~40ms, 30ms~50ms, 30ms~60ms, 30ms~70ms, 30ms~80m s, 30ms~90ms, 30ms~100ms, 30ms~150ms, 30ms~200ms, 40ms~50ms, 40ms~60ms, 40ms~70ms, 40ms~80ms, 40ms~9 0ms, 40ms~100ms, 40ms~150ms, 40ms~200ms, 50ms~60ms, 50ms~70ms, 50ms~80ms, 50ms~90ms, 50ms~100ms, 50 ms~150ms, 50ms~200ms, 60ms~70ms, 60ms~80ms, 60ms~90ms, 60ms~100ms, 60ms~150ms, 60ms~200ms, 70ms~80m The device can detect facial expressions or movements within 10 ms, 20 ms, 30 ms, 40 ms, 50 ms, 60 ms, 70 ms, 80 ms, 90 ms, 100 ms, 150 ms, 150 ms, or 200 ms. The device can detect facial expressions or movements within 10 ms, 20 ms, 30 ms, 40 ms, 50 ms, 60 ms, 70 ms, 80 ms, 90 ms, 100 ms, 150 ms, or 200 ms. The device can detect facial expressions or movements within at least 10 ms, 20 ms, 30 ms, 40 ms, 50 ms, 60 ms, 70 ms, 80 ms, 90 ms, 100 ms, or 150 ms.The device can detect facial expressions or movements within at most 20ms, 30ms, 40ms, 50ms, 60ms, 70ms, 80ms, 90ms, 100ms, 150ms, or 200ms.

[0135] In some embodiments, the systems and devices disclosed herein include biosensors to obtain measurements of a user. For example, biosensors may be used to measure an individual's heart rate, blood glucose level, respiration rate, or activity level. In some embodiments, biosensors include electrocardiogram or electroencephalogram sensors, potentiometers, accelerometers, or gyrometers. Such measurements can be used to assess or augment an individual's response and / or interaction assessment in digital diagnostic and / or therapeutic activities, such as interactive storytelling activities, during which an individual is guided through a story that elicits verbal, visual, and / or digital responses that are used to dynamically modify the activity.

[0136] Digital Processing Device In some embodiments, the platforms, systems, media, and methods described herein include a digital processing device, or the use thereof. In further embodiments, the digital processing device comprises one or more hardware central processing units (CPUs) or general purpose graphic processing units (GPGPUs) that perform the functions of the device. In still further embodiments, the digital processing device further comprises an operating system configured to execute the executable instructions. In some embodiments, the digital processing device is optionally connected to a computer network. In further embodiments, the digital processing device is optionally connected to the Internet to access the World Wide Web. In still further embodiments, the digital processing device is optionally connected to a cloud computing infrastructure. In other embodiments, the digital processing device is optionally connected to an intranet. In other embodiments, the digital processing device is optionally connected to a data storage device.

[0137] According to the description herein, non-limiting examples of suitable digital processing devices include server computers, desktop computers, laptop computers, notebook computers, sub-notebook computers, netbook computers, netpad computers, set-top computers, media streaming devices, handheld computers, Internet appliances, mobile smart phones, tablet computers, personal digital assistants, video game consoles, and vehicles. Those skilled in the art will recognize that many smart phones are suitable for use with the systems described herein. Those skilled in the art will also recognize that selected televisions, video players, and digital music players, with optional computer network connectivity, are suitable for use with the systems described herein. Suitable tablet computers include those with booklet, slate, and convertible configurations known to those skilled in the art.

[0138] In some embodiments, the digital processing device includes an operating system configured to execute executable instructions. The operating system is software with programs and data that, for example, manages the device's hardware and provides services for the implementation of applications. Those skilled in the art will recognize that suitable server operating systems include, but are not limited to, FreeBSD, OpenBSD, NetBSD®, Linux®, Apple®, Mac OS X Server®, Oracle® Solaris®, Windows Server®, and Novell® NetWare®. Those skilled in the art will recognize that suitable personal computer operating systems include, but are not limited to, Microsoft® Windows®, Apple® Mac OS X®, UNIX®, and UNIX-like operating systems such as GNU / Linux®. In some embodiments, the operating system is provided by cloud computing. Those skilled in the art will also recognize that suitable mobile smartphone operating systems include, but are not limited to, Nokia® Symbian® OS, Apple® iOS®, Research In Motion® BlackBerry OS®, Google® Android®, Microsoft® Windows Phone® OS, Microsoft® Windows Mobile® OS, Linux®, and Palm® WebOS®.Those skilled in the art will also recognize that suitable media streaming device operating systems include, but are not limited to, Apple TV®, Roku®, Boxee®, Google TV®, Google Chromecast®, Amazon Fire®, and Samsung® HomeSync®. Those skilled in the art will also recognize that suitable video game console operating systems include, but are not limited to, Sony® PS3®, Sony® PS4®, Microsoft® Xbox 360®, Microsoft Xbox One, Nintendo® Wii®, Nintendo® Wii U®, and Ouya®.

[0139] In some embodiments, the device includes a storage device and / or a memory device. A storage device and / or a memory device is one or more physical devices used to temporarily or permanently store data or programs. In some embodiments, the device is a volatile memory and requires power to maintain stored information. In some embodiments, the device is a non-volatile memory and retains stored information when power is not provided to the digital processing device. In further embodiments, the non-volatile memory includes a flash memory. In some embodiments, the non-volatile memory includes a dynamic random access memory (DRAM). In some embodiments, the non-volatile memory includes a ferroelectric random access memory (FRAM). In some embodiments, the non-volatile memory includes a phase change random access memory (PRAM). In other embodiments, the device is a storage device, including but not limited to CD-ROMs, DVDs, flash memory devices, magnetic disks, magnetic tape drives, optical disk drives, and cloud computing based storage devices. In further embodiments, the storage device and / or memory device is a combination of devices such as those disclosed herein.

[0140] In some embodiments, the digital processing device includes a display for communicating visual information to a user. In some embodiments, the display is a liquid crystal display (LCD). In further embodiments, the display is a thin film transistor liquid crystal display (TFT-LCD). In some embodiments, the display is an organic light emitting diode (OLED) display. In various further embodiments, the OLED display is a passive matrix OLED (PMOLED) or active matrix OLED (AMOLED) display. In some embodiments, the display is a plasma display. In other embodiments, the display is a video projector. In still other embodiments, the display is a head-mounted display in communication with the digital processing device, such as a VR headset. In further embodiments, suitable VR headsets include, but are not limited to, HTC Vive, Oculus Rift, Samsung Gear VR, Microsoft HoloLens, Razer OSVR, FOVE VR, Zeiss VR One, Avegant Glyph, Freefly VR headsets, and the like. In still further embodiments, the display is a combination of devices such as those disclosed herein.

[0141] In some embodiments, the digital processing device includes an input device for receiving information from a user. In some embodiments, the input device is a keyboard. In some embodiments, the input device is a pointing device, including but not limited to a mouse, a trackball, a trackpad, a joystick, a game controller, or a stylus. In some embodiments, the input device is a touch screen or a multi-touch screen. In other embodiments, the input device is a microphone that captures voice or other sound input. In other embodiments, the input device is a video camera or other sensor that captures motion or visual input. In further embodiments, the input device is a Kinect, Leap Motion, or the like. In still further embodiments, the input device is a combination of devices such as those disclosed herein.

[0142] Non-transitory computer-readable storage medium In some examples, the platforms, systems, media, and methods disclosed herein include one or more non-transitory computer-readable storage media encoded with a program including instructions executable by an operating system of an optionally networked digital processing device. In further embodiments, the computer-readable storage medium is a tangible part of the digital processing device. In still further embodiments, the computer-readable storage medium is optionally removable from the digital processing device. In some embodiments, the computer-readable storage medium includes, but is not limited to, CD-ROMs, DVDs, flash memory devices, solid-state memories, magnetic disks, magnetic tape drives, optical disk drives, cloud computing systems, and services. In some cases, the programs and instructions are encoded on the medium permanently, nearly permanently, semi-permanently, or non-transitoryly.

[0143] Computer Programs In some embodiments, the platform, system, medium, and method disclosed herein includes at least one computer program, or the use thereof. A computer program is executable in a CPU of a digital processing device and includes a set of instructions written to perform a particular task. The computer readable instructions may be implemented as program modules, such as functions, objects, application program interfaces (APIs), data structures, etc., that perform particular tasks or implement particular abstract data types. In light of the disclosure provided herein, one skilled in the art will recognize that computer programs may be written in various versions of various languages.

[0144] The functionality of the computer readable instructions may be combined or distributed as desired in various environments. In some embodiments, the computer program comprises one sequence of instructions. In some embodiments, the computer program comprises multiple sequences of instructions. In some embodiments, the computer program is provided from one location. In other embodiments, the computer program is provided from multiple locations. In various embodiments, the computer program comprises one or more software modules. In various embodiments, the computer program comprises, in part or in whole, one or more web applications, one or more mobile applications, one or more stand-alone applications, one or more web browser plug-ins, extensions, add-ins, or add-ons, or combinations thereof.

[0145] Web Applications In some embodiments, the computer program comprises a web application. In light of the disclosure provided herein, one skilled in the art will recognize that web applications, in various embodiments, utilize one or more software frameworks and one or more database systems. In some embodiments, the web application is created on a software framework such as Microsoft® .NET or Ruby on Rails (RoR). In some embodiments, the web application utilizes one or more database systems, including, but not limited to, relational, non-relational, object-oriented, associative, and XML database systems. In further embodiments, suitable relational database systems include, but are not limited to, Microsoft® SQL Server, mySQL™, and Oracle®. One skilled in the art will recognize that web applications, in various embodiments, are written in one or more versions of one or more languages. Web applications may be written in one or more markup languages, presentation definition languages, client-side scripting languages, server-side coding languages, database query languages, or combinations thereof. In some embodiments, the web application is written to some extent in a markup language such as Hypertext Markup Language (HTML), Extensible Hypertext Markup Language (XHTML), or eXtensible Markup Language (XML). In some embodiments, the web application is written to some extent in a presentation definition language such as Cascading Style Sheets (CSS). In some embodiments, the web application is written to some extent in client-side script such as Asynchronous Javascript and XML (AJAX), Flash® Actionscript, Javascript, or Silverlight®.In some embodiments, the web application is written to some extent in a server-side coding language such as Active Server Pages (ASP), ColdFusion®, Perl, Java™, JavaServer Pages (JSP), Hypertext Preprocessor (PHP), Python™, Ruby, Tcl, Smalltalk, WebDNA®, or Groovy. In some embodiments, the web application is written to some extent in a database query language such as Structured Query Language (SQL). In some embodiments, the web application integrates with an enterprise server product such as IBM® Lotus Domino®. In some embodiments, the web application includes a media player element. In various further embodiments, the media player element utilizes one or more of many suitable multimedia techniques, including, but not limited to, Adobe® Flash®, HTML 5, Apple® QuickTime®, Microsoft® Silverlight®, Java®, and Unity®.

[0146] Mobile Applications In some embodiments the computer program comprises a mobile application provided to the mobile digital processing device, in some embodiments the mobile application is provided to the mobile digital processing device at the time of manufacture, in other embodiments the mobile application is provided to the mobile digital processing device via a computer network as described herein.

[0147] In light of the disclosure provided herein, mobile applications are created by techniques known to those skilled in the art using hardware, languages, and development environments known in the art. Those skilled in the art will recognize that mobile applications are written in a variety of languages. Suitable programming languages ​​include, but are not limited to, C, C++, C#, Objective-C, Java™, Javascript, Pascal, Object Pascal, Python™, Ruby, VB.NET, WML, and XHTML / HTML with or without CSS, or combinations thereof.

[0148] Suitable mobile application development environments are available from a variety of sources. Commercially available development environments include, but are not limited to, Airplay SDK, alcheMo, Appcelerator®, Celsius, Bedrock, Flash Lite, .NET Compact Framework, Rhomobile, and WorkLight Mobile Platform. Other development environments are available at no cost and include, but are not limited to, Lazarus, MobiFlex, MoSync, and Phonegap. Mobile device manufacturers also distribute software development kits, including, but not limited to, iPhone® and iPad® (iOS) SDK, Android™ SDK, BlackBerry® SDK, BREW SDK, Palm® OS SDK, Symbian SDK, webOS SDK, and Windows® Mobile SDK.

[0149] Those skilled in the art will recognize that various commercial forums are available for the distribution of mobile applications, including, but not limited to, the Apple® App Store, Google® Play, Chrome WebStore, BlackBerry® App World, App Store for Palm devices, App Catalog for webOS, Windows® Marketplace for Mobile, Ovi Store for Nokia® devices, Samsung® App, and Nintendo® DSi Shop.

[0150] Standalone Applications In some embodiments, the computer program comprises a standalone application, which is a program that runs as an independent computer process, rather than an add-on, e.g., a plug-in, to an existing process. Those skilled in the art will recognize that standalone applications are frequently compiled. A compiler is a computer program that converts source code written in a programming language into binary object code, such as assembly language or machine code. Suitable compiled programming languages ​​include, but are not limited to, C, C++, Objective-C, COBOL, Delphi, Eiffel, Java™, Lisp, Python™, Visual Basic, and VB .NET, or combinations thereof. Compilation is often performed at least in part to create an executable program. In some embodiments, the computer program comprises one or more executable compiled applications.

[0151] Web Browser Plug-ins In some embodiments, the computer program comprises a web browser plug-in (e.g., an extension). In computing, a plug-in is one or more software components that add specific functionality to a larger software application. Software application manufacturers support plug-ins to allow third-party developers to create capabilities that extend the application, facilitate easy addition of new features, and reduce the size of the application. When supported, plug-ins allow customization of the functionality of the software application. For example, plug-ins are commonly used in web browsers to play video, generate dialogue, scan for viruses, and display specific file types. Those skilled in the art are familiar with various web browser plug-ins, including Adobe® Flash® Player, Microsoft® Silverlight®, and Apple® QuickTime®.

[0152] Given the disclosure provided herein, one of skill in the art will recognize that a variety of plugin frameworks are available that allow for the development of plugins in a variety of programming languages, including but not limited to C++, Delphi, Java™, PHP, Python™, and VB.NET.

[0153] A web browser (also called an Internet browser) is a software application designed for use on networked digital processing devices to search, present, and traverse information resources on the World Wide Web. Suitable web browsers include, but are not limited to, Microsoft® Internet Explorer®, Mozilla® Firefox®, Google® Chrome, Apple® Safari®, Opera Software® Opera®, and KDE Konqueror. In some embodiments, the web browser is a mobile web browser. Mobile web browsers (also called microbrowsers, minibrowsers, and wireless browsers) are designed for use on mobile digital processing devices, including, but not limited to, handheld computers, tablet computers, netbook computers, subnotebook computers, smartphones, music players, personal digital assistants (PDAs), and handheld video game systems. Suitable mobile web browsers include, but are not limited to, Google® Android® browser, RIM BlackBerry® browser, Apple® Safari®, Palm® Blazer, Palm® WebOS® Browser, Mozilla® Firefox® for mobile, Microsoft® Internet Explorer® Mobile, Amazon® Kindle® Basic Web, Nokia® browser, Opera Software® Opera® Mobile, and Sony® PSP™ browser.

[0154] Software Module In some embodiments, the platforms, systems, media, and methods disclosed herein include software, server, and / or database modules, or the use thereof. In light of the disclosure provided herein, the software modules are created by techniques known to those skilled in the art using machines, software, and languages ​​known in the art. The software modules disclosed herein are implemented in a number of ways. In various embodiments, the software modules include files, sections of code, programming objects, programming structures, or combinations thereof. In further various embodiments, the software modules include multiple files, multiple sections of code, multiple programming objects, multiple programming structures, or combinations thereof. In various embodiments, the one or more software modules include, but are not limited to, web applications, mobile applications, and stand-alone applications. In some embodiments, the software modules are in one computer program or application. In other embodiments, the software modules are in more than one computer program or application. In some embodiments, the software modules are hosted on one machine. In other embodiments, the software modules are hosted on more than one machine. In further embodiments, the software modules are hosted on a cloud computing platform. In some embodiments, the software modules are hosted on one or more machines in one location. In other embodiments, the software modules are hosted on one or more machines in more than one location.

[0155] Database In some embodiments, the platforms, systems, media, and methods described herein include one or more databases, or the use thereof. In various embodiments, suitable databases include, but are not limited to, relational databases, non-relational databases, object-oriented databases, object databases, entity-relationship model databases, associative databases, and XML databases. Further non-limiting examples include SQL, PostgreSQL, MySQL, Oracle, DB2, and Sybase. In some embodiments, the database is Internet-based. In further embodiments, the database is web-based. In still further embodiments, the database is cloud computing-based. In other embodiments, the database is based on one or more local computer storage devices.

[0156] A platform for assessing behavioral disorders, developmental delays, and nervous system dysfunction In some embodiments herein, a platform for assessing behavioral disorders, developmental delays, and nervous system dysfunction is disclosed, the platform comprising one or more computing devices, each comprising an application that allows communication and / or sharing of data between the one or more computing devices. In some embodiments, the application provides users with specialized portals, such as, for example, a healthcare provider portal and a patient portal. Features provided by the applications on the platform described herein include recording individuals and assessing individuals using the techniques described herein.

[0157] In some embodiments, a user records video of the individual being assessed through use of a recording application in a first user application on a first computing device. In some embodiments, the user application provides instructions to the user as to the type and length of recording. In some embodiments, the recording is a record of the individual's movements, responses to questions or requests, eating, emotional expression, sleeping, playing, talking, watching (e.g., television), responses to or progress on their work.

[0158] In some embodiments, the records analyzed by the machine learning software module provide the clinician, through use of the clinician application, with a probability score for each possible diagnosis. In some embodiments, the probability score must exceed a threshold number to be displayed in the clinician application. In some embodiments, probability scores below a certain threshold are displayed in a separate tab or screen in the clinician application. In some embodiments, the clinician reviews the analysis results and requests additional data through the clinician application. For example, the clinician may receive results showing that a child has a probability score of 35% for a form of autism, 48% for a form of mental retardation, and 10% for a speech disorder. The probability score threshold is set to 25% and the clinician reviews the scores for autism and mental retardation. The clinician orders behavioral tests through the application and requests additional footage of the child performing a task showing one or both instructions. In some embodiments, the clinician diagnoses the individual with the aid of the results provided by the machine learning algorithm. In some embodiments, the clinician enters the diagnosis into the application and the data is made available to medical personnel.

[0159] In some embodiments, the healthcare provider is able to coordinate the individual's treatment and provide treatment advice to the user and the individual, hi some embodiments, clinicians treating the individual in the healthcare provider's network are able to access records and diagnostic tests. EXAMPLES

[0160] Example 1 A parent is concerned that their 2-year-old child is missing several developmental milestones and contacts a clinician to have the child evaluated for developmental delay. The clinician asks the parent to download an application on their smartphone to videotape the child responding to questions, playing with an adult, and playing independently. The application provides guidance for filming the child regarding filming angles, filming distance, and lighting considerations. The parent films the video through the application and enters personal information about the child, such as gender, date of birth, and the type of activity being filmed. The footage is sent to the clinician and automatically analyzed by one or more machine learning algorithms. The clinician receives a likelihood score for a diagnosis of developmental delay. The clinician reviews the higher-order behaviors and behavioral patterns detected by the one or more machine learning algorithms to ensure that the data used to make the diagnosis are clinically relevant. The clinician may order additional testing to confirm the proposed diagnosis. The clinician and parent meet to discuss the diagnosis and treatment. The clinician can point out specific behaviors in the footage and show the parent the behaviors he or she has observed. The clinician provides treatment recommendations and requests that images be taken at intervals to monitor the child's progress and the effectiveness of the proposed treatment. If progress is not seen, the clinician may modify treatment. The clinician may instead receive a proposed diagnosis after subsequent video analysis and prescribe additional tests or treatment.

[0161] Example 2 A child exhibits many behaviors that are indicative of both autism and ADHD. One or more machine learning algorithms described herein analyze the video recording of the child. The machine learning algorithms are capable of recognizing behavioral units, higher order behaviors, and behavioral patterns that are specific to each of the two diagnoses but difficult to recognize with the human eye. The one or more machine learning algorithms provide a probability score for each of the two diagnoses and find that the child is highly likely to have ADHD and low likely to have autism.

[0162] Example 3 A parent is concerned that their 2-year-old child is missing several developmental milestones and contacts a clinician to have the child evaluated for developmental delays. The clinician asks the parent to download an application onto their smartphone and help the child immerse themselves in interactive activities provided within the application. The interactive activities provide a digital therapy in the form of continuous assessment of the child based on input data collected from the child's immersion while promoting and improving the child's speech and language development. The application provides a virtual character that provides guidance on the activities and engages the child by asking questions or giving prompts to elicit feedback or responses from the child.

[0163] The application performs an automated in-home assessment of young children's speech and language development by providing an interactive module with a variety of interactive digital activities for the child to immerse themselves in. A parent launches the software application on a tablet and selects a storybook reading activity for their child. The storybook reading activity provides a digital character to provide guidance (see FIG. 5A). The child is presented with a story sequence and its accompanying corresponding text (see FIG. 5A). After the story sequence is completed, the digital character asks the child questions related to the story sequence to assess the child's comprehension of the story (FIG. 5C).

[0164] The digital character then asks the child to tell the story to the parent, thereby testing the child's ability to retell the story (see Figure 6A). A visual sequence from the story is shown in Figure 6B. The child's retelling of the story is analyzed to determine proficiency in retelling the sequence. The digital character can further ask the child to come up with a story based on the picture prompts and evaluate the child's responses to determine proficiency in providing a picture-elicited story (see Figures 7A and 7B).

[0165] Additional activities provided by the interactive module may include coloring in colorless pictures or images, manipulation of the story sequence (e.g., placing the sequence in the correct numerical order or alternatively coming up with a new sequence order; see FIG. 8), and / or individualization of the story by allowing the child to create his or her own story by manipulation and / or rearrangement of the story sequence, characters, objects, colors, or other visual elements (see FIG. 9). Individualized stories may include rearrangement in addition to manipulation of the audio sequence, e.g., matching an audio sequence or clip to a visual story sequence. As an example, the child is provided with a graphic user interface via the application with selectable interactive buttons to add, delete, scale, move, color, or edit various virtual objects (e.g., characters, animals, tools, clothing, vehicles, buildings, and other objects). Text or dialogue can also be inserted along with corresponding audio clips.

[0166] Additional possibilities offered by interactive modules may include embedded intervention via social stories (see Figure 10) or game-based life skills training (see Figure 11).

[0167] While preferred embodiments of the present invention have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Numerous variations, changes, and substitutions will now occur to those skilled in the art without departing from the invention. It will be understood that various alternatives to the embodiments of the invention described herein are available for practicing the invention. It is intended that the following claims define the scope of the invention, and that methods and structures within the scope of these claims and their equivalents are covered thereby.

Claims

1. One or more computer processors; a computer readable memory containing machine readable code that, when executed by the one or more computer processors, performs a method for automated video assessment of an individual; and The method includes: (a) receiving, with a computing device, input data including at least one of audio or video information for the individual for whom the automated video assessment is to be performed, the input data being received in conjunction with the provision of a storytelling activity configured to elicit a plurality of behavioral units of the individual using a story told through one or more images shown on a display of the computing device; (b) identifying, using the computing device, the plurality of behavioral units in the input data, each behavioral unit of the plurality of behavioral units comprising a behavior that constitutes a higher order behavior; (c) using the computing device, identifying the higher order behavior based on at least one behavior unit of the plurality of behavior units; Including, the system.

2. The system of claim 1 , wherein at least one of the plurality of behavioral units comprises a machine-detectable movement or sound made by the individual.

3. The system of claim 1 , wherein at least one of the plurality of behavior units comprises a facial movement, a body movement, or a sound made by the individual.

4. The system of claim 1 , wherein at least one of the higher order behaviors includes verbal or non-verbal communication.

5. The system of claim 4 , wherein the non-verbal communication includes facial expressions, posture, gestures, eye contact, or touch.

6. The system of claim 1 , wherein identifying the plurality of behavioral units comprises analyzing the video information using facial recognition to detect one or more facial movements.

7. The system of claim 1 , wherein identifying the plurality of behavioral units comprises analyzing the audio information using speech recognition to detect one or more sounds, words, or phrases.

8. The system of claim 1 , wherein identifying the higher order behavior comprises generating a timeline of a plurality of behavioral units.

9. The system described in claim 8, wherein the method further includes a step of generating a behavior pattern based on the timeline or the higher-level behavior of the plurality of behavioral units.

10. The system of claim 1 , wherein the input data further comprises information or responses provided by a caregiver of the individual.

11. The system of claim 10 , wherein the information or response comprises an answer to a question.

12. The system of claim 1, wherein the method further comprises generating a prediction including a positive, negative, or indeterminate classification for behavioral disorder, developmental delay, or neurological dysfunction.

13. The system of claim 1, wherein the method further comprises obtaining at least one of the audio or video information via a mobile computing device.

14. The system of claim 13 , wherein the mobile computing device comprises a smartphone, a tablet computer, a laptop, a smart watch, or other wearable computing device.

15. 14. The system of claim 13, wherein obtaining at least one of the audio or video information comprises capturing video footage or audio recordings of the individual or interactions of the individual with others.

16. 13. The system of claim 12, wherein the behavioral disorder, developmental delay, or nervous system dysfunction comprises Pervasive Developmental Disorder (PDD), Autism Spectrum Disorder (ASD), Social Communication Disorder, Restricted Repetitive Behaviors, Interests, and Activities (RRB), Autism ("Classic Autism"), Asperger's Syndrome ("High Functioning Autism"), Pervasive Developmental Disorder Not Otherwise Specified (PDD-NOS, "Atypical Autism"), Attention Deficit Disorder (ADD), Attention Deficit Hyperactivity Disorder (ADHD), Speech and Language Delay, Obsessive-Compulsive Disorder (OCD), Depression, Schizophrenia, Alzheimer's Disease, Dementia, Intellectual Disability, or Learning Disability.

17. 17. The system of claim 16, wherein the behavioral disorder, developmental delay, or nervous system dysfunction is an autism spectrum disorder or autism.

18. The system of claim 1 , wherein identifying the plurality of behavioral units in the input data and / or identifying the higher order behaviors is performed using a machine learning software module.

19. 20. The system of claim 18, wherein the machine learning algorithm of the machine learning software module is a supervised learning algorithm.

20. 20. The system of claim 18, wherein the machine learning software module is selected from nearest neighbor, naive Bayes, decision trees, linear regression, support vector machines, or neural networks.

21. The system of claim 1, wherein the method further includes providing an activity configured to elicit a response by the individual, the response including the input data including at least one of the audio information or video information.

22. 22. The system of claim 21, wherein the activity provides a virtual character that guides the individual through the activity.

23. 22. The system of claim 21, wherein the higher order behaviors or behavioral patterns of the individual include speech articulation, fluency, phonation, ability to understand and decode language, ability to produce and use language, or any combination thereof.

24. 22. The system of claim 21, wherein identifying the plurality of behavioral units comprises analyzing the audio information using speech recognition to detect one or more sounds, words, or phrases.

25. The system of claim 21, wherein the method further comprises adjusting the activity based on the higher-order behavior identified in the individual.

26. 26. The system of claim 25, wherein the activity is dynamically adjusted while the individual is engaged in the activity.

Images