Platforms and systems for digital personalized medicine

The digital personalized health system addresses the inefficiencies in conventional diagnosis and treatment by using AI and machine learning to analyze diverse patient data, offering personalized and efficient care for cognitive and mental health disorders, reducing assessment time and costs.

JP7584388B2Active Publication Date: 2024-11-15COGNOA INC
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2021172257
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2015-12-18
Filing Date
2021-10-21
Publication Date
2024-11-15
Estimated Expiration
2036-12-16

AI Technical Summary

Technical Problem

Conventional methods for digital patient diagnosis and treatment are less than ideal in integrating digital data with patient treatment, often failing to provide personalized and efficient care for cognitive and developmental disorders, neurodegenerative diseases, and mental health issues, leading to increased healthcare costs and burdens.

Method used

A digital personalized health system using machine learning, artificial intelligence, and statistical modeling to analyze patient data from various sources, including mobile devices and wearable monitors, to provide personalized diagnostic and therapeutic interventions, reducing the number of questions and observations needed for accurate diagnosis and treatment.

Benefits of technology

The system improves accuracy and efficiency in diagnosing and treating cognitive and mental health disorders by personalizing treatment plans, reducing assessment time and costs, and providing real-time monitoring and feedback to adapt interventions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007584388000001
    Figure 0007584388000001
  • Figure 0007584388000002
    Figure 0007584388000002
  • Figure 0007584388000003
    Figure 0007584388000003
Patent Text Reader

Abstract

Provides a platform and system for digital personalized medicine. A digital personalized medicine system that provides digital diagnostics and digital therapeutics to patients uses digital data to assess or diagnose a patient's condition, and feedback from the patient's response to treatment updates personalized therapeutic interventions. The method also diagnoses and treats a subject's cognitive function and determines multiple behavioral, neurological, or mental health disorders with fewer questions and in less time, providing clinically acceptable sensitivity and specificity in diagnosis and treatment.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] cross reference This application claims priority to U.S. Provisional Patent Application No. 62 / 269,638, entitled “PLATFORM AND SYSTEM FOR DIGITAL PERSONALIZED MEDICINE,” filed December 18, 2015 (Attorney Docket No. 46173-703.101), the entire disclosure of which is incorporated herein by reference. [Background technology]

[0002] 2. Background of the Invention Conventional methods and devices for digital diagnosis and treatment of patients are less than ideal in at least some respects. Digital data can be obtained from a patient in many ways, but the integration of this digital data with patient treatment is less than ideal. For example, simply recording a patient's activities and suggesting activities according to a predefined treatment plan may not provide the patient with the best treatment.

[0003] Although digital diagnostics using machine learning have been proposed, the integration of digital diagnostics with patient treatment may not be ideal: for example, the classifiers used to diagnose the patient may not be ideally suited to most effectively match the treatment with the diagnosis or to monitor the treatment.

[0004] Conventional methods and devices for diagnosing and treating cognitive function of people, such as people suffering from developmental disorders, may be less than ideal in at least some respects.Unfortunately, less than ideal time, energy and cost may be required to diagnose and treat and determine whether a subject is at risk of cognitive decline, such as dementia, Alzheimer's disease, or developmental disorders.Examples of cognitive and developmental disorders that are less than ideally treated by conventional approaches include, for example, autism, autism spectrum, attention deficit disorder, attention deficit hyperactivity disorder, and speech and learning disorders.Examples of mood and psychotic disorders that are less than ideally treated by conventional approaches include depression, anxiety, ADHD, obsessive-compulsive disorder, and substance disorders, such as drug abuse and eating disorders. Conventional approaches to the diagnosis and treatment of some neurodegenerative diseases may be less than ideal in many cases, including, for example, age-related cognitive decline, cognitive impairment, Alzheimer's disease, Parkinson's disease, Huntington's chorea, and amyotrophic lateral sclerosis ("ALS"). Healthcare systems are under increasing pressure to provide care at lower costs, and conventional methods and devices for clinically diagnosing or identifying subjects as at risk for developmental disorders may result in greater than ideal costs and burdens on the health care system. Furthermore, at least some subjects are not treated as soon as they ideally arise, and thus the burden on the health care system increases with the additional care required for these subjects.

[0005] Identifying and treating cognitive disorders in subjects can pose difficult technical problems with both accuracy and efficiency. Many conventional methods for identifying and treating such disorders are often time-consuming and resource-intensive, requiring subjects to answer numerous questions or undergo extensive observation under the supervision of qualified clinicians, whose number and availability may be limited depending on the subject's geographic location. In addition, many conventional methods for identifying and treating behavioral, neurological or mental health disorders have less than ideal accuracy and consistency, as subjects to be assessed using such methods present a wide range of variation that may be difficult to capture and classify. A technical solution to such technical problems would be desirable, which could improve both accuracy and efficiency for diagnosis and treatment. Ideally, such a technical solution would reduce the time and resources required to implement a method for identifying and treating cognitive function attributes, such as behavioral, neurological or mental health disorders, and improve the accuracy and consistency of subject identification outcomes.

[0006] Furthermore, while traditional lengthy tests with questions can be administered to caregivers, such as parents, to diagnose or identify subjects as at risk for developmental disorders, these tests can be very lengthy and laborious. For example, at least some of these tests have more than 100 questions, and multiple such lengthy tests may be administered, further increasing the burden on health care providers and caregivers. Additional data, such as clinical observations of the subject, may be required, and clinical visits may further increase the time and burden on the health care system. As a result, the time between a subject being identified as needing to be assessed and being clinically identified as at risk for developmental delay or diagnosed as suffering from developmental delay may be several months, and in some cases, more than a year.

[0007] It would also be useful if diagnostic methods and treatments could be administered to subjects with high, normal, and impaired cognitive function to promote cognitive function.

[0008] In light of the above, there is a need for improved digital therapeutics for patients. Ideally, such digital therapeutics would provide a customized treatment plan for the patient and receive updated diagnostic data according to the customized treatment plan to determine progress and update the treatment plan accordingly. There is also a need for improved methods and devices for diagnosing, treating, and identifying at-risk subjects. Ideally, such methods and devices would provide clinically acceptable sensitivity and specificity in clinical or non-clinical environments that could be used to monitor patients with fewer questions and less time, and to monitor and adapt treatment effectiveness. Ideally, such methods and devices could also be used to determine the subject's developmental progression and suggest treatments to promote developmental progression. Summary of the Invention [Means for solving the problem]

[0009] The digital personalized medicine systems and methods described herein provide digital diagnosis and digital therapeutics for patients, using digital data to assess or diagnose patient conditions in a manner that informs personalized or more appropriate therapeutic interventions and improved diagnoses.

[0010] In one aspect, the digital personalized medicine system includes a digital device having a processor and associated software configured to use the data to evaluate and diagnose a patient, incorporate interaction and feedback data that identifies relative levels of efficacy, compliance, and response resulting from a therapeutic intervention, and perform data analytics, including at least one of machine learning, artificial intelligence, and statistical models, to evaluate user data and user profiles and further personalize, improve, or evaluate the effectiveness of the therapeutic intervention.

[0011] In some cases, the system is configured to use digital diagnostics and therapeutics, which may include systems or methods for digitally collecting information and processing and analyzing the provided data to improve an individual's medical, psychological, or physiological condition. Digital therapeutics systems may apply software-based learning to analyze user data and monitor and improve diagnostic and therapeutic interventions provided by the system.

[0012] The digital diagnostic data in the system can include data and metadata collected from the patient, or caregiver, or a party unrelated to the individual being evaluated. In some cases, the collected data can include monitoring behavior, and the observations, judgments, or evaluations can be made by a party other than the individual. In further cases, the evaluation can include an adult performing the evaluation or providing data for the evaluation of a child or minor. The data and metadata can be in digital form, either active or passive, via one or more digital devices, such as a mobile phone, video capture, audio capture, activity monitor, or wearable digital monitor.

[0013] Digital diagnostics uses data collected by the system about the patient, which may include complementary diagnostic data captured outside the digital diagnostics, along with analytical results from tools such as machine learning, artificial intelligence, and statistical modeling to assess or diagnose the patient's condition. Digital diagnostics also provide an assessment of changes in the patient's condition or performance, directly or indirectly through the data and metadata, which may be analyzed by tools such as machine learning, artificial intelligence, and statistical modeling, thereby providing feedback to the system to improve or refine the diagnosis and potential therapeutic interventions.

[0014] Data evaluation and machine learning from digital diagnoses and corresponding responses, or lack thereof, from therapeutic interventions can result in the identification of novel diagnoses for patients and novel treatment regimens for both patients and caregivers.

[0015] The types of data collected and utilized by the system can include, for example, patient and caregiver video, audio, responses to questions or activities, as well as active or passive data streams from user interactions with the system's activities, games, or software features. Such data can also include metadata from patient or caregiver interactions with the system, for example, when performing recommended activities. Examples of specific metadata include data from user interactions with the system's devices or mobile apps that capture aspects of user behavior, profile, activities, interactions with software systems, interactions with games, frequency of use, session times, selected options or features, and content and activity preferences. Data can also include data and metadata from various third party devices, such as activity monitors, games, or interactive content.

[0016] Digital therapeutics may include instructions, feedback, activities, or interactions provided by the system to the patient or caregiver. Examples include suggested actions, activities, games, or interactive sessions by the system software and / or third-party devices.

[0017] In further embodiments, the digital therapeutics method and apparatus disclosed herein can diagnose, treat, and provide treatment recommendations to a subject in a clinical or non-clinical setting as at risk for having one or more behavioral, neurological, or mental health disorders, with fewer questions, less time, and with clinically acceptable sensitivity and specificity in a clinical environment. This can be useful, for example, when a subject begins treatment based on an incorrect diagnosis. The processor can be configured with instructions to identify the most predictive next question or the most informative next symptom or observation, so that the person can be diagnosed or identified as at risk and treated with fewer questions or observations. Identifying the most predictive next question or the most informative next symptom or observation in response to multiple answers has the advantage of increasing sensitivity and specificity and providing treatment with fewer questions. In some cases, an additional processor can be provided to predict or collect information about the next more relevant symptoms. The methods and devices disclosed herein can be configured to assess and treat a subject for multiple related disorders using a single test, and diagnose or determine that a subject is at risk for one or more of multiple disorders using a single test. Reducing the number of questions posed or symptoms or measurements used can be particularly useful when a subject presents multiple possible disorders that can be treated. Using only a single test to assess a subject for multiple possible disorders can greatly reduce the length and cost of the assessment procedure and improve treatment. The methods and devices disclosed herein can diagnose and treat a subject at risk for having a single disorder of multiple possible disorders that may have overlapping symptoms.

[0018] The most predictive next question, most informative next symptom or observation to be used in a digital therapeutic treatment can be determined in many ways, but in many cases, the most predictive next question, symptom or observation is determined in response to multiple answers to a previous question or observation, which may include a previous most predictive next question, symptom or observation to assess the treatment and perform a closed-loop evaluation of the subject. The most predictive next question, symptom or observation may be determined statistically, and a set of possible most predictive next questions, symptoms or observations may be assessed to determine the most predictive next question, symptom or observation. In many cases, the observation or answer to each possible most predictive next question is related to the relevance of the question or observation, and the relevance of the question or observation may be determined in response to the combined feature weight of each possible answer to the question or observation. After treatment is initiated, the question, symptom or different question, symptom or observation may be used to more accurately monitor progress and suggest changes to the digital treatment. The relevance of the next question, symptom, or observation may also depend on the likely variance in the final ratings among the different answer options for the question or potential options for the observation. For example, a question in which the answer options may significantly affect the final rating down the line may be considered more relevant than a question in which the answer options may only serve to discriminate between differences in severity for one particular condition or are otherwise less consequential.

[0019] Aspects of the present disclosure provide a digital therapeutic system for treating a subject with a personal therapeutic treatment plan. An exemplary system may include one or more processors including software instructions for a diagnostic module and a therapeutic module. The diagnostic module may receive data from a subject and output diagnostic data for the subject. The diagnostic module may include one or more of machine learning, classifiers, artificial intelligence, or statistical modeling based on a subject population to determine diagnostic data for the subject. The therapeutic module may receive diagnostic data and output a personal therapeutic treatment plan for the subject. The therapeutic module may include one or more of machine learning, classifiers, artificial intelligence, or statistical modeling based on at least a portion of a subject population to determine and output a personal therapeutic treatment plan for the subject. The diagnostic module may be configured to receive updated subject data from the subject in response to treatment of the subject and generate updated diagnostic data from the subject. The therapeutic module may be configured to receive updated diagnostic data and output an updated personal therapeutic treatment plan for the subject in response to the diagnostic data and the updated diagnostic data.

[0020] In some embodiments, the diagnostic module includes a diagnostic machine learning classifier trained on a subject population and the therapeutic module includes a therapeutic machine learning classifier trained on at least a portion of the subject population. The diagnostic module and the therapeutic module may be arranged such that the diagnostic module provides feedback to the therapeutic module based on execution of a treatment plan. The therapeutic classifier may include instructions trained on a dataset that includes a population of which the subject is not a member. The subject may include a person who is not a member of a population.

[0021] In some embodiments, the diagnostic module includes a diagnostic classifier trained on a plurality of profiles of a subject population of at least 10,000 people and a therapeutic profile trained on a plurality of profiles of the subject population.

[0022] Aspects of the present disclosure also provide a digital personalized treatment system. An exemplary system may include (i) software and digital devices that use data to evaluate and diagnose subjects, (ii) software and digital devices that capture interaction and feedback data that identifies the relative levels of efficacy, compliance, and response resulting from therapeutic intervention, and (iii) data analytics capabilities, including machine learning, AI, and statistical models, that evaluate user data and user profiles and further personalize, improve, or evaluate the efficacy of therapeutic intervention.

[0023] In some embodiments, the system includes software-based learning capabilities that enable the system to use its user data to monitor and improve diagnostic and therapeutic interventions.

[0024] In some embodiments, the digital diagnostics in the system include data and metadata collected from the subject or caregiver in one or more of active or passive digital formats via different digital devices, such as a cell phone, video capture, audio capture, activity monitor, or wearable digital monitor.

[0025] In some embodiments, a digital diagnostic uses data collected about a subject by the system along with analytical results from tools such as machine learning, artificial intelligence, and statistical modeling to assess or diagnose a subject's condition, with or without complementary diagnostic data captured outside the digital diagnostic.

[0026] In some embodiments, the digital diagnostic further enables assessment of changes in a subject's condition or performance, directly or indirectly, via data and metadata that can be analyzed by tools such as machine learning, artificial intelligence, and statistical modeling, thereby providing feedback to the system to improve or refine diagnosis and potential therapeutic interventions.

[0027] In some embodiments, data evaluation and machine learning from the digital diagnosis and the corresponding response, or lack thereof, from the therapeutic intervention is configured to identify novel diagnoses for subjects and novel treatment regimens for both patients and caregivers.

[0028] In some embodiments, the types of data collected and utilized by the system include subject and caregiver video, audio, responses to questions or activities, as well as active or passive data streams from user interactions with the system's activities, games, or software features.

[0029] In some embodiments, the metadata includes data from a user's interactions with a system's devices or mobile apps that capture a profile of one or more of the user's behavior, profile, activity, interactions with a software system, interactions with a game, frequency of use, session times, selected options or features, content or activity preferences.

[0030] In some embodiments, the data includes data and metadata from various third party devices, such as activity monitors, games, or interactive content.

[0031] In some embodiments, the digital therapeutic comprises instructions, feedback, activities, or interactions provided to a subject or caregiver by a mobile device.

[0032] In some embodiments, the system includes instructions for providing a suggested action, activity, game, or interactive session by the system software and / or a third party device.

[0033] In some embodiments, the system includes instructions for diagnosing and treating one or more of cognitive and behavioral disorders or conditions, including cognitive or behavioral development, neurodegenerative conditions, and mood disorders.

[0034] Aspects of the present disclosure also provide a system for diagnosing and treating a subject. An exemplary system may include a diagnostic module for receiving subject data and outputting diagnostic data for the subject, and a therapeutic module for receiving the diagnostic data and outputting a therapeutic treatment for the subject, where the diagnostic module and the therapeutic module are arranged to include a feedback loop to update the treatment in response to the diagnostic data.

[0035] Aspects of the present disclosure also provide a mobile device for delivering digital personalized treatment. An exemplary mobile device may include a display and a processor configured with instructions to generate a user profile in response to a user's interaction with the device, receive and display treatment instructions to the user in response to the user profile, update the user profile in response to the treatment, transmit the updated user profile to a remote server, receive updated treatment instructions from the server, and display the treatment updated instructions to the user.

[0036] Aspects of the present disclosure also provide a digital therapeutic system for treating a subject with an individualized therapeutic treatment plan. An exemplary system may include a processor including instructions for a diagnostic module to receive data from a subject and output diagnostic data for the subject, and a therapeutic module to receive the diagnostic data and output an individualized therapeutic treatment plan for the subject. The individualized therapeutic treatment plan may include a digital therapeutic.

[0037] In some embodiments, the digital therapeutic comprises instructions, feedback, activities, or interactions provided to a subject or a caregiver. The digital therapeutic may be implemented on a mobile device.

[0038] In some embodiments, the diagnostic data and the personalized therapeutic treatment plan are provided to a third party system, which may include a healthcare professional's computer system or a therapeutic delivery system.

[0039] In some embodiments, the diagnostic module is configured to receive updated subject data from the subject in response to the subject's feedback data and generate updated diagnostic data. The therapeutic module may be configured to receive the updated diagnostic data and output an updated individualized treatment plan for the subject in response to the diagnostic data and the updated diagnostic data. The updated subject data is received in response to feedback data identifying the relative levels of efficacy, compliance, and response resulting from the individualized therapeutic treatment plan.

[0040] In some embodiments, the diagnostic module includes machine learning, classifiers, artificial intelligence, or statistical modeling based on the subject population to determine diagnostic data, and the therapeutic module includes machine learning, classifiers, artificial intelligence, or statistical modeling based on at least a portion of the subject population to determine an individualized therapeutic treatment plan for the subject.

[0041] In some embodiments, the diagnostic module includes a diagnostic machine learning classifier trained on the subject population. The therapeutic module may include a therapeutic machine learning classifier trained on at least a portion of the subject population. The diagnostic module may be configured to provide feedback to the therapeutic module based on execution of the individualized therapeutic treatment plan.

[0042] In some embodiments, the data from the subject includes at least one of subject and caregiver video, audio, responses to questions or activities, and active or passive data streams from user interaction with system activities, games, or software features.

[0043] In some embodiments, the subject is at risk for a disorder selected from the group consisting of a behavioral disorder, a neurological disorder, and a mental health disorder. The behavioral, neurological, or mental health disorder may be selected from the group consisting of autism, autism spectrum, attention deficit disorder, depression, obsessive-compulsive disorder, schizophrenia, Alzheimer's disease, dementia, attention deficit hyperactivity disorder, and speech and learning disabilities.

[0044] In some embodiments, the diagnostic module is configured for an adult to perform the evaluation or to provide data for the evaluation of a child or minor.

[0045] In some embodiments, the diagnostic module is configured to allow a caregiver or family member to perform the evaluation or provide data for the evaluation of a subject.

[0046] Aspects of the present disclosure also provide a method of treating a subject with an individualized therapeutic treatment plan. An exemplary method may include a diagnostic process that receives data from the subject and outputs diagnostic data for the subject, and a therapeutic process that receives the diagnostic data and outputs an individualized therapeutic treatment plan for the subject. The individualized therapeutic treatment plan may include a digital therapeutic.

[0047] In some embodiments, the digital therapeutic comprises instructions, feedback, activities, or interactions provided to a subject or a caregiver. The digital therapeutic may be implemented on a mobile device.

[0048] In some embodiments, the method may further include providing the diagnostic data and the personalized therapeutic treatment plan to a third party system, which may include a healthcare professional's computer system or a therapeutic delivery system.

[0049] In some embodiments, the diagnostic process further includes receiving updated subject data from the subject in response to the subject's feedback data and generating updated diagnostic data, and the therapeutic process further includes receiving the updated diagnostic data and outputting an updated individualized treatment plan for the subject in response to the diagnostic data and the updated diagnostic data. The updated subject data may be received in response to feedback data identifying relative levels of efficacy, compliance, and response resulting from the individualized therapeutic treatment plan.

[0050] In some embodiments, the diagnostic process is performed by a process selected from the group consisting of machine learning, classifiers, artificial intelligence, and statistical modeling based on the subject population to determine diagnostic data. The therapeutic process may be performed by a process selected from the group consisting of machine learning, classifiers, artificial intelligence, or statistical modeling based on at least a portion of the subject population to determine an individualized therapeutic treatment plan for the subject.

[0051] In some embodiments, the diagnostic process is performed by a diagnostic machine learning classifier trained on the subject population. The therapeutic process may be performed by a therapeutic machine learning classifier trained on at least a portion of the subject population. The diagnostic process may further include providing feedback to a therapeutic module based on execution of the individualized therapeutic treatment plan.

[0052] In some embodiments, the data from the subject includes at least one of subject and caregiver video, audio, responses to questions or activities, and active or passive data streams from user interactions with activities, games, or software features.

[0053] In some embodiments, the subject is at risk for a behavioral disorder, a neurological disorder, and a mental health disorder. The risk may be selected from the group consisting of autism, autism spectrum, attention deficit disorder, depression, obsessive-compulsive disorder, schizophrenia, Alzheimer's disease, dementia, attention deficit hyperactivity disorder, and speech and learning disabilities. The diagnostic process may be performed by an adult performing the assessment or providing data for the assessment of a child or minor. The diagnostic process may allow a caregiver or family member to perform the assessment or provide data for the subject's assessment.

[0054] Another aspect of the present disclosure provides a therapeutic system for treating a subject with an individualized therapeutic treatment plan. An exemplary system may include a processor including software instructions for a diagnostic module for receiving data from a subject and outputting diagnostic data for the subject, and a therapeutic module for receiving the diagnostic data and outputting an individualized therapeutic treatment plan for the subject. The diagnostic module may be configured to receive updated subject data from the subject in response to treating the subject, and generate updated diagnostic data from the subject. The therapeutic module may be configured to receive the updated diagnostic data, and output an updated individualized treatment plan for the subject in response to the diagnostic data and the updated diagnostic data.

[0055] In some embodiments, the updated subject data is received in response to feedback data identifying relative levels of efficacy, compliance, and response resulting from the individualized therapeutic treatment plan.

[0056] In some embodiments, the personalized therapeutic treatment plan includes a digital therapeutic. The digital therapeutic may include instructions, feedback, activities, or interactions provided to the subject or a caregiver. The digital therapeutic may be implemented on a mobile device.

[0057] In some embodiments, the diagnostic data and the personalized therapeutic treatment plan are provided to a third party system, which may include a healthcare professional's computer system or a therapeutic delivery system.

[0058] In some embodiments, the diagnostic module includes machine learning, classifiers, artificial intelligence, or statistical modeling based on the subject population to determine diagnostic data. The therapeutic module may include machine learning, classifiers, artificial intelligence, or statistical modeling based on at least a portion of the subject population to determine an individualized therapeutic treatment plan for the subject.

[0059] In some embodiments, the diagnostic module includes a diagnostic machine learning classifier trained on the subject population. The therapeutic module may include a therapeutic machine learning classifier trained on at least a portion of the subject population. The diagnostic module may be configured to provide feedback to the therapeutic module based on execution of the individualized therapeutic treatment plan.

[0060] In some embodiments, the data from the subject includes at least one of subject and caregiver video, audio, responses to questions or activities, and active or passive data streams from user interaction with system activities, games, or software features.

[0061] In some embodiments, the diagnostic module is configured for an adult to perform the evaluation or to provide data for the evaluation of a child or minor.

[0062] In some embodiments, the diagnostic module is configured to allow a caregiver or family member to perform the evaluation or provide data for the evaluation of a subject.

[0063] In some embodiments, the subject is at risk for a behavioral, neurological, and mental health disorder selected from the group consisting of autism, autism spectrum, attention deficit disorder, depression, obsessive-compulsive disorder, schizophrenia, Alzheimer's disease, dementia, attention deficit hyperactivity disorder, and speech and learning disabilities.

[0064] Aspects of the present disclosure also provide a method of treating a subject with an individualized therapeutic treatment plan. An exemplary method may include a diagnostic process receiving data from a subject and outputting diagnostic data for the subject, and a therapeutic process receiving the diagnostic data and outputting an individualized therapeutic treatment plan for the subject. The diagnostic process may include receiving updated subject data from the subject in response to treating the subject, and generating updated diagnostic data from the subject. The therapeutic process may include receiving the updated diagnostic data, and outputting an updated individualized treatment plan for the subject in response to the diagnostic data and the updated diagnostic data.

[0065] In some embodiments, the updated subject data is received in response to feedback data identifying relative levels of efficacy, compliance, and response resulting from the individualized therapeutic treatment plan.

[0066] In some embodiments, the personalized therapeutic treatment plan includes a digital therapeutic. The digital therapeutic may include instructions, feedback, activities, or interactions provided to the subject or a caregiver. The digital therapeutic may be implemented on a mobile device.

[0067] In some embodiments, the method further includes providing the diagnostic data and the personalized therapeutic treatment plan to a third party system, which may include a healthcare professional's computer system or a therapeutic delivery system.

[0068] In some embodiments, the diagnostic process is performed by a process selected from the group consisting of machine learning, classifiers, artificial intelligence, or statistical modeling based on a subject population to determine diagnostic data. The therapeutic process may be performed by a process selected from the group consisting of machine learning, classifiers, artificial intelligence, or statistical modeling based on at least a portion of a subject population to determine an individualized therapeutic treatment plan for the subject.

[0069] In some embodiments, the diagnostic process is performed by a diagnostic machine learning classifier trained on the subject population. The therapeutic process may be performed by a therapeutic machine learning classifier trained on at least a portion of the subject population. The diagnostic process may include providing feedback to a therapeutic module based on the execution of an individualized therapeutic treatment plan.

[0070] In some embodiments, the data from the subject includes at least one of subject and caregiver video, audio, responses to questions or activities, and active or passive data streams from user interactions with activities, games, or software features.

[0071] In some embodiments, the diagnostic process is performed by an adult performing the evaluation or providing data for the evaluation of a child or minor.

[0072] In some embodiments, the diagnostic process allows a caregiver or family member to perform the evaluation or provide data for the subject's evaluation.

[0073] In some embodiments, the subject is at risk for a condition selected from the group consisting of a behavioral disorder, a neurological disorder, and a mental health disorder.

[0074] In some embodiments, the risk is selected from the group consisting of autism, autism spectrum, attention deficit disorder, depression, obsessive-compulsive disorder, schizophrenia, Alzheimer's disease, dementia, attention deficit hyperactivity disorder, and speech and learning disabilities.

[0075] Aspects of the present disclosure also provide a therapeutic system for treating a subject with an individualized therapeutic treatment plan. An exemplary system may include a processor including software instructions for a diagnostic module for receiving data from a subject and outputting diagnostic data for the subject, and a therapeutic module for receiving the diagnostic data and outputting an individualized therapeutic treatment plan for the subject. The diagnostic module may be configured to generate the diagnostic data by: (1) receiving a plurality of answers to a plurality of asked questions of a plurality of questions, the plurality of answers corresponding to a clinical characteristic of the subject related to the subject's developmental progression, and a plurality of remaining unasked questions of the plurality of questions including a most predictive next question; (2) determining the subject's developmental progression based on the plurality of answers; and (3) identifying the most predictive next question of the plurality of remaining unasked questions in response to determining the subject's developmental progression.

[0076] In some embodiments, the diagnostic module includes a pre-processing module, a training module, and a prediction module. The data processing module may extract training data from a database or a user, apply transformations to standardize the training data, and pass the standardized training data to the training module. The training module may build an evaluation model based on the standardized training data. The prediction module may generate a predicted classification of the subject.

[0077] In some embodiments, the training module utilizes machine learning algorithms to build and train the evaluation model.

[0078] In some embodiments, the prediction module is configured to generate a predicted classification of the subject by fitting new data to the assessment model, the new data being standardized by the pre-processing module. The prediction module may check whether the fitting of the new data produces a prediction of the particular disorder within a confidence interval that exceeds a threshold.

[0079] In some embodiments, the prediction module includes a question recommendation module. The question recommendation module may be configured to identify, select, or recommend a most predictive next question to ask the subject based on a plurality of answers to a plurality of asked questions to shorten the length of the evaluation. The question recommendation module may select a candidate question to recommend as a next question to be presented to the subject. The question recommendation module may assess a predicted feature importance of each one of the candidate questions. The question recommendation module may select the most predictive next question from the candidate questions based on the predicted feature importance of each one of the candidate questions. The predicted feature importance of each one of the candidate questions may be determined by a predicted feature importance determination algorithm. The evaluation model may include a random forest classifier.

[0080] In some embodiments, the personalized therapeutic treatment plan includes a digital therapeutic. The digital therapeutic may include instructions, feedback, activities, or interactions provided to the subject or a caregiver. The digital therapeutic may be implemented on a mobile device.

[0081] In some embodiments, the diagnostic data and the individualized therapeutic treatment plan are provided to a third party system, which may include a computer system of a healthcare professional.

[0082] In some embodiments, the diagnostic module is configured to receive updated subject data from the subject in response to the subject's feedback data and generate updated diagnostic data. The therapeutic module may be configured to receive the updated diagnostic data and output an updated individualized treatment plan for the subject in response to the diagnostic data and the updated diagnostic data. The updated subject data may be received in response to feedback data identifying the relative levels of efficacy, compliance, and response resulting from the individualized therapeutic treatment plan.

[0083] In some embodiments, the diagnostic module includes instructions selected from the group consisting of machine learning, classifiers, artificial intelligence, and statistical modeling based on the subject population to determine diagnostic data. The therapeutic module may include instructions selected from the group consisting of machine learning, classifiers, artificial intelligence, or statistical modeling based on at least a portion of the subject population to determine an individualized therapeutic treatment plan for the subject.

[0084] In some embodiments, the diagnostic module includes a diagnostic machine learning classifier trained on the subject population. The therapeutic module may include a therapeutic machine learning classifier trained on at least a portion of the subject population. The diagnostic module may be configured to provide feedback to the therapeutic module based on execution of the individualized therapeutic treatment plan.

[0085] In some embodiments, the data from the subject includes at least one of subject and caregiver video, audio, responses to questions or activities, and active or passive data streams from user interaction with system activities, games, or software features.

[0086] In some embodiments, the subject is at risk for a disorder selected from the group consisting of a behavioral disorder, a neurological disorder, and a mental health disorder. The risk may be selected from the group consisting of autism, autism spectrum, attention deficit disorder, depression, obsessive-compulsive disorder, schizophrenia, Alzheimer's disease, dementia, attention deficit hyperactivity disorder, and speech and learning disabilities.

[0087] In some embodiments, the diagnostic module is configured for an adult to perform the evaluation or to provide data for the evaluation of a child or minor.

[0088] In some embodiments, the diagnostic module is configured to allow a caregiver or family member to perform the evaluation or provide data for the evaluation of a subject.

[0089] Aspects of the present disclosure provide a method of treating a subject with an individualized therapeutic treatment plan. An exemplary system may include a diagnostic process that receives data from a subject and outputs diagnostic data for the subject, and a therapeutic process that receives the diagnostic data and outputs an individualized therapeutic treatment plan for the subject. The diagnostic process may include generating the diagnostic data by: (1) receiving a plurality of answers to a plurality of asked questions of a plurality of questions, the plurality of answers corresponding to a clinical characteristic of the subject related to the subject's developmental progression, and a plurality of remaining unasked questions of the plurality of questions including a most predictive next question; (2) determining the subject's developmental progression based on the plurality of answers; and (3) identifying the most predictive next question of the plurality of remaining unasked questions in response to determining the subject's developmental progression.

[0090] In some embodiments, the diagnostic process includes a pre-processing process, a training process, and a prediction process. The data processing process may extract training data from a database or a user, apply one or more transformations to standardize the training data, and pass the standardized training data to the training process. The training process may build an evaluation model based on the standardized training data. The prediction process may generate a predicted classification of the subject.

[0091] In some embodiments, the training process utilizes machine learning algorithms to build and train the evaluation model.

[0092] In some embodiments, the prediction process generates a predicted classification of the subject by fitting new data to the assessment model, where the new data is standardized by a pre-processing process. The prediction process may check whether the fitting of the new data produces a prediction of one or more particular disorders within a confidence interval that exceeds a threshold.

[0093] In some embodiments, the prediction process includes a question recommendation process. The question recommendation process may identify, select, or recommend a most predictive next question to ask the subject based on multiple answers to multiple asked questions to shorten the length of the evaluation. The question recommendation process may select one or more candidate questions to recommend as the next question to be presented to the subject. The question recommendation process may assess a predicted feature importance of each one of the candidate questions. The question recommendation process may select one or more most predictive next questions from the candidate questions based on the predicted feature importance of each one of the candidate questions. The predicted feature importance of each one of the candidate questions may be determined by a predicted feature importance determination algorithm.

[0094] In some embodiments, the evaluation process includes a random forest classifier.

[0095] In some embodiments, the personalized therapeutic treatment plan includes a digital therapeutic. The digital therapeutic includes instructions, feedback, activities, or interactions provided to the subject or a caregiver. The digital therapeutic may be implemented on a mobile device.

[0096] In some embodiments, the method may further include providing the diagnostic data and the individualized therapeutic treatment plan to a third party system. The third party system may include a computer system of a healthcare professional.

[0097] In some embodiments, the diagnostic process may include receiving updated subject data from the subject in response to the subject's feedback data and generating updated diagnostic data. The therapeutic process may include receiving the updated diagnostic data and outputting an updated individualized treatment plan for the subject in response to the diagnostic data and the updated diagnostic data. The updated subject data may be received in response to feedback data identifying the relative levels of efficacy, compliance, and response resulting from the individualized therapeutic treatment plan.

[0098] In some embodiments, the diagnostic process is performed by a process selected from the group consisting of machine learning, classifiers, artificial intelligence, or statistical modeling based on a subject population to determine diagnostic data. The therapeutic process may be performed by a process selected from the group consisting of machine learning, classifiers, artificial intelligence, or statistical modeling based on at least a portion of a subject population to determine an individualized therapeutic treatment plan for the subject.

[0099] In some embodiments, the diagnostic process is performed by a diagnostic machine learning classifier trained on the subject population. The therapeutic process may be performed by a therapeutic machine learning classifier trained on at least a portion of the subject population. The diagnostic process may include providing feedback to a therapeutic module based on the execution of an individualized therapeutic treatment plan.

[0100] In some embodiments, the data from the subject includes at least one of subject and caregiver video, audio, responses to questions or activities, and active or passive data streams from user interaction with system activities, games, or software features.

[0101] In some embodiments, the subject is at risk for a disorder selected from the group consisting of a behavioral disorder, a neurological disorder, and a mental health disorder. The risk may be selected from the group consisting of autism, autism spectrum, attention deficit disorder, depression, obsessive-compulsive disorder, schizophrenia, Alzheimer's disease, dementia, attention deficit hyperactivity disorder, and speech and learning disabilities.

[0102] In some embodiments, the diagnostic process is performed by an adult performing the evaluation or providing data for the evaluation of a child or minor.

[0103] In some embodiments, the diagnostic process allows a caregiver or family member to perform the evaluation or provide data for the subject's evaluation.

[0104] Aspects of the present disclosure provide a therapeutic system for treating a subject having two or more related behavioral, neurological or mental health disorders with an individualized therapeutic treatment plan. An exemplary system may include a processor including software instructions for a diagnostic module for receiving data from the subject and outputting diagnostic data for the subject, and a therapeutic module for receiving the diagnostic data and outputting an individualized therapeutic treatment plan for the subject. The diagnostic module may be configured to generate the diagnostic data by: (1) receiving a plurality of answers to a plurality of asked questions of the plurality of questions, the plurality of answers corresponding to a clinical characteristic of the subject related to two or more related behavioral, neurological, or mental health disorders, and a plurality of remaining unasked questions of the plurality of questions including a most predictive next question; (2) determining whether the subject is at higher risk for a first developmental disorder or a second developmental disorder of the two or more related behavioral, neurological, or mental health disorders based on the plurality of answers; and (3) identifying the most predictive next question of the plurality of remaining unasked questions in response to determining that the subject is at higher risk for the first developmental disorder or the second developmental disorder of the two or more related behavioral, neurological, or mental health disorders.

[0105] In some embodiments, the question that is most predictive of the first developmental disorder is identified as the most predictive next question in response to determining that the subject has a higher risk of the first developmental disorder.

[0106] In some embodiments, the question that is most predictive of the second developmental disorder is identified as the most predictive next question in response to determining that the subject has a higher risk of the second developmental disorder.

[0107] In some embodiments, the system further comprises a memory in which an assessment model is stored. The assessment model may comprise a statistical correlation between a plurality of clinical characteristics and a clinical diagnosis of two or more behavioral, neurological or mental health disorders.

[0108] In some embodiments, the processor is further configured with instructions to determine whether the subject has a higher risk for the first developmental disorder or the second developmental disorder in response to the assessment model.

[0109] In some embodiments, the processor is configured with instructions to display the question and the most predictive next question.

[0110] In some embodiments, the processor includes instructions for identifying the most predictive next question in response to a plurality of answers corresponding to a plurality of clinical characteristics of the subject.

[0111] In some embodiments, the processor is configured with instructions to identify the most predictive next question responsive to the estimated predictive utility of each remaining question.

[0112] In some embodiments, the processor is configured with sufficient statistics to identify the most predictive next question that best predicts the first developmental disorder. In some embodiments, the processor is configured with sufficient statistics of a machine learning algorithm configured responsive to a plurality of clinically assessed subject populations to identify the most predictive next question that best predicts a higher risk of the first developmental disorder.

[0113] In some embodiments, the processor is configured with instructions to identify the most predictive next question in response to an estimated predictive utility of the most predictive next question with respect to each of two or more behavioral, neurological or mental health disorders.

[0114] In some embodiments, the processor is configured to determine that the subject is at risk for a developmental disorder with a confidence interval selected from the group consisting of at least 85% and with a sensitivity and specificity of at least 85%.

[0115] In some embodiments, the personalized therapeutic treatment plan includes a digital therapeutic. The digital therapeutic may include instructions, feedback, activities, or interactions provided to the subject or a caregiver. The digital therapeutic may be implemented on a mobile device.

[0116] In some embodiments, the diagnostic data and the individualized therapeutic treatment plan are provided to a third party system, which may include a computer system of a healthcare professional.

[0117] In some embodiments, the diagnostic module is configured to receive updated subject data from the subject in response to the subject's feedback data and generate updated diagnostic data. The therapeutic module may be configured to receive the updated diagnostic data and output an updated individualized treatment plan for the subject in response to the diagnostic data and the updated diagnostic data. The updated subject data may be received in response to feedback data identifying the relative levels of efficacy, compliance, and response resulting from the individualized therapeutic treatment plan.

[0118] In some embodiments, the diagnostic module includes instructions selected from the group consisting of machine learning, classifiers, artificial intelligence, or statistical modeling based on the subject population to determine diagnostic data. The therapeutic module may include instructions selected from the group consisting of machine learning, classifiers, artificial intelligence, or statistical modeling based on at least a portion of the subject population to determine an individualized therapeutic treatment plan for the subject.

[0119] In some embodiments, the diagnostic module includes a diagnostic machine learning classifier trained on the subject population. The therapeutic module may include a therapeutic machine learning classifier trained on at least a portion of the subject population. The diagnostic module may be configured to provide feedback to the therapeutic module based on execution of the individualized therapeutic treatment plan.

[0120] In some embodiments, the data from the subject includes at least one of subject and caregiver video, audio, responses to questions or activities, and active or passive data streams from user interaction with system activities, games, or software features.

[0121] In some embodiments, the subject is at risk of behavioral, neurological, and mental health disorders. The behavioral, neurological, or mental health disorders can include at least one of autism, autism spectrum, attention deficit disorder, attention deficit hyperactivity disorder, and speech and learning disabilities.

[0122] In some embodiments, the diagnostic module is configured for an adult to perform the evaluation or to provide data for the evaluation of a child or minor.

[0123] In some embodiments, the diagnostic module is configured to allow a caregiver or family member to perform the evaluation or provide data for the evaluation of a subject.

[0124] Aspects of the present disclosure also provide a method of treating a subject having a behavioral, neurological or mental health disorder of two or more related behavioral, neurological or mental health disorders with an individualized therapeutic treatment plan. An exemplary method may include a diagnostic process that receives data from the subject and outputs diagnostic data for the subject, and a therapeutic process that receives the diagnostic data and outputs an individualized therapeutic treatment plan for the subject. The diagnostic data may be generated by (1) receiving a plurality of answers to a plurality of asked questions of the plurality of questions, the plurality of answers corresponding to a subject's clinical characteristics related to two or more related behavioral, neurological, or mental health disorders, and a plurality of remaining unasked questions of the plurality of questions including a most predictive next question; (2) determining whether the subject is at higher risk for a first developmental disorder or a second developmental disorder of the two or more related behavioral, neurological, or mental health disorders based on the plurality of answers; and (3) identifying a most predictive next question of the plurality of remaining unasked questions in response to determining that the subject is at higher risk for a first developmental disorder or a second developmental disorder of the two or more related behavioral, neurological, or mental health disorders.

[0125] In some embodiments, the question that is most predictive of the first developmental disorder is identified as the most predictive next question in response to determining that the subject has a higher risk of the first developmental disorder.

[0126] In some embodiments, the question that is most predictive of the second developmental disorder is identified as the most predictive next question in response to determining that the subject has a higher risk of the second developmental disorder.

[0127] In some embodiments, the method may further include an assessment model that stores the process. The assessment model may include a statistical correlation between a plurality of clinical characteristics and a clinical diagnosis of two or more behavioral, neurological or mental health disorders.

[0128] In some embodiments, the method further comprises determining whether the subject has a higher risk of the first developmental disorder or the second developmental disorder in response to the assessment model.

[0129] In some embodiments, the method further includes displaying the question and the most predictive next question.

[0130] In some embodiments, the method further comprises identifying the most predictive next question in response to the plurality of answers corresponding to the plurality of clinical characteristics of the subject.

[0131] In some embodiments, the method further includes identifying the most predictive next question responsive to the estimated predictive utility of each remaining question.

[0132] In some embodiments, the diagnostic process includes providing sufficient statistics to identify the most predictive next question that best predicts the first developmental disorder.

[0133] In some embodiments, the diagnostic process includes providing sufficient statistics for a machine learning algorithm configured in response to a plurality of clinically assessed subject populations to identify the most predictive next questions that best predict a higher risk of the first developmental disorder.

[0134] In some embodiments, the diagnostic process includes identifying the most predictive next question in response to an estimated predictive utility of the most predictive next question for each of two or more behavioral, neurological or mental health disorders.

[0135] In some embodiments, the diagnostic process includes determining that the subject is at risk for a developmental disorder with a confidence interval selected from the group consisting of at least 85% and with a sensitivity and specificity of at least 85%.

[0136] In some embodiments, the personalized therapeutic treatment plan includes a digital therapeutic. The digital therapeutic includes instructions, feedback, activities, or interactions provided to the subject or a caregiver. The digital therapeutic may be implemented on a mobile device.

[0137] In some embodiments, the method further includes providing the diagnostic data and the individualized therapeutic treatment plan to a third party system, which may include a computer system of a healthcare professional.

[0138] In some embodiments, the diagnostic process includes receiving updated subject data from the subject in response to the subject's feedback data and generating updated diagnostic data. The therapeutic process may include receiving the updated diagnostic data and outputting an updated individualized treatment plan for the subject in response to the diagnostic data and the updated diagnostic data. The updated subject data may be received in response to feedback data identifying the relative levels of efficacy, compliance, and response resulting from the individualized therapeutic treatment plan.

[0139] In some embodiments, the diagnostic process is performed by a process selected from the group consisting of machine learning, classifiers, artificial intelligence, and statistical modeling based on a subject population to determine diagnostic data.

[0140] In some embodiments, the therapeutic process is performed by a process selected from the group consisting of machine learning, classifiers, artificial intelligence, and statistical modeling based on at least a portion of the subject population to determine an individualized therapeutic treatment plan for the subject.

[0141] In some embodiments, the diagnostic process is performed by a diagnostic machine learning classifier trained on the subject population. The therapeutic process may be performed by a therapeutic machine learning classifier trained on at least a portion of the subject population. The diagnostic process may include providing feedback to a therapeutic module based on the execution of an individualized therapeutic treatment plan.

[0142] In some embodiments, the data from the subject includes at least one of subject and caregiver video, audio, responses to questions or activities, and active or passive data streams from user interaction with system activities, games, or software features.

[0143] In some embodiments, the subject is at risk for a disorder selected from the group consisting of a behavioral disorder, a neurological disorder, and a mental health disorder. The risk may be selected from the group consisting of autism, autism spectrum, attention deficit disorder, depression, obsessive-compulsive disorder, schizophrenia, Alzheimer's disease, dementia, attention deficit hyperactivity disorder, and speech and learning disabilities.

[0144] In some embodiments, the diagnostic process is performed by an adult performing the evaluation or providing data for the evaluation of a child or minor.

[0145] In some embodiments, the diagnostic process allows a caregiver or family member to perform the evaluation or provide data for the subject's evaluation.

[0146] Aspects of the present disclosure may also be embodied in a tangible medium comprising instructions that, when executed, cause a processor to receive updated subject data in response to treating the subject, and output an updated personalized treatment plan for the subject in response to the updated subject data.

[0147] In some embodiments, the diagnostic module and the therapeutic module each include a classifier trained on a subject-free population.

[0148] In some embodiments, the processor includes multiple processors. Literature citations

[0149] All publications, patents, and patent applications mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated by reference.

[0150] The novel features of the invention are set forth with particularity in the appended claims. The features and advantages of the present invention will be better understood by reference to the following detailed description that sets forth illustrative embodiments, in which the principles of the invention are utilized, and the accompanying drawings of which: The present invention provides, for example, the following items. (Item 1) 1. A digital therapeutic system for treating a subject with an individualized therapeutic treatment plan, comprising: a diagnostic module for receiving data from the subject and outputting diagnostic data for the subject, the diagnostic module including one or more of machine learning, classifiers, artificial intelligence, or statistical modeling based on a subject population to determine the diagnostic data for the subject; and a treatment module for receiving the diagnostic data and outputting the individualized therapeutic treatment plan for the subject, the treatment module including one or more of machine learning, classifiers, artificial intelligence, or statistical modeling based on at least a portion of the subject population to determine and output the individualized therapeutic treatment plan for the subject; 1. A digital therapeutic system comprising: a diagnostic module configured to receive updated subject data from the subject in response to the treatment of the subject and generate updated diagnostic data from the subject; and a therapeutic module configured to receive the updated diagnostic data and output an updated personalized treatment plan for the subject in response to the diagnostic data and the updated diagnostic data. (Item 2) 2. The system of claim 1, wherein the diagnostic module includes a diagnostic machine learning classifier trained on the subject population and the therapeutic module includes a therapeutic machine learning classifier trained on at least the portion of the subject population, and the diagnostic module and the therapeutic module are configured such that the diagnostic module provides feedback to the therapeutic module based on execution of the treatment plan. (Item 3) 3. The system of claim 2, wherein the treatment classifier comprises instructions trained on a dataset that includes a population of which the subject is not a member, and the subject includes a person who is not a member of the population. (Item 4) 2. The system of claim 1, wherein the diagnostic module includes a diagnostic classifier trained on a plurality of profiles of a subject population of at least 10,000 people and a treatment profile trained on the plurality of profiles of the subject population. (Item 5) 1. A digital personalized treatment system, comprising: (i) software and digital devices that use data to assess and diagnose subjects; (ii) software and digital devices that capture interaction and feedback data that identifies relative levels of efficacy, compliance, and response resulting from the therapeutic intervention; (iii) data analytics capabilities including machine learning, AI, and statistical models that evaluate user data and user profiles to further personalize, improve, or evaluate the effectiveness of the therapeutic intervention. (Item 6) 6. The system of claim 5, wherein the system includes software-based learning capabilities that enable the system to use its user data to monitor and improve its diagnostic and therapeutic interventions. (Item 7) 6. The system of claim 5, wherein the digital diagnostics in the system include data and metadata collected from the subject or caregiver in one or more active or passive digital forms via different digital devices such as a mobile phone, video capture, audio capture, activity monitor, or wearable digital monitor. (Item 8) The system of item 5, wherein the digital diagnosis uses the data collected by the system about the subject, with or without complementary diagnostic data captured outside the digital diagnosis, along with analytical results from tools such as machine learning, artificial intelligence, and statistical modeling to assess or diagnose the condition of the subject. (Item 9) The digital diagnostics are powered by tools such as machine learning, artificial intelligence, and statistical modeling. The system of claim 5, further enabling assessment of changes in a subject's condition or performance directly or indirectly through data and metadata that can be analyzed to provide feedback to the system to improve or refine the diagnosis and potential therapeutic intervention. (Item 10) The system of claim 5, wherein the data evaluation and machine learning from the digital diagnosis and corresponding response, or lack thereof, from the therapeutic intervention is configured to identify a novel diagnosis for the subject and a novel treatment regimen for both the patient and the caregiver. The system of item 5, wherein the types of data collected and utilized by the system include subject and caregiver video, audio, responses to questions or activities, as well as active or passive data streams from user interaction with the system's activities, games, or software features. (Item 12) The system of claim 5, wherein the metadata includes data from a user's interactions with a device or mobile app of the system that captures a profile of one or more of the user's behavior, profile, activity, interaction with the software system, interaction with a game, frequency of use, session time, selected options or features, content or activity preferences. (Item 13) 6. The system of claim 5, wherein the data includes data and metadata from various third party devices, such as activity monitors, games, or interactive content. (Item 14) 6. The system of claim 5, wherein the digital therapeutic comprises instructions, feedback, activities, or interactions provided to the subject or caregiver by a mobile device. (Item 15) 6. The system of claim 5, wherein the system includes instructions for providing a suggested action, activity, game, or interactive session by system software and / or a third party device. (Item 16) The system of claim 5, wherein the system comprises instructions for diagnosing and treating one or more of cognitive or behavioral development, neurodegenerative conditions, cognitive and behavioral disorders or conditions, including mood disorders. (Item 17) 1. A system for diagnosing and treating a subject, the apparatus comprising: a diagnostic module for receiving object data and outputting diagnostic data for the object; and a therapeutic module for receiving the diagnostic data and outputting a therapeutic treatment for the subject, the diagnostic module and the therapeutic module being arranged such that they include a feedback loop to update the treatment in response to the diagnostic data. (Item 18) 1. A mobile device for delivering digital personalized treatment, comprising: A display and 1. A processor comprising: generating a user profile in response to user interaction with the device; receiving and displaying treatment instructions to the user in response to the user profile; updating the user profile in response to the action and transmitting the updated user profile to a remote server; receiving updated treatment instructions from the server; Displaying treatment update instructions to the user and a processor configured with instructions for: (Item 19) 1. A digital therapeutic system for treating a subject with an individualized therapeutic treatment plan, the system comprising: a diagnostic module for receiving data from the subject and outputting diagnostic data for the subject; and a therapy module for receiving the diagnostic data and outputting the personalized therapeutic treatment plan for the subject, the personalized therapeutic treatment plan including a digital therapeutic. 20. The system of claim 19, wherein the digital therapeutic comprises instructions, feedback, activities, or interactions provided to the subject or caregiver. (Item 21) 21. The system of claim 20, wherein the digital therapeutic is provided with a mobile device. (Item 22) 20. The system of claim 19, wherein the diagnostic data and the individualized therapeutic treatment plan are provided to a third party system. (Item 23) 23. The system of claim 22, wherein the third party system comprises a healthcare professional's computer system or a treatment delivery system. (Item 24) 20. The system of claim 19, wherein the diagnostic module is configured to receive updated subject data from the subject in response to the subject's feedback data and generate updated diagnostic data, and the therapeutic module is configured to receive the updated diagnostic data and output an updated individualized treatment plan for the subject in response to the diagnostic data and the updated diagnostic data. (Item 25) 25. The system of claim 24, wherein the updated subject data is received in response to feedback data identifying relative levels of efficacy, compliance, and response resulting from the individualized therapeutic treatment plan. (Item 26) 20. The system of claim 19, wherein the diagnostic module comprises machine learning, a classifier, artificial intelligence, or statistical modeling based on a subject population to determine the diagnostic data. (Item 27) 27. The system of claim 26, wherein the treatment module includes machine learning, a classifier, artificial intelligence, or statistical modeling based on at least a portion of the subject population to determine the individualized therapeutic treatment plan for the subject. (Item 28) 20. The system of claim 19, wherein the diagnostic module includes a diagnostic machine learning classifier trained on a subject population and the therapeutic module includes a therapeutic machine learning classifier trained on at least a portion of the subject population, and the diagnostic module is configured to provide feedback to the therapeutic module based on execution of the individualized therapeutic treatment plan. (Item 29) 20. The system of claim 19, wherein the data from the subject includes at least one of subject and caregiver video, audio, responses to questions or activities, and active or passive data streams from user interaction with activities, games, or software features of the system. (Item 30) 20. The system of claim 19, wherein the subject is at risk for a disorder selected from the group consisting of a behavioral disorder, a neurological disorder, and a mental health disorder. (Item 31) 31. The system of claim 30, wherein the behavioral, neurological or mental health disorder is selected from the group consisting of autism, autism spectrum, attention deficit disorder, depression, obsessive-compulsive disorder, schizophrenia, Alzheimer's disease, dementia, attention deficit hyperactivity disorder, and speech and learning disabilities. (Item 32) 20. The system of claim 19, wherein the diagnostic module is configured for an adult to perform an evaluation or provide data for an evaluation of a child or minor. (Item 33) 20. The system of claim 19, wherein the diagnostic module is configured to allow a caregiver or family member to perform an evaluation or provide data for evaluation of the subject. (Item 34) 1. A method of treating a subject with an individualized therapeutic treatment plan, the method comprising: a diagnostic process for receiving data from the subject and outputting diagnostic data for the subject; a therapy process receiving the diagnostic data and outputting the personalized therapy treatment plan for the subject, the personalized therapy treatment plan including a digital therapeutic. (Item 35) 35. The method of claim 34, wherein the digital therapeutic comprises instructions, feedback, activities, or interactions provided to the subject or caregiver. (Item 36) 36. The method of claim 35, wherein the digital therapeutic is provided with a mobile device. (Item 37) 35. The method of claim 34, further comprising providing the diagnostic data and the individualized therapeutic treatment plan to a third party system. (Item 38) 38. The method of claim 37, wherein the third party system comprises a healthcare professional's computer system or a treatment delivery system. (Item 39) 35. The method of claim 34, wherein the diagnostic process further includes receiving updated subject data from the subject in response to the subject's feedback data and generating updated diagnostic data, and the therapeutic process further includes receiving the updated diagnostic data and outputting an updated individualized treatment plan for the subject in response to the diagnostic data and the updated diagnostic data. (Item 40) 40. The method of claim 39, wherein the updated subject data is received in response to feedback data identifying relative levels of efficacy, compliance, and response resulting from the individualized therapeutic treatment plan. (Item 41) 35. The method of claim 34, wherein the diagnostic process is performed by a process selected from the group consisting of machine learning, classifiers, artificial intelligence, and statistical modeling based on a subject population to determine the diagnostic data. (Item 42) The therapeutic process comprises machine learning, classifiers, artificial intelligence, or statistical modeling based on at least a portion of the subject population to determine the individualized therapeutic treatment plan for the subject. 42. The method according to claim 41, wherein the method is carried out by a process selected from the group consisting of: (Item 43) 35. The method of claim 34, wherein the diagnostic process is performed by a diagnostic machine learning classifier trained on a subject population and the therapeutic process is performed by a therapeutic machine learning classifier trained on at least a portion of the subject population, and the diagnostic process further comprises providing feedback to the therapeutic module based on execution of the individualized therapeutic treatment plan. (Item 44) 35. The method of claim 34, wherein the data from the subject includes at least one of subject and caregiver video, audio, responses to questions or activities, and active or passive data streams from user interaction with activities, games, or software features. (Item 45) 35. The method of claim 34, wherein the subject is at risk for a disorder selected from the group consisting of a behavioral disorder, a neurological disorder, and a mental health disorder. (Item 46) 46. ​​The method of claim 45, wherein the risk is selected from the group consisting of autism, autism spectrum, attention deficit disorder, depression, obsessive-compulsive disorder, schizophrenia, Alzheimer's disease, dementia, attention deficit hyperactivity disorder, and speech and learning disabilities. (Item 47) 46. ​​The method of claim 45, wherein the diagnostic process is carried out by an adult performing the evaluation or providing data for the evaluation of a child or minor. (Item 48) 46. ​​The method of claim 45, wherein the diagnostic process allows a caregiver or family member to perform an evaluation or provide data for the evaluation of the subject. (Item 49) 1. A therapeutic system for treating a subject with an individualized therapeutic treatment plan, the system comprising: a diagnostic module for receiving data from the subject and outputting diagnostic data for the subject; a processor including software instructions for receiving the diagnostic data and a therapy module for outputting the individualized therapeutic treatment plan for the subject; A treatment system, wherein the diagnostic module is configured to receive updated subject data from the subject in response to treatment of the subject and generate updated diagnostic data from the subject, and the treatment module is configured to receive the updated diagnostic data and output an updated individualized treatment plan for the subject in response to the diagnostic data and the updated diagnostic data. (Item 50) 50. The system of claim 49, wherein the updated subject data is received in response to feedback data identifying relative levels of efficacy, compliance, and response resulting from the individualized therapeutic treatment plan. (Item 51) 50. The system of claim 49, wherein the personalized therapeutic treatment plan includes a digital therapeutic. (Item 52) 52. The system of claim 51, wherein the digital therapeutic comprises instructions, feedback, activities, or interactions provided to the subject or caregiver. (Item 53) 53. The system of claim 52, wherein the digital therapeutic is provided with a mobile device. (Item 54) The diagnostic data and the personalized therapeutic treatment plan are provided to a third party system. 50. The system according to item 49, (Item 55) 55. The system of claim 54, wherein the third party system comprises a healthcare professional's computer system or a treatment delivery system. (Item 56) 50. The system of claim 49, wherein the diagnostic module includes machine learning, a classifier, artificial intelligence, or statistical modeling based on a subject population to determine the diagnostic data. (Item 57) 57. The system of claim 56, wherein the treatment module includes machine learning, a classifier, artificial intelligence, or statistical modeling based on at least a portion of the subject population to determine the individualized therapeutic treatment plan for the subject. (Item 58) 50. The system of claim 49, wherein the diagnostic module includes a diagnostic machine learning classifier trained on a subject population and the therapeutic module includes a therapeutic machine learning classifier trained on at least a portion of the subject population, and the diagnostic module is configured to provide feedback to the therapeutic module based on execution of the individualized therapeutic treatment plan. (Item 59) 50. The system of claim 49, wherein the data from the subject includes at least one of subject and caregiver video, audio, responses to questions or activities, and active or passive data streams from user interaction with activities, games, or software features of the system. (Item 60) 50. The system of claim 49, wherein the diagnostic module is configured for an adult to perform an evaluation or provide data for an evaluation of a child or minor. (Item 61) 50. The system of claim 49, wherein the diagnostic module is configured to allow a caregiver or family member to perform an evaluation or provide data for evaluation of the subject. (Item 62) 50. The system of claim 49, wherein the subject is at risk for a disorder selected from the group consisting of behavioral, neurological, and mental health disorders. (Item 63) Item 63. The system of item 62, wherein the risk is selected from the group consisting of autism, autism spectrum, attention deficit disorder, depression, obsessive-compulsive disorder, schizophrenia, Alzheimer's disease, dementia, attention deficit hyperactivity disorder, and speech and learning disabilities. (Item 64) 1. A method of treating a subject with an individualized therapeutic treatment plan, the method comprising: a diagnostic process for receiving data from the subject and outputting diagnostic data for the subject; a therapeutic process for receiving the diagnostic data and outputting the individualized therapeutic treatment plan for the subject; The method, wherein the diagnostic process includes receiving updated subject data from the subject in response to treating the subject, and generating updated diagnostic data from the subject, and the therapeutic process includes receiving the updated diagnostic data, and outputting an updated individualized treatment plan for the subject in response to the diagnostic data and the updated diagnostic data. (Item 65) 65. The method of claim 64, wherein the updated subject data is received in response to feedback data identifying relative levels of efficacy, compliance, and response resulting from the individualized therapeutic treatment plan. (Item 66) 65. The method of claim 64, wherein the personalized therapeutic treatment plan comprises a digital therapeutic. (Item 67) 67. The method of claim 66, wherein the digital therapeutic comprises instructions, feedback, activities, or interactions provided to the subject or caregiver. (Item 68) 68. The method of claim 67, wherein the digital therapeutic is provided with a mobile device. (Item 69) 65. The method of claim 64, further comprising providing the diagnostic data and the individualized therapeutic treatment plan to a third party system. (Item 70) 70. The method of claim 69, wherein the third party system comprises a healthcare professional's computer system or a treatment delivery system. (Item 71) Item 65. The method of item 64, wherein the diagnostic process is performed by a process selected from the group consisting of machine learning, a classifier, artificial intelligence, or statistical modeling based on a subject population to determine the diagnostic data. (Item 72) 72. The method of claim 71, wherein the therapeutic process is performed by a process selected from the group consisting of machine learning, a classifier, artificial intelligence, or statistical modeling based on at least a portion of the subject population to determine the individualized therapeutic treatment plan for the subject. (Item 73) Item 65. The method of item 64, wherein the diagnostic process is performed by a diagnostic machine learning classifier trained on a subject population, and the therapeutic process is performed by a therapeutic machine learning classifier trained on at least a portion of the subject population, and the diagnostic process includes providing feedback to the therapeutic module based on execution of the individualized therapeutic treatment plan. (Item 74) Item 65. The method of item 64, wherein the data from the subject includes at least one of subject and caregiver video, audio, responses to questions or activities, and active or passive data streams from user interaction with activities, games, or software features. (Item 75) Item 65. The method of item 64, wherein the diagnostic process is carried out by an adult performing the evaluation or providing data for the evaluation of a child or minor. (Item 76) 65. The method of claim 64, wherein the diagnostic process allows a caregiver or family member to perform an evaluation or provide data for the evaluation of the subject. (Item 77) 65. The method of claim 64, wherein the subject is at risk for a disorder selected from the group consisting of a behavioral disorder, a neurological disorder, and a mental health disorder. (Item 78) 65. The method of claim 64, wherein the risk is selected from the group consisting of autism, autism spectrum, attention deficit disorder, depression, obsessive-compulsive disorder, schizophrenia, Alzheimer's disease, dementia, attention deficit hyperactivity disorder, and speech and learning disabilities. (Item 79) 1. A therapeutic system for treating a subject with an individualized therapeutic treatment plan, the system comprising: a diagnostic module for receiving data from the subject and outputting diagnostic data for the subject; receiving the diagnostic data and outputting the personalized therapeutic treatment plan for the subject; Treatment module and a processor including software instructions for The diagnostic module is configured to generate the diagnostic data by: (1) receiving a plurality of answers to a plurality of asked questions of a plurality of questions, the plurality of answers corresponding to clinical characteristics of the subject related to the subject's developmental progression, and a plurality of remaining unasked questions of the plurality of questions including a most predictive next question; (2) determining the developmental progression of the subject based on the plurality of answers; and (3) identifying the most predictive next question of the plurality of remaining unasked questions in response to determining the developmental progression of the subject. (Item 80) 80. The system of claim 79, wherein the diagnostic module includes a pre-processing module, a training module, and a prediction module, wherein the data processing module extracts training data from a database or a user, applies a transformation to standardize the training data, and passes the standardized training data to the training module, which builds an evaluation model based on the standardized training data, and the prediction module generates a predicted classification of the subject. (Item 81) Item 81. The system of item 80, wherein the training module utilizes machine learning algorithms to build and train the evaluation model. (Item 82) Item 81. The system of item 80, wherein the prediction module is configured to generate the predicted classification of the subject by fitting new data to the assessment model, the new data being standardized by the pre-processing module. (Item 83) 83. The system of claim 82, wherein the prediction module checks whether the fit of the new data produces a prediction of a particular disorder within a confidence interval that exceeds a threshold. (Item 84) Item 81. The system of item 80, wherein the prediction module includes a question recommendation module, the question recommendation module configured to identify, select, or recommend the most predictive next question to ask the subject based on the plurality of answers to the plurality of asked questions to shorten a length of an assessment. (Item 85) Item 85. The system of item 84, wherein the question recommendation module selects a candidate question to recommend as the next question to be presented to the subject. (Item 86) 86. The system of claim 85, wherein the question recommendation module assesses an expected feature importance of each one of the candidate questions. (Item 87) 87. The system of claim 86, wherein the question recommendation module selects a most predictive next question from the candidate questions based on the predicted feature importance of each one of the candidate questions. (Item 88) 87. The system of claim 86, wherein the expected feature importance of each one of the candidate questions is determined by an expected feature importance determination algorithm. (Item 89) Item 81. The system of item 80, wherein the evaluation model includes a random forest classifier. (Item 90) 80. The system of claim 79, wherein the personalized therapeutic treatment plan includes a digital therapeutic. (Item 91) The digital therapeutic may include instructions, functions, and / or instructions provided to the subject or a caregiver. Item 91. The system of item 90, including feedback, activity, or interaction. (Item 92) 92. The system of claim 91, wherein the digital therapeutic is provided with a mobile device. (Item 93) 80. The system of claim 79, wherein the diagnostic data and the individualized therapeutic treatment plan are provided to a third party system. (Item 94) 94. The system of claim 93, wherein the third party system comprises a computer system of a healthcare professional. (Item 95) 80. The system of claim 79, wherein the diagnostic module is configured to receive updated subject data from the subject in response to the subject's feedback data and generate updated diagnostic data, and the therapeutic module is configured to receive the updated diagnostic data and output an updated individualized treatment plan for the subject in response to the diagnostic data and the updated diagnostic data. (Item 96) 96. The system of claim 95, wherein the updated subject data is received in response to feedback data identifying relative levels of efficacy, compliance, and response resulting from the individualized therapeutic treatment plan. (Item 97) 80. The system of claim 79, wherein the diagnostic module comprises instructions selected from the group consisting of machine learning, classifiers, artificial intelligence, and statistical modeling based on a subject population to determine the diagnostic data. (Item 98) 98. The system of claim 97, wherein the treatment module comprises instructions selected from the group consisting of machine learning, classifiers, artificial intelligence, or statistical modeling based on at least a portion of the subject population to determine the individualized therapeutic treatment plan for the subject. (Item 99) 80. The system of claim 79, wherein the diagnostic module includes a diagnostic machine learning classifier trained on a subject population and the therapeutic module includes a therapeutic machine learning classifier trained on at least a portion of the subject population, and the diagnostic module is configured to provide feedback to the therapeutic module based on execution of the individualized therapeutic treatment plan. (Item 100) 80. The system of claim 79, wherein the data from the subject includes at least one of subject and caregiver video, audio, responses to questions or activities, and active or passive data streams from user interaction with activities, games, or software features of the system. (Item 101) 80. The system of claim 79, wherein the subject is at risk for a disorder selected from the group consisting of a behavioral disorder, a neurological disorder, and a mental health disorder. (Item 102) Item 102. The system of item 101, wherein the risk is selected from the group consisting of autism, autism spectrum, attention deficit disorder, depression, obsessive-compulsive disorder, schizophrenia, Alzheimer's disease, dementia, attention deficit hyperactivity disorder, and speech and learning disabilities. (Item 103) 80. The system of claim 79, wherein the diagnostic module is configured for an adult to perform an evaluation or provide data for an evaluation of a child or minor. (Item 104) The diagnostic module may be configured to allow a caregiver or family member to perform an assessment or to allow the subject 80. The system of claim 79, configured to provide data for evaluation of. (Item 105) 1. A method of treating a subject with an individualized therapeutic treatment plan, the method comprising: a diagnostic process for receiving data from the subject and outputting diagnostic data for the subject; a therapeutic process for receiving the diagnostic data and outputting the individualized therapeutic treatment plan for the subject; The method, wherein the diagnostic process includes generating the diagnostic data by: (1) receiving a plurality of answers to a plurality of asked questions of a plurality of questions, the plurality of answers corresponding to clinical characteristics of the subject related to the subject's developmental progression, and a plurality of remaining unasked questions of the plurality of questions including a most predictive next question; (2) determining the developmental progression of the subject based on the plurality of answers; and (3) identifying the most predictive next question of the plurality of remaining unasked questions in response to determining the developmental progression of the subject. (Item 106) Item 106. The method of item 105, wherein the diagnostic process includes a pre-processing process, a training process, and a prediction process, wherein the data processing process extracts training data from a database or a user, applies one or more transformations to standardize the training data, and passes the standardized training data to the training process, wherein the training process builds an evaluation model based on the standardized training data, and wherein the prediction process generates a predicted classification of the subject. (Item 107) Item 107. The method of item 106, wherein the training process utilizes a machine learning algorithm to build and train the evaluation model. (Item 108) Item 107. The method of item 106, wherein the prediction process generates the predicted classification of the subject by fitting new data to the assessment model, the new data being standardized by the pre-processing process. (Item 109) Item 109. The method of item 108, wherein the prediction process checks whether the fit of the new data produces a prediction of one or more particular disorders within a confidence interval that exceeds a threshold. (Item 110) Item 107. The method of item 106, wherein the prediction process includes a question recommendation process that identifies, selects, or recommends the most predictive next question to ask the subject based on the plurality of answers to the plurality of asked questions to shorten the length of an evaluation. (Item 111) Item 111. The method of item 110, wherein the question recommendation process selects one or more candidate questions to recommend as the next question to be presented to the subject. (Item 112) Item 112. The method of item 111, wherein the question recommendation process assesses a predicted feature importance for each one of the candidate questions. (Item 113) Item 113. The method of item 112, wherein the question recommendation process selects one or more most predictive next questions from the candidate questions based on the predicted feature importance of each one of the candidate questions. (Item 114) Item 113. The method of item 112, wherein the expected feature importance of each one of the candidate questions is determined by an expected feature importance determination algorithm. (Item 115) Item 107. The method of item 106, wherein the evaluation process includes a random forest classifier. (Item 116) 106. The method of claim 105, wherein the personalized therapeutic treatment plan includes a digital therapeutic. (Item 117) 117. The method of claim 116, wherein the digital therapeutic comprises instructions, feedback, activities, or interactions provided to the subject or caregiver. (Item 118) Item 118. The method of item 117, wherein the digital therapeutic is provided with a mobile device. (Item 119) 106. The method of claim 105, further comprising providing the diagnostic data and the individualized therapeutic treatment plan to a third party system. (Item 120) Item 120. The method of item 119, wherein the third party system includes a computer system of a healthcare professional. (Item 121) Item 106. The method of item 105, wherein the diagnostic process includes receiving updated subject data from the subject in response to the subject's feedback data and generating updated diagnostic data, and the therapeutic process includes receiving the updated diagnostic data and outputting an updated individualized treatment plan for the subject in response to the diagnostic data and the updated diagnostic data. (Item 122) Item 122. The method of item 121, wherein the updated subject data is received in response to feedback data identifying relative levels of efficacy, compliance, and response resulting from the individualized therapeutic treatment plan. (Item 123) Item 106. The method of item 105, wherein the diagnostic process is performed by a process selected from the group consisting of machine learning, a classifier, artificial intelligence, or statistical modeling based on a subject population to determine the diagnostic data. (Item 124) Item 124. The method of item 123, wherein the therapeutic process is performed by a process selected from the group consisting of machine learning, a classifier, artificial intelligence, or statistical modeling based on at least a portion of the subject population to determine the individualized therapeutic treatment plan for the subject. (Item 125) Item 106. The method of item 105, wherein the diagnostic process is performed by a diagnostic machine learning classifier trained on a subject population, and the therapeutic process is performed by a therapeutic machine learning classifier trained on at least a portion of the subject population, and the diagnostic process includes providing feedback to the therapeutic module based on execution of the individualized therapeutic treatment plan. (Item 126) Item 106. The method of item 105, wherein the data from the subject includes at least one of subject and caregiver video, audio, responses to questions or activities, and active or passive data streams from user interaction with activities, games, or software features of the system. (Item 127) 106. The method of claim 105, wherein the subject is at risk for a disorder selected from the group consisting of a behavioral disorder, a neurological disorder, and a mental health disorder. (Item 128) The risks include autism, autism spectrum disorders, attention deficit disorder, depression, obsessive-compulsive disorder, schizophrenia, Alzheimer's disease, dementia, attention deficit hyperactivity disorder, and speech and learning disorders. Item 128. The method of item 127, wherein the said patient is selected from the group consisting of impotence. (Item 129) Item 106. The method of item 105, wherein the diagnostic process is carried out by an adult performing the evaluation or providing data for the evaluation of a child or minor. (Item 130) 106. The method of claim 105, wherein the diagnostic process allows a caregiver or family member to perform an evaluation or provide data for the evaluation of the subject. (Item 131) 1. A therapeutic system for treating a subject having two or more related behavioral, neurological or mental health disorders with an individualized therapeutic treatment plan, the system comprising: a diagnostic module for receiving data from the subject and outputting diagnostic data for the subject; a therapy module for receiving the diagnostic data and outputting the individualized therapeutic treatment plan for the subject; a processor including software instructions for a plurality of questions, each of which corresponds to a clinical characteristic of the subject related to two or more related behavioral, neurological, or mental health disorders, and a plurality of remaining unasked questions of the plurality of questions includes a most predictive next question; (2) determining whether the subject is at higher risk for a first developmental disorder or a second developmental disorder of the two or more related behavioral, neurological, or mental health disorders based on the plurality of answers; and (3) identifying the most predictive next question from the plurality of remaining unasked questions in response to determining that the subject is at higher risk for a first developmental disorder or a second developmental disorder of the two or more related behavioral, neurological, or mental health disorders. (Item 132) Item 132. The system of item 131, wherein the question most predictive of the first developmental disorder is identified as the most predictive next question in response to determining that the subject has a higher risk of the first developmental disorder. (Item 133) Item 132. The system of item 131, wherein the question most predictive of the second developmental disorder is identified as the most predictive next question in response to determining that the subject has a higher risk of the second developmental disorder. (Item 134) Item 132. The system of item 131, further comprising a memory in which an assessment model is stored, the assessment model comprising a statistical correlation between a plurality of clinical characteristics and clinical diagnoses of the two or more behavioral, neurological or mental health disorders. (Item 135) Item 132. The system of item 131, wherein the processor is further configured with instructions to determine whether the subject has a higher risk of the first developmental disorder or the second developmental disorder in response to the evaluation model. (Item 136) Item 132. The system of item 131, wherein the processor is configured with instructions to display the question and the most predictive next question. (Item 137) Item 132. The system of item 131, wherein the processor comprises instructions for identifying the most predictive next question in response to the plurality of answers corresponding to the plurality of clinical characteristics of the subject. (Item 138) Item 132. The system of item 131, wherein the processor is configured with instructions to identify the most predictive next question in response to an estimated predictive utility of each remaining question. (Item 139) Item 132. The system of item 131, wherein the processor is configured with sufficient statistics to identify the most predictive next question that best predicts the first developmental disorder. (Item 140) Item 141. The system of item 131, wherein the processor is configured with sufficient statistics of a machine learning algorithm configured in response to a plurality of clinically evaluated subject populations to identify the most predictive next question that best predicts a higher risk of the first developmental disorder. Item 132. The system of item 131, wherein the processor is configured with instructions to identify the most predictive next question in response to an estimated predictive utility of the most predictive next question with respect to each of the two or more behavioral, neurological or mental health disorders. (Item 142) Item 132. The system of item 131, wherein the processor is configured to determine that the subject is at risk for the developmental disorder with a confidence interval selected from the group consisting of at least 85% and a sensitivity and specificity of at least 85%. (Item 143) Item 132. The system of item 131, wherein the personalized therapeutic treatment plan includes a digital therapeutic. (Item 144) Item 144. The system of item 143, wherein the digital therapeutic comprises instructions, feedback, activities, or interactions provided to the subject or caregiver. (Item 145) Item 145. The system of item 144, wherein the digital therapeutic is provided with a mobile device. (Item 146) Item 132. The system of item 131, wherein the diagnostic data and the individualized therapeutic treatment plan are provided to a third party system. (Item 147) Item 147. The system of item 146, wherein the third party system includes a health care professional's computer system. (Item 148) Item 132. The system of item 131, wherein the diagnostic module is configured to receive updated subject data from the subject in response to the subject's feedback data and generate updated diagnostic data, and the therapeutic module is configured to receive the updated diagnostic data and output an updated individualized treatment plan for the subject in response to the diagnostic data and the updated diagnostic data. (Item 149) Item 149. The system of item 148, wherein the updated subject data is received in response to feedback data identifying relative levels of efficacy, compliance, and response resulting from the individualized therapeutic treatment plan. (Item 150) Item 132. The system of item 131, wherein the diagnostic module comprises instructions selected from the group consisting of machine learning, classifiers, artificial intelligence, or statistical modeling based on a subject population to determine the diagnostic data. (Item 151) The treatment module is configured to determine the individualized treatment plan for the subject. Item 151. The system of item 150, comprising instructions selected from the group consisting of machine learning, a classifier, artificial intelligence, or statistical modeling based on at least a portion of the population. (Item 152) Item 132. The system of item 131, wherein the diagnostic module includes a diagnostic machine learning classifier trained on a subject population, and the therapeutic module includes a therapeutic machine learning classifier trained on at least a portion of the subject population, and the diagnostic module is configured to provide feedback to the therapeutic module based on execution of the individualized therapeutic treatment plan. (Item 153) Item 132. The system of item 131, wherein the data from the subject includes at least one of subject and caregiver video, audio, responses to questions or activities, and active or passive data streams from user interaction with activities, games, or software features of the system. (Item 154) Item 132. The system of item 131, wherein the subject is at risk for a behavioral, neurological or mental health disorder. (Item 155) Item 155. The system of item 154, wherein the behavioral, neurological or mental health disorder comprises at least one of autism, autism spectrum, attention deficit disorder, attention deficit hyperactivity disorder, and speech and learning disabilities. (Item 156) Item 132. The system of item 131, wherein the diagnostic module is configured for an adult to perform an evaluation or provide data for an evaluation of a child or minor. (Item 157) Item 126. The system of item 125, wherein the diagnostic module is configured to allow a caregiver or family member to perform an evaluation or provide data for evaluation of the subject. (Item 158) 1. A method of treating a subject having two or more related behavioral, neurological or mental health disorders with an individualized therapeutic treatment plan, the method comprising: a diagnostic process for receiving data from the subject and outputting diagnostic data for the subject; a therapeutic process for receiving the diagnostic data and outputting the individualized therapeutic treatment plan for the subject; The diagnostic data is generated by: (1) receiving a plurality of answers to a plurality of asked questions of a plurality of questions, the plurality of answers corresponding to clinical characteristics of the subject related to two or more related behavioral, neurological, or mental health disorders, and a plurality of remaining unasked questions of the plurality of questions including a most predictive next question; (2) determining whether the subject is at higher risk for a first developmental disorder or a second developmental disorder of the two or more related behavioral, neurological, or mental health disorders based on the plurality of answers; and (3) identifying the most predictive next question of the plurality of remaining unasked questions in response to determining that the subject is at higher risk for a first developmental disorder or a second developmental disorder of the two or more related behavioral, neurological, or mental health disorders. (Item 159) Item 159. The method of item 158, wherein the question most predictive of the first developmental disorder is identified as the most predictive next question in response to determining that the subject has a higher risk of the first developmental disorder. (Item 160) The question that is most predictive of the second developmental disorder is identified as the most predictive next question in response to determining that the subject has a higher risk of the second developmental disorder. The method according to item 158, (Item 161) 159. The method of claim 158, further comprising an assessment model memory process, said assessment model comprising a statistical correlation between a plurality of clinical characteristics and clinical diagnoses of said two or more behavioral, neurological or mental health disorders. (Item 162) Item 159. The method of item 158, further comprising determining whether the subject has a higher risk of the first developmental disorder or the second developmental disorder in response to the evaluation model. (Item 163) Item 159. The method of item 158, further comprising displaying the question and the most predictive next question. (Item 164) Item 159. The method of item 158, further comprising identifying the most predictive next question in response to the plurality of answers corresponding to the plurality of clinical characteristics of the subject. (Item 165) Item 159. The method of item 158, further comprising identifying the most predictive next question in response to the estimated predictive utility of each remaining question. (Item 166) Item 159. The method of item 158, wherein the diagnostic process includes providing sufficient statistics to identify the most predictive next question that best predicts the first developmental disorder. (Item 167) Item 159. The method of item 158, wherein the diagnostic process includes providing sufficient statistics for a machine learning algorithm configured in response to a plurality of clinically evaluated subject populations to identify the most predictive next question that best predicts a higher risk of the first developmental disorder. (Item 168) Item 159. The method of item 158, wherein the diagnostic process comprises identifying the most predictive next question in response to an estimated predictive utility of the most predictive next question with respect to each of the two or more behavioral, neurological or mental health disorders. (Item 169) Item 159. The method of item 158, wherein the diagnostic process comprises determining that the subject is at risk for the developmental disorder with a confidence interval selected from the group consisting of at least 85% and a sensitivity and specificity of at least 85%. (Item 170) 159. The method of claim 158, wherein the personalized therapeutic treatment plan includes a digital therapeutic. (Item 171) 171. The method of claim 170, wherein the digital therapeutic comprises instructions, feedback, activities, or interactions provided to the subject or caregiver. (Item 172) Item 172. The method of item 171, wherein the digital therapeutic is provided with a mobile device. (Item 173) 159. The method of claim 158, further comprising providing the diagnostic data and the individualized therapeutic treatment plan to a third party system. (Item 174) Item 174. The method of item 173, wherein the third party system includes a computer system of a health care professional. (Item 175) The diagnostic process is updated from the subject in response to feedback data from the subject. Item 159. The method of item 158, further comprising: receiving updated subject data; and generating updated diagnostic data, wherein the therapeutic process includes receiving the updated diagnostic data; and outputting an updated individualized treatment plan for the subject in response to the diagnostic data and the updated diagnostic data. (Item 176) Item 174. The method of item 173, wherein the updated subject data is received in response to feedback data identifying relative levels of efficacy, compliance, and response resulting from the individualized therapeutic treatment plan. (Item 177) Item 159. The method of item 158, wherein the diagnostic process is performed by a process selected from the group consisting of machine learning, a classifier, artificial intelligence, and statistical modeling based on a subject population to determine the diagnostic data. (Item 178) Item 159. The method of item 158, wherein the therapeutic process is performed by a process selected from the group consisting of machine learning, classifiers, artificial intelligence, and statistical modeling based on at least a portion of the subject population to determine the individualized therapeutic treatment plan for the subject. (Item 179) Item 159. The method of item 158, wherein the diagnostic process is performed by a diagnostic machine learning classifier trained on a subject population, and the therapeutic process is performed by a therapeutic machine learning classifier trained on at least a portion of the subject population, and the diagnostic process includes providing feedback to the therapeutic module based on execution of the individualized therapeutic treatment plan. (Item 180) Item 159. The method of item 158, wherein the data from the subject includes at least one of subject and caregiver video, audio, responses to questions or activities, and active or passive data streams from user interaction with activities, games, or software features of the system. (Item 181) 159. The method of claim 158, wherein the subject is at risk for a disorder selected from the group consisting of a behavioral disorder, a neurological disorder, and a mental health disorder. (Item 182) 182. The method of claim 181, wherein said risk is selected from the group consisting of autism, autism spectrum, attention deficit disorder, depression, obsessive-compulsive disorder, schizophrenia, Alzheimer's disease, dementia, attention deficit hyperactivity disorder, and speech and learning disabilities. (Item 183) Item 159. The method of item 158, wherein the diagnostic process is carried out by an adult performing the evaluation or providing data for the evaluation of a child or minor. (Item 184) Item 159. The method of item 158, wherein the diagnostic process allows a caregiver or family member to perform an evaluation or provide data for the evaluation of the subject. (Item 185) A tangible medium that, when executed, causes a processor to: receiving updated subject data in response to treating the subject; and outputting an updated individualized treatment plan for the subject in response to the updated subject data. (Item 186) 2. The method, apparatus, or tangible medium of any one of the preceding items, wherein the diagnostic module and the treatment module each comprise a classifier trained on a population that does not include the subject. (Item 187) 2. The method of claim 1, wherein the processor comprises: A method, apparatus, or tangible medium. [Brief description of the drawings]

[0151] [Figure 1A] FIG. 1A is an exemplary system diagram for a digital personalized medicine platform.

[0152] [Figure 1B] FIG. 1B is a detailed diagram of an exemplary diagnostic module.

[0153] [Figure 1C] FIG. 1C is a diagram of an exemplary treatment module.

[0154] [Diagram 2] FIG. 2 illustrates an exemplary method for diagnosis and treatment to be provided in a digital personalized medicine platform.

[0155] [Diagram 3] FIG. 3 is an exemplary flow diagram illustrating the treatment of autism-related developmental delays. [Figure 4] FIG. 4 is an overall diagram of the data processing flow for a digital personalized medicine system, including diagnostic and therapeutic modules, configured to integrate information from multiple sources.

[0156] [Figure 5-1] 5A and 5B are diagrams showing some exemplary developmental disorders that can be diagnosed and treated using the methods for diagnosis and treatment as described herein. [Figure 5-2]5A and 5B are diagrams showing some exemplary developmental disorders that can be diagnosed and treated using the methods for diagnosis and treatment as described herein.

[0157] [Figure 6] FIG. 6 is a schematic diagram of an exemplary data processing module for providing a diagnostic test as described herein.

[0158] [Figure 7] FIG. 7 is a schematic diagram illustrating a portion of an exemplary evaluation model based on a random forest classifier.

[0159] [Figure 8] FIG. 8 is a diagram of an example operational flow of a prediction module as described herein.

[0160] [Figure 9] FIG. 9 is a diagram of an example operational flow of a feature recommendation module as described herein.

[0161] [Figure 10] FIG. 10 is a diagram of an example operational flow of a predicted feature importance determination algorithm as performed by the feature recommendation module described herein.

[0162] [Figure 11] FIG. 11 is a diagram illustrating a method for performing a diagnostic test as described herein.

[0163] [Figure 12] FIG. 12 illustrates an exemplary computer system suitable for incorporating with the methods and apparatus described herein.

[0164] [Figure 13]FIG. 13 is an exemplary system diagram for a digital personalized medicine platform including a feedback loop and reduced testing times.

[0165] [Figure 14] FIG. 14 illustrates a receiver operating characteristic (ROC) curve mapping sensitivity versus fallout for an exemplary evaluation model as described herein.

[0166] [Figure 15] FIG. 15 is a scatter plot illustrating performance metrics for a feature recommendation module as described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0167] Detailed Description of the Invention In one aspect, the digital personalized medicine system includes a digital device having a processor and associated software configured to receive data to evaluate and diagnose a patient, incorporate interaction and feedback data that identifies relative levels of efficacy, compliance, and response resulting from a therapeutic intervention, and perform data analytics, including at least one of machine learning, artificial intelligence, and statistical models, to evaluate user data and user profiles and further personalize, improve, or evaluate the effectiveness of the therapeutic intervention.

[0168] In some cases, the system is configured to use digital diagnostics and therapeutics. Digital diagnostics and therapeutics can include systems or methods that include collecting digital information and processing and analyzing the provided data to improve an individual's medical, psychological, or physiological condition. Digital therapeutics systems can apply software-based learning to analyze user data and monitor and improve diagnostic and therapeutic interventions provided by the system.

[0169] The digital diagnosis in the system may be composed of data and metadata collected from the patient, or caregiver, or a party unrelated to the individual being evaluated. In some cases, the collected data may include monitoring behavior, and the observations, judgments, or evaluations may be made by a party other than the individual. In further cases, the evaluation may involve an adult performing the evaluation or providing data for the evaluation of a child or minor.

[0170] The data source may be either an active or passive source in digital form via one or more digital devices such as a mobile phone, video capture, audio capture, activity monitor, or wearable digital monitor, etc. Examples of active data collection include devices, systems, or methods for tracking eye movements, recording body or appendage movements, monitoring sleep patterns, and recording speech patterns. In some instances, active sources can include audio feed data sources such as speech patterns, vocabulary / syntactic patterns (e.g., vocabulary size, correct / incorrect use of pronouns, correct / incorrect inflections and conjugations, use of grammatical structures such as active / passive voice, and sentence flow), higher order language patterns (e.g., coherence, comprehension, conversation participation, and curiosity), touchscreen data sources (e.g., fine motor function, dexterity, pointing accuracy and frequency, swiping accuracy and frequency, and focus / attention range), and video recordings of the subject's face during activity (e.g., quality / quantity of fixations versus saccades, eye focus heatmaps on the screen, focus / attention range, variability of facial expressions, and quality of responses to emotional stimuli). Passive data collection can include devices, systems, or methods for collecting data from users using mobile applications, sensor-embedded toys, or recordings or measurements derived from a recording unit. In some cases, passive sources may include sensors embedded in smart toys (e.g., fine motor function, gross motor function, focus / attention range, and problem-solving skills) and wearable devices (e.g., level of activity, amount / quality of rest).

[0171] The data used in diagnosis and treatment may come from multiple sources and may include a combination of passive and active data collection gathered from a single device, such as a mobile device with which a user interacts, or from other sources, such as microbiome sampling and genetic sampling of a subject.

[0172] The methods and devices disclosed herein are suitable for the diagnosis and digital therapeutic treatment of cognitive and developmental disorders, mood and psychiatric disorders, and neurodegenerative diseases. Examples of cognitive and developmental disorders include speech and learning disorders, intelligence quotient ("IQ"), non-verbal and verbal IQ, and other disorders as described herein. Examples of mood and psychiatric disorders that may affect children and adults include behavioral disorders, mood disorders, depression, attention deficit hyperactivity disorder ("ADHD"), obsessive-compulsive disorder ("OCD"), schizophrenia, and substance disorders such as eating disorders and drug abuse. Examples of neurodegenerative diseases include age-related cognitive decline, Alzheimer's disease and cognitive impairment that progresses with age, Parkinson's disease and Huntington's chorea, and amyotrophic lateral sclerosis ("ALS"). The methods and devices disclosed herein can digitally diagnose and treat children, continue treatment until the subject reaches adulthood, and provide lifelong treatment based on a personalized profile.

[0173] Digital diagnosis and treatment as described herein is suitable for behavioral intervention combined with biological or chemical therapeutic treatment. By collecting user interaction data as described herein, feedback effective treatment for combination of behavioral intervention data pharmaceutical treatment and biological treatment can be provided.

[0174] The mobile device as described herein may include sensors to collect subject data that can be used as part of a feedback loop to improve outcomes and reduce dependency on user input. The mobile device may include passive or active sensors as described herein to collect subject data after treatment. The same mobile device or a second mobile device, such as an iPad® or iPhone® or similar device, may include a software application that interacts with the user to teach the user what to do in the improvement procedure at regular intervals, such as daily, hourly, etc. The user mobile device may be configured to send notifications to the user according to the treatment progress. The mobile device may include a drug delivery device configured to monitor the delivery amount of the therapeutic agent delivered to the subject.

[0175] The methods and devices disclosed herein are suitable, for example, for treatment of both a parent and a child. Both the parent and the child may receive separate treatments as described herein. For example, the neurological condition of the parent may be monitored and treated, and the developmental progress of the child may be monitored and treated.

[0176] The mobile device used to acquire the subject's data may be configured in many ways, for example, combining multiple devices. Sleep patterns may be related to, for example, autism, and sleep data may be acquired and used as input to diagnostic and treatment modules as described herein. The mobile device may include a mobile wearable for sleep monitoring for children, which provides input for diagnosis and treatment, and may include feedback loop components as described herein.

[0177] Many types of sensors, biosensors, and data can be used to collect data of a subject and input it into the diagnosis and treatment of the subject. For example, research related to the embodiment suggests that microbiome data can be useful in the diagnosis and treatment of autism. Microbiome data can be collected in many ways known to those skilled in the art and can include data selected from fecal samples, enema, or other samples of the subject's intestinal flora. Genetic data can also be obtained as input to the diagnosis and treatment module. Genetic data can include the subject's full genome sequencing, sequencing and identification of specific markers.

[0178] The diagnostic and treatment module as disclosed herein can receive data from multiple sources, such as data obtained from the group consisting of genetic data, flora data, sleep sensors, ankle-wearable sleep sensors, booties to monitor sleep, and eye tracking of the subject. Eye tracking can be performed in many ways to determine the direction and duration of gaze. Tracking can be performed with glasses, helmets, other sensors for the direction and duration of gaze. Data can be obtained from any combination of games, video games, captured videos of the subject, which can be used to determine the subject's facial expression and gaze. This data can be obtained and provided to the treatment and diagnostic module as described herein before, during, and after treatment to initially diagnose the subject, determine the subject's treatment, modify the subject's treatment, and monitor the subject after treatment.

[0179] Information on visual gaze, duration of gaze, and facial expression may be acquired by methods and devices known in the art and may be acquired as input into the diagnostic and treatment module. The data may be acquired by an app that includes software instructions that may be downloaded. For example, facial processing is described in Gloarai et al., "Autism and Neuropsychiatry: A Guide to the Treatment of Neuropsychiatric Disorders," in "A ... The development of face processing, Clinical Neuroscience Research, vol. 6 (2006), pp. 145-160. The Autism Research Group at Duke University has been conducting research into autism and beyond with a software app downloaded onto a mobile device as described on the webpage at autismandbeyond.researchkit.duke.edu. Data from such devices is particularly suitable for combination according to the present disclosure. Face recognition data and gaze data may be input into diagnostic and treatment modules as described herein.

[0180] Classifiers as disclosed herein are particularly suitable for combining with this data to achieve improved treatments and procedures. The data can be stratified and used with feedback loops as described herein. For example, feedback data can be combined with drug therapy to determine differential response and distinguish between responders and non-responders. Alternatively, or in combination, feedback data can be combined with non-drug therapy, such as behavioral therapy.

[0181] With regard to genetics, recent studies suggest that some people may carry genes that predispose them to autism. A subject's genetic makeup may predispose the subject to environmental influences that may result in the onset of symptoms and affect the severity of symptoms. Environmental influences may include, for example, damage from toxins, viruses, or other substances. Without being bound to a particular theory, this may result in mechanisms that alter gene expression regulation. Changes in gene expression may be related to changes in gastrointestinal ("GI") flora, and these changes in flora may affect symptoms related to autism. Alternatively, or in combination, damage to the gut microbiome may result in changes in the subject's microbiome, resulting in the subject having less than ideal homeostasis, which may affect associated symptoms related to autism. The inventors have published a paper entitled "Methods of Gene Expression in the Human Genome," ... We note that preliminary studies with B. fragilis conducted by Mazmanian et al. suggest that changes in this microbe may be related to autism and the development of autism. (Melinda Wenner Moyer, "Gut Bacteria May Play a Role in Autism," Scientific (See also American, September 1, 2014)

[0182] Digital diagnostics uses data about the patient collected by the system, which may include complementary diagnostic data captured outside the digital diagnostics, along with analytical results from tools such as machine learning, artificial intelligence, and statistical modeling to assess or diagnose the patient's condition. Digital diagnostics also provide assessments of changes in the patient's condition or performance, directly or indirectly through data and metadata that may be analyzed by tools such as machine learning, artificial intelligence, and statistical modeling, thereby providing feedback to the system to improve or refine the diagnosis and potential therapeutic interventions.

[0183] Data evaluation and machine learning from digital diagnoses and corresponding responses, or lack thereof, from therapeutic interventions can result in the identification of novel diagnoses for patients and novel treatment regimens for both patients and caregivers.

[0184] The types of data collected and utilized by the system can include, for example, patient and caregiver video, audio, responses to questions or activities, as well as active or passive data streams from user interactions with the system's activities, games, or software features. Such data can also include metadata from patient or caregiver interactions with the system, for example, when performing recommended activities. Examples of specific metadata include data from user interactions with the system's devices or mobile apps that capture aspects of user behavior, profile, activities, interactions with software systems, interactions with games, frequency of use, session times, selected options or features, and content and activity preferences. Data can also include data and metadata from various third party devices, such as activity monitors, games, or interactive content.

[0185] Digital therapeutics as described herein may consist of instructions, feedback, activities, or interactions provided by the system to the patient or caregiver. Examples include suggested behaviors, activities, games, or interactive sessions with system software and / or third party devices (e.g., Internet of Things "IoT" enabled therapeutic devices as understood by one of the prior art in the art).

[0186] FIG. 1A shows a system diagram for a digital personalized medicine platform 100 for performing diagnosis and treatment related to behavioral, neurological or mental health disorders. For example, the platform 100 can perform diagnosis and treatment of pediatric cognitive and behavioral conditions associated with developmental delay. A user digital device 110 - e.g., a mobile device such as a smartphone, activity monitor, or wearable digital monitor - records data and metadata related to the patient. Data can be collected based on the patient's interaction with the device, as well as with caregivers and health care professionals. Data can be collected actively, such as by administering tests, recording speech and / or video, and recording responses to diagnostic questions. Data can also be collected passively, such as by monitoring the online behavior of the patient and caregiver, such as recording questions asked and topics explored related to the diagnosed developmental disorder.

[0187] The digital device 110 is connected to a computer network 120, which allows the device to share data with and receive data from the connected computers. In particular, the device can communicate with a personalized medicine system 130, which includes a server configured to communicate with the digital device 110 over the computer network 120. The personalized medicine system 130 includes a diagnostic module 132 for providing initial and incremental diagnosis of a patient's developmental condition, as well as a therapeutic module 134 for making personalized treatment recommendations in response to the diagnosis of the diagnostic module 132.

[0188] Each of the diagnostic modules 132 and 134 communicates with the user digital device 110 during the treatment process. The diagnostic modules provide diagnostic tests to the digital device 110, receive diagnostic feedback from the digital device 110, and use the feedback to determine a diagnosis for the patient. An initial diagnosis may be based on a comprehensive set of tests and questions, for example, while incremental updates may be made to the diagnosis using smaller data samples. For example, the diagnostic module may diagnose an autism-related speech delay based on questions asked of the caregiver and tests administered to the patient, such as vocabulary or oral communication tests. The diagnosis may indicate the number of months or years of delay in speech skills. Later tests may be administered and questions may be asked to update this diagnosis, for example, indicating a smaller or larger degree of delay.

[0189] The diagnostic module communicates its diagnosis to the digital device 110 and further to the therapy module 134, which uses the diagnosis to suggest a therapy to be performed to treat the diagnosed condition. The therapy module 134 sends its recommended therapy to the digital device 110, including instructions for the patient and caregiver to perform the recommended therapy in a given time frame. After performing the therapy in a given time frame, the caregiver or patient can indicate completion of the recommended therapy and a report can be sent from the digital device 110 to the therapy module 134. The therapy module 134 can then indicate to the diagnostic module 132 that the latest therapy has been completed and that a new diagnosis is required. The diagnostic module 132 can then provide new diagnostic tests and questions to the digital device 110, and even retrieve inputs of data provided as part of the therapy, such as viewing a record of a learning session or a caregiver or patient history related to the therapy or diagnosed condition, from the therapy module. The diagnostic module 132 then repeats the process and provides an updated diagnosis to perform the next step of the therapy.

[0190] Information related to diagnosis and treatment may also be provided from the personalized medicine system 130 to a third party system 140, such as a healthcare professional's computer system. The healthcare professional or other third party may be alerted to significant deviations from treatment schedules, including whether a patient is falling behind expected schedules or improving faster than expected. Appropriate further action may then be taken by the third party based on this provided information.

[0191] FIG. 1B illustrates a detailed view of the diagnosis module 132. The diagnosis module 132 includes a test administration module 142 that generates tests to administer to the subject and corresponding instructions. The diagnosis module 132 also includes a subject data receiving module 144 where subject data is received, such as test results, caregiver feedback, metadata from patient and caregiver interactions with the system, and video, audio, and gaming interactions with the system. The subject evaluation module 146 generates a diagnosis of the subject based on the data from the subject data receiving module 144 as well as past diagnoses of the subject and similar subjects. The machine learning module 148 evaluates the relative sensitivity of each input to the diagnosis to determine which type of measurement provides the most information about the patient's diagnosis. These results can be used by the test administration module 142 to provide tests that most efficiently inform the diagnosis, and by the subject evaluation module 146 to weight the diagnostic data to improve diagnostic accuracy and consistency. The diagnostic data related to each treated patient is stored, for example, in a database, forming a library of diagnostic data for pattern matching and machine learning. A large number of subject profiles, for example 10,000 or more subject profiles, may be stored simultaneously in such a database.

[0192] FIG. 1C illustrates a detailed view of the treatment module 134. The treatment module 134 includes a treatment evaluation module 152 that scores treatments based on their effectiveness. Previously suggested treatments are assessed based on the diagnosis provided by the diagnosis module both before and after the treatment to determine the degree of improvement. This degree of improvement is used to score the effectiveness of the treatment. A treatment may have an effectiveness that correlates with a particular class of diagnosis, for example, a treatment may be considered effective for a subject with one type of diagnosis but ineffective for a subject with a second type of diagnosis. A treatment matching module 154 is also implemented that compares the subject's diagnosis from the diagnosis module 132 with a list of treatments, thereby determining that the set of treatments determined by the treatment evaluation module 152 is most effective in treating a diagnosis similar to the subject's diagnosis. The treatment recommendation module 156 then generates a recommended treatment that includes one or more of the treatments identified as promising by the treatment matching module 154 and sends the recommendation to the subject along with instructions for implementing the recommended treatment. The treatment tracking module 158 then tracks the progress of the recommended treatment and determines when a new diagnosis should be performed by the diagnosis module 132 or when a given treatment should be continued and progress further monitored. The treatment data relating to each treated patient is stored, for example, in a database, forming a library of treatment data for pattern matching and machine learning. A large number of subject profiles, for example 10,000 or more, can be stored simultaneously in such a database. The treatment data is correlated with the diagnosis data of the diagnosis module 132, so that effective treatments and diagnoses can be matched.

[0193] Therapy may include digital therapeutics. Digital therapeutics may include single or multiple therapeutic activities or interventions that may be performed by the patient or caregiver. Digital therapeutics may include defined interactions with third party devices such as sensors, computers, medical devices, and therapeutic delivery systems. Digital therapeutics may support FDA approved medical claims, sets of diagnostic codes, single diagnostic codes.

[0194] 2 illustrates a method 200 for diagnosis and treatment to be provided in a digital personalized medicine platform. The digital personalized medicine platform communicates with a subject, which may include a patient with one or more caregivers, to perform a diagnosis and recommended treatment.

[0195] In step 210, the diagnostic module evaluates the subject to determine a diagnosis, for example, by applying a diagnostic test to the subject. The diagnostic test may be directed to determining a number of features and corresponding feature values ​​for the subject. For example, the test may include a number of questions presented to the subject, observations of the subject, or tasks assigned to the subject. The test may also include indirect testing of the subject, such as patient performance versus specific behaviors and / or milestones, caregiver feedback, metadata from patient and caregiver interactions with the system, and video, audio, and gaming interactions with the system or with third party tools that provide data on patient and caregiver behavior and performance. For initial tests, a more comprehensive test regimen may be performed aiming to generate an accurate initial diagnosis. Subsequent tests used to update previous diagnoses to track progress may involve less comprehensive tests and may rely more on indirect tests, such as behavior tracking and treatment relationship records and metadata.

[0196] In step 212, the diagnostic module receives new data from the subject. The new data may include an array of features and corresponding feature values ​​for a particular subject. As described herein, the features may include a number of questions presented to the subject, observations of the subject, or tasks assigned to the subject. The feature values ​​may include input data from the subject that corresponds to characteristics of the subject, such as the subject's answers or responses to questions asked. The feature values ​​may also include recorded feedback, metadata, and system interaction data as described above.

[0197] In step 214, the diagnostic module may load a previously saved assessment model from a local memory and / or a remote server configured to store the model. Alternatively, if no assessment model exists for the patient, a default model may be loaded, for example based on one or more initial diagnostic indications.

[0198] In step 216, the new data is fitted to the assessment model to generate an updated assessment model. This assessment model may include an initial diagnosis for a previously untreated subject, or an updated diagnosis for a previously treated subject. The updated diagnosis may include a measurement of progression in one or more aspects of the condition, such as memory, attention and joint attention, cognition, behavioral responses, emotional responses, language use, language skills, frequency of specific behaviors, sleep, sociality, non-verbal communication, and developmental milestones. Analysis of the data to determine progression and current diagnosis may include automated analysis, such as question scoring and speech recognition for vocabulary and speech analysis. Analysis may also include human scoring with analysis reviewing video, audio, and text data.

[0199] In step 218, the updated assessment model is provided to a treatment module, which determines what progress has been made as a result of the previously recommended treatments. The treatment module scores the treatments based on the degree of progress in the assessment model, with greater progress corresponding to a higher score and resulting in a successful treatment, making it more likely that a similar treatment will be recommended to the subject in future similar assessments. Thus, the set of available treatments is updated to reflect the new assessment of efficacy as it correlates with the subject's diagnosis.

[0200] In step 220, a new treatment is recommended based on the score assigned to the collection of candidate treatments based on the assessment model, the degree of success of previous treatments, if any, and previous use of those treatments with the subject and other subjects with similar assessments. The recommended treatment is sent to the subject for implementation along with instructions for a specific length of time to deliver it. For example, the treatment may include language drills to be performed with the patient every day for a week, with each drill recorded in an audio file in a mobile device used by the caregiver or the patient.

[0201] In step 222, the progress of the new treatment is monitored to determine whether the duration of the treatment should be extended. This monitoring may include periodic re-diagnosis, which may be performed by returning to step 210. Alternatively, basic milestones may be recorded without a full re-diagnosis, and the progress may be compared to a predicted progress schedule generated by the treatment module. For example, if the treatment is initially unsuccessful, the treatment module may suggest re-diagnosis and a new treatment, or to repeat one or more times before suggesting intervention by a medical professional.

[0202] FIG. 3 is a flow diagram 300 illustrating the treatment of suspected or confirmed speech and language delay.

[0203] In step 302, an initial assessment is determined by the diagnostic module 132. The initial assessment may assess the patient's performance in one or more areas, such as speech and language use, and assess the degree and type of developmental delay along multiple axes, as disclosed herein. The assessment may further place the subject on one of several overall progress tracks, for example, the subject may be assessed verbally or nonverbally.

[0204] If the subject is determined to be non-verbal, as in step 310, one or more non-verbal treatments 312 may be recommended by the treatment module 134, such as making choices, attending to tasks, or responding to names or other words. Further suggestions of useful devices and products that may aid in progress may also be made, and all suggestions may be modified to suit the subject's needs as dictated by the subject's diagnosis and progress report.

[0205] While administering the recommended treatment, progress is monitored in step 314 to determine if the diagnosis is improving at a predicted rate.

[0206] If improvement is measured in step 314, the system determines whether the subject is still non-verbal in step 316, and if so, the system returns to step 310 and generates a new recommended treatment 312 to provide further improvement.

[0207] If no improvement is measured in step 314, the system can recommend that the treatment be repeated a predetermined number of times. The system can also recommend trying changes in the treatment to try and get better results. If such repetitions and changes are not possible, the system can recommend a therapist visit in step 318 to more directly address the issues impeding development.

[0208] After the subject is determined to be verbal as indicated in step 320, a verbal treatment 322 may be generated by the treatment module 134. For example, the verbal treatment 322 may include one or more of language drills, vocal exercises, and expressive requests or communications. Further suggestions of useful devices and products that may aid in progression may also be made, and all suggestions may be modified to suit the subject's needs as indicated by the subject's diagnosis and progress report.

[0209] As with non-verbal tracking, progress in response to verbal therapy is continually monitored in step 324 to determine whether the diagnosis is improving at the expected rate.

[0210] If improvement is measured in step 324, the system reports on the progress in step 326 and generates new recommended treatments 322 to effect further improvement.

[0211] If no improvement is detected in step 324, the system may recommend that the treatment be repeated a predetermined number of times. The system may also recommend trying changes in the treatment to try and get better results. If such repetitions and changes are not possible, the system may recommend a therapist visit in step 328 to more directly address the issues impeding development.

[0212] The steps for non-verbal and linguistic therapy may be repeated as long as necessary to stimulate the subject's continued learning and progress, and to prevent or slow regression through loss of linguistic skills and abilities. Although the specific treatment plan illustrated in FIG. 3 is directed to pediatric speech and language development delay, similar plans may be generated for other subjects with developmental or cognitive problems, including plans for adult patients. For example, neurodegenerative conditions and / or age-related cognitive decline may be treated with similar diagnostic and treatment schedules, using treatments selected to suit such conditions. Additional conditions that may be treated in adult or pediatric patients by the methods and systems disclosed herein include mood disorders such as depression, OCD, and schizophrenia, cognitive impairment and decline, sleep disorders, addictive behaviors, eating disorders, and behavior-related weight management issues.

[0213] FIG. 4 is an overall diagram of a data processing flow for a digital personalized medicine system including a diagnostic module and a therapeutic module that is configured to integrate information from multiple sources. The data can include passive data sources (501), which may be configured to provide fine-grained information and can include data sets taken over a longer period of time in more natural conditions. Passive data sources can include, for example, data collected from a wearable device, data collected from a video feed (e.g., a video-enabled toy, a video feed collected from a mobile device), eye-tracking data from video footage, information about a subject's dexterity based on information gathered from a three-axis sensor or gyroscope (e.g., a sensor embedded in a toy or other device that a patient may interact with, for example, at home or under normal conditions outside of a medical setting), a smart device that measures any one or combination of a subject's speech patterns, movements, touch response times, prosody, lexical analysis, facial expressions, and other characteristics expressed by the subject. Passive data can include data regarding one or more movements of a user and can include subtle information that may or may not be easily detectable to an untrained individual. In some cases, passive data can provide information that may be more comprehensive.

[0214] Passively collected data may include data collected continuously from various environments. Passively collected data may paint a more complete picture of the subject and improve the quality of the assessment. For example, in some cases, passively collected data may include data collected both inside and outside the medical setting. Passively collected data taken in the medical setting may differ from passively collected data taken from outside the medical setting. Thus, continuously collected passive data may include a more complete picture of the subject's general behavior and mannerisms, and thus may include data or information that the medical practitioner may not otherwise have access to. For example, a subject being assessed in a medical setting may exhibit symptoms, gestures, or characteristics that represent the subject's response to the medical setting, and thus may not paint a complete and accurate picture of the subject's behavior outside the medical setting under more familiar conditions. The relative importance of one or more features derived from an assessment within a medical setting (e.g., features evaluated by a diagnostic module) may differ from the relative importance of one or more features derived from or evaluated outside the clinical setting.

[0215] The data can include information collected through diagnostic tests, diagnostic questions, or questionnaires (505). In some cases, data from diagnostic tests (505) can include data collected from a secondary observer (e.g., a parent, guardian, or an individual other than the subject being analyzed). The data can include data collected from active data sources (510), such as devices configured to track eye movements or measure or analyze speech patterns.

[0216] As illustrated in FIG. 4, the data input may be provided to a diagnostic module that may include data analysis (515) that uses, for example, a classifier, algorithm (e.g., machine learning algorithm), or statistical model to make a diagnosis of whether the subject is likely to have the tested disorder (e.g., autism spectrum disorder) (520) or unlikely to have the tested disorder (525). In cases where the subject is likely to have the disorder (520), a secondary party (e.g., a medical practitioner, parent, guardian, or other individual) may be presented with an informational display. The informational display may present symptoms of the disorder that may be displayed as a graph showing the covariance of symptoms displayed by the subject and symptoms displayed by the average population. A list of characteristics associated with a particular diagnosis may be displayed with a confidence value, correlation coefficient, or other means to display the relationship between the subject's performance and the average population or a population of those with similar disorders.

[0217] If the digital personalized medicine system predicts that the user is likely to have a diagnosable condition (e.g., autism spectrum disorder), the treatment module can provide behavioral treatment (530), which can include behavioral interventions, prescribed activities or training, interventions with medical devices or other treatment functions for a specific duration or at a specific time or instance. As the subject undergoes treatment, data (e.g., passive data and diagnostic question data) continues to be collected, so that follow-up evaluations can be performed, for example, to determine whether the treatment is working. The collected data can be subjected to data analysis (540) (e.g., machine learning, statistical modeling, classification tasks, predictive algorithms) to make decisions regarding the sustainability of a given subject. Growth curve displays can be used to show the subject's progress against a baseline (e.g., against an age-matched cohort). An individual's performance or progress can be measured, thereby tracking compliance for subjects with behavioral treatments suggested by the treatment module, which can be presented as historical and predicted performance on a growth curve. The procedure for assessing individual subject performance can be repeated or iterated until an appropriate behavioral treatment is identified ( 535 ).

[0218] The methods and apparatus for digital therapeutic treatment described with reference to Figures 1-4 are particularly suited for combination with methods and apparatus for assessing a subject with fewer questions described herein with reference to Figures 5A-14. For example, components of the diagnostic module 132 as described herein may be configured to assess a subject with a reduced set of questions including the most relevant questions as described herein, and then the subject may be assessed in the therapeutic module 134 and assessed with a subsequent set of questions including the most relevant questions for monitoring treatment as described herein.

[0219] 5A and 5B show some exemplary behavioral, neurological or mental health disorders that can be diagnosed and treated using the methods for diagnosis and treatment as described herein. Diagnostic tests can be configured to assess a subject's risk for having one or more behavioral, neurological or mental health disorders, such as two or more related behavioral, neurological or mental health disorders. The behavioral, neurological or mental health disorders can have at least some overlap in symptoms or characteristics of the subject. Such behavioral, neurological or mental health disorders may include pervasive developmental disorder (PDD), autism spectrum disorder (ASD), social communication disorder, restricted repetitive behaviors, interests, and activities (RRB), autism ("classic autism"), Asperger syndrome ("high-functioning autism"), pervasive developmental disorder not otherwise specified (PDD-NOS, "atypical autism"), attention deficit hyperactivity disorder (ADHD), speech and language development delay, obsessive-compulsive disorder (OCD), intellectual disability, learning disability, or other related developmental disorders, such as disorders defined in any edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM). The diagnostic test may be configured to determine the subject's risk for having each of the multiple disorders. The diagnostic test may be configured to determine the subject as having a greater risk of a first disorder or a second disorder of the multiple disorders. The diagnostic test can be configured to determine the subject as having a risk of a first disorder or a second disorder with comorbidity.The diagnostic test can be configured to predict that the subject has normal development or has a low risk of having any of the disorders that the procedure is configured to screen for.The diagnostic test can further be configured to have high sensitivity and specificity to distinguish different severity ratings for the disorder, for example, the procedure can be configured to predict the subject's risk of having level 1 ASD, level 2 ASD, or level 3 ASD as defined in the fifth edition of the DSM (DSM-V).

[0220] Many behavioral, neurological or mental health disorders may have similar or overlapping symptoms, which may complicate the evaluation of a subject's developmental disorder. The diagnostic test described herein may be configured to assess multiple characteristics of a subject that may be associated with one or more behavioral, neurological or mental health disorders. The procedure may include an assessment model that is trained using a large set of clinically validated data to learn statistical relationships between the characteristics of the subject and clinical diagnoses of one or more behavioral, neurological or mental health disorders. Thus, when a subject participates in a diagnostic test, the subject's feature values ​​for each assessed feature (e.g., the subject's answers to questions) are queried against the assessment model, thereby identifying statistical correlations, if any, between the subject's feature values ​​and one or more screened behavioral, neurological or mental health disorders. Based on the feature values ​​provided by the subject and the relationship between those values ​​and the predicted risk for one or more behavioral, neurological or mental health disorders as determined by the assessment model, the diagnostic test may dynamically adjust the selection of the next feature to be assessed in the subject. The selection of the next feature to be assessed may include identifying the next most predictive feature based on determining the subject as being at risk for a particular disorder among the multiple disorders being screened.For example, if the assessment model predicts that the subject is at low risk for autism and at relatively high risk for ADHD after the subject answers the first five questions of the diagnostic test, the diagnostic test may select the feature with a higher degree of association with ADHD to be assessed next for the subject (e.g., the question whose answer has a relatively high correlation with the clinical diagnosis of ADHD may be presented next to the subject).Therefore, the diagnostic test described herein may dynamically adjust to the risk profile of a particular subject, allowing the assessment of the disorder of the subject with a high level of granularity.

[0221] FIG. 6 is a schematic diagram of an exemplary data processing module 600 for providing an assessment procedure for screening a subject for cognitive function as described herein, which may include one or more of a plurality of behavioral, neurological, or mental health disorders or conditions. The assessment procedure may assess a plurality of characteristics or properties of the subject related to cognitive function, and each characteristic may relate to a probability that the subject has at least one of a plurality of behavioral, neurological, or mental health disorders that can be screened by the procedure, for example. The assessment procedure may be performed on the subject or the subject's caregiver through a user interface provided by a computing device. In some examples, the assessment procedure may take less than 60 minutes, 45 minutes, 30 minutes, 20 minutes, 10 minutes, or less to administer to the subject. In some examples, the data processing module 600 may be at least a part of a diagnostic module as described herein. The data processing module 600 generally includes a pre-processing module 605, a training module 610, and a prediction module 620. The data processing module can extract training data 650 from a database or import new data 655 via a user interface 630. The processing module can apply one or more transformations to standardize the training data or new data for the training module or the prediction module. The processed training data can be passed to the training module to build an assessment model 660 based on the training data. The training module can further include a validation module 615 configured to validate the trained assessment model using any suitable validation algorithm (e.g., stratified K-fold cross-validation). The pre-processed new data can be passed to a prediction module, which can output a prediction 670 of the subject's developmental disorder by fitting the new data to the assessment model built in the training module. The prediction module can further include a feature recommendation module 625 configured to select or recommend the next feature to be assessed in the subject based on previously provided feature values ​​for the subject.

[0222] The training data 650 used by the training module to build the assessment model can include multiple data sets from multiple subjects, with each subject's data set including an array of features and corresponding feature values ​​and a classification of the subject's developmental disorder or condition. As described herein, the features may be assessed in the subject through one or more of questions asked of the subject, observations of the subject, or structured interactions with the subject. The feature values ​​may include, for example, one or more of answers to questions, observations of the subject, such as characterizations based on video images, or responses of the subject to the structured interactions. Each feature may be associated with the identification of one or more behavioral, neurological, or mental health disorders or conditions, and each corresponding feature value may indicate the degree of presence of the feature in a particular subject. For example, the feature may be that the subject is able to engage in imaginative or pretend play, and the feature value for a particular subject may be a score of either 0, 1, 2, 3, or 8, with each score corresponding to the degree of presence of the feature in the subject (e.g., 0=various pretend play, 1=some pretend play, 2=occasional or very repetitive pretend play, 3=no pretend play, 8=not applicable). The feature may be assessed in the subject using questions presented to the subject, such as a patient, or a caregiver, and the answers to the questions comprise the feature value. Alternatively, or in combination, the feature may be observed in the subject, for example, in a video of the subject engaging in a particular behavior, and the feature value may be identified through observation. In addition to the array of features and corresponding feature values, the dataset for each subject in the training data also includes a classification of the subject. For example, the classification may be autistic, autism spectrum disorder (ASD), or non-spectrum. Preferably, the classification includes a clinical diagnosis, assigned by a qualified individual, such as a licensed clinical psychologist, to improve the predictive accuracy of the generated assessment model.The training data may include datasets available from large data repositories, such as Autism Diagnostic Interview-Revised (ADI-R) data and / or Autism Diagnostic Observation Test (ADOS) data available from the Autism Genetic Resource Exchange (AGRE), or any other suitable repository of data (e.g., Boston Autism Consortium (AC), Simons Foundation, National Database for Autism Research, etc.). Alternatively, or in combination, the training data may include large self-report datasets that may be crowdsourced from users (e.g., via websites, mobile applications, etc.).

[0223] The pre-processing module 605 may be configured to, for example, apply one or more transformations to the extracted training data to clean and normalize the data. The pre-processing module may be configured to discard features that contain spurious metadata or have very few observations. The pre-processing module may further be configured to standardize the encoding of feature values. Different datasets often have the same feature values ​​encoded differently depending on the source of the dataset. For example, "900", "900.0", "904", "904.0", "-1", "-1.0", "None", "NaN" may all code for a "missing" feature value. The pre-processing module may be configured to recognize encoding variants for the same feature value and standardize the dataset to have a uniform encoding for a given feature value. The pre-processing module may thus reduce irregular values ​​in the input data to the training and prediction modules, thereby improving the robustness of the training and prediction modules.

[0224] In addition to standardizing the data, the preprocessing module may also be configured to re-encode some feature values ​​into different data representations. In some cases, the original data representation of the feature values ​​in the dataset is less than ideal for building an evaluation model. For example, for a categorical feature whose corresponding feature value is coded as an integer from 1 to 9, each integer value may have a different semantic content that is independent of other values. For example, a value of "1" and a value of "9" may both be highly correlated with a particular classification, but a value of "5" is not. The original data representation of the feature values, in which the feature values ​​are coded as integers themselves, may not be able to capture the unique semantic content of each value because the values ​​are represented in a linear model (e.g., a response of "5" would place the subject squarely between "1" and "9" when the feature is considered in isolation, but such an interpretation is incorrect in the aforementioned case, where "1" and "9" are highly correlated with a given classification, but "5" is not). To ensure that the semantic content of each feature value is captured in constructing the assessment model, the pre-processing module may include instructions to re-encode some feature values, such as those corresponding to categorical features, in a "one-hot" manner. In a "one-hot" representation, the feature values ​​may be represented as an array of bits having values ​​of 0 or 1, with the number of bits corresponding to the number of possible values ​​for the feature. Only the feature value for the subject may be represented as "1" and all other values ​​may be represented as "0". For example, if the subject answered "4" to a question whose possible answers include integers from 1 to 9, then the original data representation may be [4] and the one-hot representation may be [0 0 0 1 0 0 0 0 0]. Such one-hot representations of feature values ​​may allow all values ​​to be considered, independent of other possible values, in cases where such representations are necessary. Thus, by re-encoding the training data using the most suitable data representation for each feature, the pre-processing module may improve the accuracy of an evaluation model built using the training data.

[0225] The pre-processing module may be further configured to impute any missing data values, so that downstream modules can process the data correctly. For example, if a training dataset provided to the training module includes data that is missing an answer to one of the questions, the pre-processing module may provide the missing values, so that the dataset may be correctly processed by the training module. Similarly, if a new dataset provided to the prediction module is missing one or more feature values ​​(e.g., the dataset being queried includes only an answer to the first question in a series of questions to be asked), the processing module may provide the missing values ​​to enable correct processing of the dataset by the prediction module. For features having categorical feature values ​​(e.g., the range of display of a particular behavior in a subject), the missing values ​​may be provided as an appropriate data representation specifically designated as such. For example, if the categorical feature is encoded in a one-hot representation as described herein, the pre-processing module may encode the missing categorical feature value as an array of "0" bits. For features having continuous feature values ​​(e.g., the age of the subject), the average of all possible values ​​may be provided in place of the missing value (e.g., 4 years old).

[0226] The training module 610 may, for example, utilize machine learning algorithms or other algorithms to construct and train an assessment model to be used in the diagnostic test. The assessment model may be constructed to capture statistical relationships, if any, between a given feature value and a particular developmental disorder to be screened by the diagnostic test based on the training data. The assessment model may, for example, include statistical correlations between multiple clinical characteristics and clinical diagnoses of one or more behavioral, neurological or mental health disorders. A given feature value may have different predictive utility for classifying each of the multiple behavioral, neurological or mental health disorders to be assessed in the diagnostic test. For example, in the above example of a feature including a subject's ability to engage in imaginative or pretend play, a feature value of "3" or "no pretend play variety" may have high predictive utility for classifying autism, while the same feature value may have low predictive utility for classifying ADHD. Thus, for each feature value, a probability distribution may be extracted that describes the probability of the particular feature value for predicting each of the multiple behavioral, neurological or mental health disorders to be screened in the diagnostic test. Machine learning algorithms can be used to extract these statistical relationships from training data and build assessment models that are capable of making accurate predictions of developmental disorders when a dataset containing one or more feature values ​​is fitted to the model.

[0227] One or more machine learning algorithms may be used to build the assessment model, such as a support vector machine that deploys stepwise backward feature selection and / or a graphical model, both of which may have the advantage of inferring interactions between features. For example, machine learning algorithms or other statistical algorithms may be used, such as alternating decision trees (ADTree), decision stumps, function trees (FT), logistic model trees (LMT), logistic regression, random forests, linear classifiers, or any machine learning or statistical algorithms known in the art. One or more algorithms may be used together to generate an ensemble method, and the ensemble method may be optimized using a machine learning ensemble meta-algorithm, such as boosting (AdaBoost, LPBoost, TotalBoost, BrownBoost, MadaBoost, LogitBoost, etc.) to reduce bias and / or variance. After the assessment model is derived from the training data, the model may be used as a predictive tool to assess the risk of a subject for having one or more behavioral, neurological, or mental health disorders. For example, machine learning analyses may be performed using one or more of many programming languages ​​and platforms known in the art, such as R, Weka, Python, and / or Matlab.

[0228] In general, a random forest classifier, which includes multiple decision trees whose output predictions are the modes of the predicted classifications of the individual trees, can help in reducing overfitting to the training data. An ensemble of decision trees can be built using a random subset of features at each split or decision node. A Gini criterion can be used to select the best partition, and the decision node with the lowest calculated Gini impurity index is selected. At prediction time, a "vote" can be taken across all of the decision trees, and the majority vote (or the mode of predicted classification) can be output as the predicted classification.

[0229] FIG. 7 is a schematic diagram illustrating a portion of an exemplary evaluation model 660 based on a random forest classifier. The evaluation module can include multiple individual decision trees 765, such as decision trees 765a and 765b, each of which can be generated independently using a random subset of features in the training data. Each decision tree can include one or more decision nodes, such as decision nodes 766 and 767 shown in FIG. 7, each specifying a predicate condition. For example, decision node 766 prescribes the condition that for a given data set of an individual, the answer to ADI-R question #86 (age when abnormality first became evident) is 4 or less. Decision node 767 prescribes the condition that for a given data set, the answer to ADI-R question #52 (pointing and directional attention) is 8 or less. At each decision node, the decision tree may be split based on whether the predicate condition attached to the decision node is true, leading to a prediction node (e.g., 766a, 766b, 767a, 767b). Each prediction node may include an output value ("Value" in FIG. 7) that represents a "vote" for one or more of the classifications or conditions assessed by the evaluation model. For example, in the prediction node shown in FIG. 7, the output value includes a vote for an individual being classified as having autism or being non-spectrum. The prediction node may lead to one or more additional decision nodes downstream (not shown in FIG. 7), each of which results in an additional split in the decision tree associated with a corresponding prediction node having a corresponding output value. Gini impurity may be used as a criterion to find informative features based on which splits may be constructed within each decision tree.

[0230] When the data set being queried in the evaluation model reaches a "leaf", or a final prediction node without further downstream splits, the output value of the leaf may be output as a vote for a particular decision tree. Since the random forest model includes multiple decision trees, the final votes across all trees in the forest may be summed to yield a final vote and a corresponding classification of the subject. Although only two decision trees are shown in FIG. 7, the model may include any number of decision trees. A large number of decision trees may help reduce overfitting of the evaluation model to the training data by reducing the variance of each individual decision tree. For example, the evaluation model may include at least about 10 decision trees, for example, at least about 100 or more individual decision trees.

[0231] An ensemble of linear classifiers may also be suitable for deriving an evaluation model as described herein. Each linear classifier may be trained separately by stochastic gradient descent without an "intercept term". The absence of an intercept term may prevent the classifier from deriving significance from missing feature values. For example, if a subject does not answer a question such that the feature value corresponding to said question is represented as an array of "0" bits in the subject's dataset, a linear classifier trained without an intercept term will not attribute significance to the array of "0" bits. The resulting evaluation model may thereby avoid establishing a correlation between the selection of features or questions answered by the subject and the subject's final classification as determined by the model. Such an algorithm may help ensure that only the feature values ​​or answers provided by the subject, and not the features or questions, are factored into the subject's final classification.

[0232] The training module may include feature selection. One or more feature selection algorithms (such as support vector machines, convolutional neural nets, etc.) may be used to select features that can distinguish individuals with or without some behavioral, neurological or mental health disorder. Different sets of features may be selected as relevant for identifying different disorders. A step-by-step backward algorithm may be used along with other algorithms. The feature selection procedure may include determining an optimal number of features.

[0233] The training module may be configured to assess the performance of the derived evaluation model. For example, the accuracy, sensitivity, and specificity of the model in classifying data may be assessed. This assessment may be used as a guideline for selecting a suitable machine learning algorithm or its parameters. The training module may thus update and / or refine the derived evaluation model to maximize specificity (true negative rate) relative to sensitivity (true positive rate). Such maximization may be particularly useful when class imbalance or sample bias exists in the training data.

[0234] In at least some cases, the available training data may be biased towards individuals diagnosed with a particular developmental disorder. In such cases, the training data forms an assessment model that reflects the sample bias, such that the model assumes that the subject is at risk for a particular developmental disorder unless strongly asserted otherwise. An assessment model that incorporates such a particular sample bias may have less than ideal performance in generating predictions for new or unclassified data, because the new data may be drawn from a subject population that may not contain similar sample biases as those present in the training data. To reduce sample bias when constructing an assessment model using biased training data, sample weighting may be applied when training the assessment model. Sample weighting may include attaching a relatively large significance level to a particular set of samples in the model training process. For example, in model training, if the training data is biased towards individuals diagnosed with autism, a higher significance may be attributed to data from individuals not diagnosed with autism (e.g., up to 50 times higher significance than data from individuals diagnosed with autism). Such sample weighting techniques can substantially balance the sample bias present in training data, thereby forming an evaluation model with reduced bias and improved accuracy in classifying data in the real world. To further reduce training data sample bias from entering into the generation of the evaluation model, boosting techniques can be implemented in the training process. Boosting involves an iterative process, and after one iteration of training, the weight of each sample data point is updated. For example, samples that are misclassified after the iteration can be updated with higher significance. The training process can then be repeated with the updated weights for the training data.

[0235] The training module may further include a validation module 615 configured to validate the evaluation model constructed using the training data. For example, the validation module may be configured to implement stratified K-fold cross-validation, where k represents the number of partitions into which the training data is divided for cross-validation. For example, k may be an integer greater than 1, such as 3, 4, 5, 6, 7, 8, 9, or 10, or possibly higher, depending on the risk of overfitting the evaluation model to the training data.

[0236] The training module stores the trained evaluation model in local memory and / or on a remote server, and the model can be retrieved for modification by the training module or for generating predictions by the prediction module 620.

[0237] FIG. 8 is an example operational flow 800 of a method of the prediction module 620 as described herein. The prediction module 620 may be configured to generate a predicted classification (e.g., developmental disorder) for a given subject by fitting new data to the assessment model constructed in the training module. In step 805, the prediction module may receive new data that may have been processed by a pre-processing module to standardize the data, for example, by removing spurious metadata, applying uniform encoding of feature values, re-encoding selected features using a different data representation, and / or imputing missing data points, as described herein. The new data may include an array of features and corresponding feature values ​​for a particular subject. As described herein, the features may include a number of questions presented to the subject, observations of the subject, or challenges assigned to the subject. The feature values ​​may include input data from the subject that correspond to characteristics of the subject, such as the subject's answers or the subject's responses to questions asked. The new data provided to the prediction module may or may not have a known classification or diagnosis associated with the data, and in either case, the prediction module may not use pre-assigned classification information in generating a predicted classification for the subject. The new data may include a previously collected complete data set for a subject to be diagnosed or assessed for risk of having one or more of a plurality of behavioral, neurological or mental health disorders. Alternatively, or in combination, the new data may include data collected in real time from the subject or the subject's caregiver, for example, by a user interface as described in more detail herein, whereby the complete data set may be populated in real time as each new feature value provided by the subject is sequentially queried against the assessment model.

[0238] In step 810, the prediction module can load a previously saved assessment model built by the training module from a local memory and / or a remote server configured to store the model. In step 815, new data is fitted to the assessment model to generate a predicted classification of the subject. In step 820, the module can check whether the data fit can generate a prediction of one or more specific disorders (e.g., autism, ADHD, etc.) within a confidence interval above a threshold, e.g., within a 90% or higher confidence interval, e.g., within a 95% or higher confidence interval. If so, as shown in step 825, the prediction module can output one or more behavioral, neurological or mental health disorders as the subject's diagnosis or as disorders for which the subject is at risk. The prediction module can output multiple behavioral, neurological or mental health disorders for which the subject is determined to have a risk above a set threshold, optionally presenting multiple disorders in order of risk. The prediction module can output one developmental disorder for which the subject is determined to have the greatest risk. The prediction module may output two or more developmental disorders for which the subject is determined to be at risk for comorbidity. The prediction module may output a determined risk for each of one or more behavioral, neurological or mental health disorders in the assessment model. If the prediction module is unable to assign the data to a particular developmental disorder within a confidence interval at or above a specified threshold, the prediction module may determine in step 830 whether there are additional features that can be queried. If the new data includes a complete dataset previously collected and the subject cannot be queried for any additional feature values, then "no diagnosis" may be output as the predicted classification, as shown in step 840.If new data includes data collected from the subject or caregiver in real time during the prediction process, the data set is updated with each new input data value provided to the prediction module, and each updated data set is fitted to the assessment model, the prediction module may be able to query the subject for additional feature values. If the prediction module has already obtained data for all features included in the assessment module, the prediction module may output "no diagnosis" as the predicted classification of the subject, as shown in step 840. If there are features not yet presented to the subject, as shown in step 835, the prediction module may obtain additional input data values ​​from the subject, for example, by presenting additional questions to the subject. The updated data set including the additional input data is then fitted again to the assessment model (step 815), and the loop may continue until the prediction module is able to generate an output.

[0239] FIG. 9 is an exemplary operational flow 900 of the feature recommendation module 625 as described herein with non-limiting examples. The prediction module may include a feature recommendation module 625 configured to identify, select, or recommend the next most predictive or relevant features to be assessed in a subject based on previously provided feature values ​​for the subject. For example, the feature recommendation module may be a question recommendation module, where the module may select the most predictive next question to be presented to the subject or caregiver based on the answer to the previously presented question. The feature recommendation module may be configured to recommend one or more next questions or features that have the highest predictive utility in classifying a developmental disorder for a particular subject. Thus, the feature recommendation module may help dynamically tailor the assessment procedure to the subject, enabling the prediction module to generate predictions with reduced assessment length and improved sensitivity and accuracy. Additionally, the feature recommendation module may help improve the specificity of the final predictions generated by the prediction module by selecting features to be presented to a subject that are most relevant in predicting one or more particular behavioral, neurological, or mental health disorders that a particular subject is most likely to suffer from based on previously provided feature values ​​by the subject.

[0240] In step 905, the feature recommendation module can receive as input data already obtained from the subject in the assessment procedure. The input subject data can include an array of features and corresponding feature values ​​provided by the subject. In step 910, the feature recommendation module can select one or more features to be considered as "candidate features" for recommendation as the next feature to be presented to one or more of the subject, the caregiver, or the physician. Features that have already been presented can be removed from the group of candidate features to be considered. Optionally, additional features that meet certain criteria can also be removed from the group of candidate features, as described in more detail herein.

[0241] In step 915, the feature recommendation module may assess the "expected feature importance" of each candidate feature. Candidate features may be assessed for their "expected feature importance," or the estimated usefulness of each candidate feature in predicting a particular developmental disorder for a particular subject. The feature recommendation module may utilize an algorithm based on (1) the importance or relevance of a particular feature value in predicting a particular developmental disorder, and (2) the probability that a subject may provide a particular feature value. For example, if an answer of "3" to ADOS question B5 is highly correlated with a classification of autism, this answer may be considered a feature value that has high utility for predicting autism. If the current subject also has a high probability of answering "3" to question B5, the feature recommendation module may determine that this question has high expected feature importance. Algorithms that may be used to determine expected feature importance of a feature are described in further detail, for example, with reference to FIG. 10.

[0242] In step 920, the feature recommendation module can select one or more candidate features to be next presented to the subject based on the expected feature importance of the features as determined in step 915. For example, the expected feature importance of each candidate feature can be expressed as a score or real number, which can then be ranked compared to other candidate features. The candidate feature having a desired rank, e.g., top 10, top 5, top 3, top 2, or highest rank, can be selected as the feature for next presentation to the subject.

[0243] FIG. 10 is an example operational flow 1000 of a method for determining expected feature importance determination algorithm 627 as performed by feature recommendation module 625 described herein.

[0244] In step 1005, the algorithm can determine the importance or relevance of a particular feature value in predicting a particular developmental disorder. The importance or relevance of a particular feature value in predicting a particular developmental disorder can be derived from an assessment model constructed using training data. Such "feature value importance" can be conceptualized as a measure of how relevant a role of a given feature value, whether present or absent, is in determining the final classification of a subject. For example, if the assessment model includes a random forest classifier, the importance of a particular feature value can vary depending on where the feature is located within the branch of the random forest classifier. In general, a feature can have a relatively high feature importance if its average position within the decision tree is relatively high. The feature value importance can be efficiently calculated by either the feature recommendation module or the training module given a particular assessment model, and the training module can pass the calculated statistics to the feature recommendation module. Alternatively, the importance of a particular feature value can vary depending on the actual prediction confidence that would result if the feature value was provided by the subject. For each possible feature value for a given candidate feature, the feature recommendation module may be configured to calculate an actual predictive confidence for predicting one or more behavioral, neurological or mental health disorders based on the subject's previously provided feature values ​​and currently assumed feature values.

[0245] Each feature value may have a different importance for each developmental disorder that the assessment procedure is designed to screen for. Thus, the importance of each feature value may be expressed as a probability distribution that describes the probability that the feature value will produce an accurate prediction for each of multiple behavioral, neurological, or mental health disorders that are assessed.

[0246] In step 1010, the feature recommendation module can determine the probability that a subject will provide each feature value. The probability that a subject may provide a particular feature value can be calculated using an appropriate statistical model. For example, a large probabilistic graphical model can be used to evaluate an expression such as: prob(E=1|A=1,B=2,C=1) where A, B, and C represent different features or questions in the prediction module, and the integers 1 and 2 represent different possible feature values ​​for the features (or possible answers to the question). The probability that a subject will provide a particular feature value can then be calculated using Bayes' rule, with a formula such as: prob(E=1|A=1,B=2,C=1)=prob(E=1,A=1,B=2,C=1) / prob(A=1,B=2,C=1) Such a formula can be computationally expensive, both in terms of computation time and processing resources required. Alternatively, or in combination with explicitly using Bayes' rule to calculate probabilities, logistic regression or other statistical estimators may be used, with probabilities estimated using parameters derived from machine learning algorithms. For example, the following formula may be used to estimate the probability that a subject may provide a particular feature value: prob(E=1|A=1,B=2,C=1)≒sigmoid(a1*A+a2*B+a3*C+a4) a1, a2, a3, and a4 are constant coefficients determined from a trained evaluation model that is learned using an optimization algorithm that attempts to make this formula as correct as possible, and sigmoid is a non-linear function that allows this formula to be turned into a probability. Such an algorithm can be trained quickly, and the resulting formula can be quickly calculated in an application, for example, when performing an evaluation procedure. Although four coefficients are referenced, as many useful coefficients can be used, as one skilled in the art will appreciate.

[0247] In step 1015, a predicted importance for each feature value may be determined based on a combination of the metrics calculated in steps 1005 and 1010. Based on these two coefficients, the feature recommendation module may determine the predicted usefulness of a particular feature value in predicting a particular developmental disorder. Although reference is made herein to determining the predicted importance via multiplication, the predicted importance may be determined by combining the coefficients and parameters in many ways, such as, for example, using a lookup table, logic, or division.

[0248] Steps 1005-1015 may be repeated for all possible features for each candidate feature, in step 1020. For example, if a particular question has four possible answers, then the expected importance of each of the four possible answers is determined.

[0249] In step 1025, an overall predicted importance, or predicted feature importance, for each candidate feature may be determined. The predicted feature importance for each feature may be determined by summing the feature value importance of all possible feature values ​​for the feature, as determined in step 1020. By summing the predicted utility across all possible feature values ​​for a given feature in this way, the feature recommendation module can determine the overall predicted feature importance of the feature for predicting a particular developmental disorder in response to a previous answer.

[0250] In step 1030, steps 1005-1025 may be repeated for all candidate features being considered by the feature recommendation module. The candidate features may include a subset of possible features, such as questions. Thus, expected feature importance scores for all candidate features may be generated, and the candidate features may be ranked in order from highest expected feature importance to lowest expected feature importance.

[0251] Optionally, in addition to the two factors determined in steps 1005 and 1010, a third factor may also be considered when determining the importance of each feature value. Based on the subject's previously provided feature values, a probability of the subject having one or more of a plurality of behavioral, neurological or mental health disorders may be determined. Such probability may be determined based on a probability distribution stored in the evaluation model, which indicates the probability of the subject having each of a plurality of screened behavioral, neurological or mental health disorders based on the feature values ​​provided by the subject. When selecting the next feature to be presented to the subject, the algorithm may be configured to give a greater weight to the feature values ​​that are most important or most relevant to predicting the one or more behavioral, neurological or mental health disorders that the current subject is most likely to have. For example, if a subject's previously provided feature values ​​indicate that the probability that the subject has either intellectual disability or speech and language delay is higher than the probability that the subject has any of the other behavioral, neurological, or mental health disorders being assessed, the feature recommendation module may prioritize feature values ​​that have high importance for predicting either intellectual disability or speech and language delay over features that have high importance for predicting autism, ADHD, or other developmental disorder that the assessment is designed to screen for. Thus, the feature recommendation module can enable the prediction module to tailor the prediction process to the subject at hand, suggesting more features that have relevance to the subject's potential developmental disorder and that result in a final classification with greater granularity and confidence.

[0252] Although the above steps illustrate an example operational flow 1000 of the expected feature importance determination algorithm 627, one of ordinary skill in the art would recognize many variations based on the teachings described herein. These steps may be completed in a different order. Steps may be added or removed. Some of these steps may include sub-steps of other steps. Many of these steps may be repeated as many times as the user desires.

[0253] Next, an exemplary implementation of the feature recommendation module is described: Subject X provides answers (feature values) to questions (features) A, B, and C in an evaluation procedure. Target X={'A':1,'B':2,'C':1} The feature recommendation module can determine whether question D or question E should be presented next to maximize the predictive confidence with which a final classification or diagnosis can be reached. Given subject X's previous questions, the feature recommendation module determines the probability that subject X will provide each possible answer to each of questions D and E, as follows: prob(E=1|A=1,B=2,C=1)=0.1 prob(E=2|A=1,B=2,C=1)=0.9 prob(D=1|A=1,B=2,C=1)=0.7 prob(D=2|A=1,B=2,C=1)=0.3 The feature importance of each possible answer to each of questions D and E may be calculated based on the rating model as described. Alternatively, the feature importance of each possible answer to each of questions D and E may be calculated as the actual predicted confidence that would result if the subject gave a particular answer. The importance of each answer may be expressed using a range of values ​​on an appropriate numerical scale, for example: Importance (E=1)=1 Importance (E=2)=3 Importance (D=1)=2 Importance (D=2)=4 Based on the calculated probabilities and feature value importance, the feature recommendation module can calculate the expected feature importance for each question as follows: expected[importance(E)]=(prob(E=1|A=1,B=2,C=1)*importance(E=1) +(prob(E=2|A=1,B=2,C=1)*Importance(E=2) = 0.1*1 + 0.9*3 = 2.8 Prob[Importance(D)]=(prob(D=1|A=1,B=2,C=1)*Importance(D=1) +(prob(D=2|A=1,B=2,C=1)*Importance(D=2) =0.7*2+0.3*4 =2.6 Thus, the expected feature importance (also called relevance) from the answer to question E is determined to be higher than the feature importance of question D, even though question D generally has higher feature importance for its answers. Thus, the feature recommendation module can select question E as the next question to be presented to subject X.

[0254] When selecting the next best feature to be presented to the subject, the feature recommendation module 625 may be further configured to remove one or more candidate features from consideration if the candidate feature has a high covariance with features that have already been presented to the subject. The covariance of the different features may be determined based on the training data and may be stored in the evaluation model constructed by the training module. If the candidate feature has a high covariance with features that have already been presented, the candidate feature may add relatively little additional predictive utility and may therefore be omitted from future presentation to the subject to optimize the efficiency of the evaluation procedure.

[0255] The prediction module 620 may interact with a person (e.g., the subject or the subject's caregiver) participating in the assessment procedure via a user interface 630. The user interface may be provided with a user interface, such as a display of a computing device, such as a personal computer, tablet, or smartphone, that may enable a user to access the prediction module. The computing device may include a processor including instructions for providing a user interface, for example in the form of a mobile application. The user interface may be configured to display instructions to the user from the prediction module and / or receive input from the user using an input method provided by the computing device. Thus, a user may participate in the assessment procedure as described herein by interacting with the prediction module via the user interface, for example, by providing answers (feature values) in response to questions (features) posed by the prediction module. The user interface may be configured to perform the assessment procedure in real time, whereby the user answers one question at a time and the prediction module may select the next best question to ask based on recommendations made by the feature recommendation module. Alternatively, or in combination, the user interface may be configured to receive a complete set of new data from the user, for example, by allowing the user to upload a complete set of feature values ​​corresponding to a set of features.

[0256] As described herein, the characteristics of interest that are relevant to identifying one or more behavioral, neurological or mental health disorders can be assessed in a subject in many ways. For example, the subject or a caregiver or a physician may be asked a series of questions designed to assess the degree to which the characteristics of interest are present in the subject. The answers provided may then represent the corresponding characteristic values ​​of the subject. The user interface may be configured to present a series of questions to the subject (or a person participating in the assessment procedure on behalf of the subject), which may be dynamically selected from a set of candidate questions as described herein. Such a question-answer-based assessment procedure may be entirely performed by a machine, and thus may result in a very quick prediction of the developmental disorder of the subject.

[0257] Alternatively or in combination, the characteristics of interest in a subject may be assessed by, for example, observing the subject's behavior using a video of the subject. The user interface may be configured to allow the subject or the subject's caregiver to record or upload one or more videos of the subject. The video footage may then be analyzed by a qualified person to determine the subject's feature value for the characteristics of interest. Alternatively or in combination, the video analysis to determine the feature value may be performed by a machine. For example, the video analysis may include detecting objects (e.g., the subject, the spatial location of the subject, the face, eyes, mouth, hands, limbs, fingers, toes, feet, etc.) and then tracking the movements of those objects. The video analysis may infer the gender of the subject and / or the subject's proficiency in speech. The video analysis may identify the face as a whole or specific prominent parts of the face, such as the nose, eyes, lips, and mouth, to infer facial expressions and track these expressions over time. The video analysis may detect the eyes, limbs, fingers, toes, hands, feet, and track their movements over time to infer behavior. In some cases, the analysis may further infer the intent of the behavior, for example, a child being upset by noise or loud music, engaging in self-harming behavior, imitating another person's behavior, etc. Sound and / or voice recorded in the video file may also be analyzed. The analysis may infer the context of the subject's behavior. Sound / voice analysis may infer the subject's feeling. Analysis of the subject's video, performed by a human and / or machine, may generate feature values ​​for the features of interest, which may then be appropriately encoded so as to be input into a prediction module. A prediction of the subject's developmental disorder may then be generated based on fitting the subject's feature values ​​to an assessment model constructed using the training data.

[0258] Alternatively, or in combination, the target characteristic of the subject can be assessed through structured interaction with the subject. For example, the subject can be asked to play a game, such as a computer game, and the subject's performance on the game can be used to assess one or more characteristics of the subject. The subject can be given one or more stimuli (e.g., visual stimuli presented to the subject via a display), and the subject's response to the stimuli can be used to assess the target characteristic. The subject can be asked to perform a particular task (e.g., the subject can be asked to pop a soap bubble with his / her finger), and the subject's response to the request or the subject's ability to perform the requested task can be used to assess the target characteristic.

[0259] The methods and apparatus described herein may be configured in various ways to determine the next most predictive or relevant question. At least a portion of the software instructions as described herein may be configured to execute locally on the local device to provide a user interface, pose questions, and receive answers to the questions. The local device may be configured with software instructions in an application program interface (API) to query a remote server for the most predictive next question. The API may return the identified questions based on feature importance, for example, as described herein. Alternatively, or in combination, the local processor may be configured with instructions to determine the most predictive next question in response to the previous answer. For example, the prediction module 620 may include software instructions in a remote server, or software instructions in a local processor, and combinations thereof. Alternatively, or in combination, the feature recommendation module 625 may include software instructions in a remote server, or software instructions in a local processor, and combinations thereof, configured to determine, for example, the most predictive next question. An example operational flow 1000 of a method for determining an expected feature importance determination algorithm 627 as performed by the feature recommendation module 625 described herein may be performed, for example, by one or more processors as described herein.

[0260] FIG. 11 illustrates a method 1100 of performing an assessment procedure as described herein. The method 1100 may be performed by a user interface on a computing device, the computing device including a display and a user interface for receiving user input in response to instructions presented on the display. The user participating in the assessment procedure may be the subject himself or another person participating in the procedure on behalf of the subject, such as the subject's caregiver. In step 1105, an Nth question related to the Nth feature may be presented to the user by the display. In step 1110, a subject's answer including the corresponding Nth feature value may be received. In step 1115, the current subject's dataset may be updated to include the Nth feature value provided for the subject. In step 1120, the updated dataset may be fitted to the assessment model to generate a predicted classification. Step 1120 may be performed by a prediction module, as described herein. At step 1125, a check may be performed to determine whether fitting the data can produce a prediction of a specific developmental disorder (e.g., autism, ADHD, etc.) with sufficient confidence (e.g., within at least a 90% confidence interval). If so, the predicted developmental disorder may be displayed to the user, as shown at step 1130. If not, at step 1135, a check may be performed to determine whether there are additional features that can be queried. If yes, as shown at step 1140, the feature recommendation module may select the next feature to be presented to the user, and steps 1105-1125 may be repeated until a final prediction (e.g., a specific developmental disorder or "no diagnosis") can be displayed to the subject. If no additional features can be presented to the subject, "no diagnosis" may be displayed to the subject, as shown at step 1145.

[0261] Although the above steps illustrate an exemplary method 1100 for performing an evaluation procedure, one of ordinary skill in the art would recognize many variations based on the teachings described herein. The steps may be completed in a different order. Steps may be added or removed. Some of the steps may include sub-steps of other steps. Many of the steps may be repeated as many times as the user desires.

[0262] The present disclosure provides a computer control system that is programmed to implement the disclosed methods. Figure 12 shows a computer system 1201 suitable for incorporation with the methods and apparatus described herein. The computer system 1201 can process various aspects of the information of the present disclosure, such as questions and answers, responses, statistical analysis, etc. The computer system 1201 can be a computer system located remotely with respect to the user's electronic device or the electronic device. The electronic device can be a mobile electronic device.

[0263] The computer system 1201 includes a central processing unit (CPU, also referred to herein as a "processor" and a "computer processor") 1205, which may be a single-core or multi-core processor, or multiple processors for parallel processing. The computer system 1201 also includes memory or memory locations 1210 (e.g., random access memory, read-only memory, flash memory), an electronic storage unit 1215 (e.g., a hard disk), a communication interface 1220 (e.g., a network adapter) for communicating with one or more other systems, and peripheral devices 1225, such as cache, other memory, data storage devices, and / or electronic display adapters. The memory 1210, the storage unit 1215, the interface 1220, and the peripheral devices 1225 communicate with the CPU 1205 through a communication bus (solid lines), such as a motherboard. The storage unit 1215 may be a data storage unit (or data repository) for storing data. Computer system 1201 may be operatively coupled to a computer network ("network") 1230 with the aid of communication interface 1220. Network 1230 may be the Internet, an Internet and / or an extranet, or an intranet and / or an extranet in communication with the Internet. Network 1230, in some cases, is a remote communication and / or data network. Network 1230 may include one or more computer servers, which may enable distributed computing, such as cloud computing. Network 1230 may implement a peer-to-peer network, which in some cases, with the aid of computer system 1201, may enable devices coupled to computer system 1201 to operate as clients or servers.

[0264] The CPU 1205 can execute sequences of machine-readable instructions, which may be embodied in a program or software. The instructions may be stored in a memory location, such as the memory 1210. These instructions may be directed to the CPU 1205, which can then be programmed or otherwise configured to implement the methods of the present disclosure. Examples of operations performed by the CPU 1205 can include fetch, decode, execute, and write back.

[0265] The CPU 1205 may be part of a circuit, such as an integrated circuit. One or more other components of the system 1201 may be included in the circuit. In some cases, the circuit is an application specific integrated circuit (ASIC).

[0266] The storage unit 1215 can store files, such as drivers, libraries, and saved programs. The storage unit 1215 can store user data, such as user settings and user programs. The computer system 1201 can in some cases include one or more additional data storage units that are external to the computer system 1201, such as located on a remote server that communicates with the computer system 1201 through an intranet or the Internet.

[0267] The computer system 1201 can communicate with one or more remote computer systems through the network 1230. For example, the computer system 1201 can communicate with a remote computer system of a user (e.g., a parent). Examples of remote computer systems and mobile communication devices include personal computers (e.g., portable PCs), slate or tablet PCs (e.g., Apple® iPad®, Samsung® Galaxy Tab), phones, smartphones (e.g., Apple® iPhone®, Android-enabled devices, Blackberry®), personal digital assistants, wearable medical devices (e.g., Fitbits), or medical device monitors (e.g., seizure monitors). A user can access the computer system 1201 through the network 1230.

[0268] Methods as described herein may be implemented using machine (e.g., computer processor) executable code stored on an electronic storage location of computer system 1201, such as memory 1210 or electronic storage unit 1215. Machine executable or machine readable code may be provided in the form of software. During use, the code may be executed by processor 1205. In some cases, the code may be retrieved from storage unit 1215 and stored on memory 1210 for ready access by processor 1205. In some situations, electronic storage unit 1215 may be excluded and machine executable instructions are stored on memory 1210.

[0269] The code may be precompiled and configured for use with a machine having a processor adapted to execute the code, or it may be compiled at run time. The code may be provided in a programming language that may be selected to allow the code to be executed in a precompiled or compiled manner.

[0270] Aspects of the systems and methods provided herein, such as computer system 401, may be embodied in programming. Various aspects of the technology may be considered as "products" or "articles of manufacture" that typically take the form of machine (or processor) executable code and / or associated data carried or embodied on some type of machine-readable medium. The machine executable code may be stored on an electronic storage unit, such as a memory (e.g., read-only memory, random access memory, flash memory) or a hard disk. A "storage" type medium may include any or all of the tangible memory of a computer, processor or the like, or associated modules thereof, such as various semiconductor memories, tape drives, disk drives, and the like, which may provide non-transitory storage from time to time for software programming. All or portions of the software may be communicated from time to time over the Internet or various other remote communication networks. Such communication may, for example, enable loading of the software from one computer or processor to another, for example, from a management server or host computer to a computer platform of an application server. Thus, another type of medium that may contain software elements includes light waves, radio waves, and electromagnetic waves, such as those used on physical interfaces between local devices, through wired and optical landline networks, and on various air links. Physical elements that carry such waves, such as wired or wireless links, optical links, or the like, may also be considered as media containing software. As used herein, unless limited to non-transitory tangible "storage" media, terms such as computer or machine "readable medium" refer to media that participate in providing instructions to a processor for execution.

[0271] Thus, a machine-readable medium, such as a computer executable code, may take many forms, including but not limited to a tangible storage medium, a carrier wave medium, or a physical transmission medium. Non-volatile storage media include optical or magnetic disks, such as any of the storage devices of a computer or the like, such as may be used to implement databases, etc., shown in the drawings. Volatile storage media include dynamic memory, such as the main memory of such a computer platform. Tangible transmission media include coaxial cables, copper wire, and fiber optics, including the wires that comprise a bus within a computer system. Carrier wave transmission media may take the form of electric or electromagnetic signals, or sound or light waves, such as those generated during radio frequency (RF) and infrared (IR) data communications. Thus, common forms of computer readable media include, for example, floppy disks, flexible disks, hard disks, magnetic tape, any other magnetic media, CD-ROMs, DVDs or DVD-ROMs, any other optical media, punch cards, paper tape, other physical storage media with patterns of holes, RAM, ROM, PROMs and EPROMs, FLASH-EPROMs, any other memory chips or cartridges, carrier waves that transport data or instructions, cables or links that transport such carrier waves, or any other medium from which a computer may read programming code and / or data. Many of these forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to a processor for execution.

[0272] The computer system 1201 can include or communicate with an electronic display 1235 that includes a user interface (UI) 1240 to provide, for example, questions and answers, analysis results, recommendations. Examples of UIs include, without limitation, graphical user interfaces (GUIs) and web-based user interfaces.

[0273] The methods and systems of the present disclosure may be implemented using one or more algorithms and by instructions provided with one or more processors as disclosed herein. The algorithms may be implemented using software when executed by the central processing unit 1205. The algorithms may be, for example, random forests, graphical models, support vector machines, or others.

[0274] Although the above steps illustrate a method of the system by example, one of ordinary skill in the art would recognize many variations based on the teachings described herein. The steps may be completed in a different order. Steps may be added or removed. Some of the steps may include sub-steps. Many of the steps may be repeated as is advantageous for the platform.

[0275] Each of the examples as described herein may be combined with one or more of the other examples. Further, one or more components of one or more of the examples may be combined with the other examples.

[0276] 13 shows an exemplary system diagram for a digital personalized medicine platform 1300 with feedback loops and reduced testing. For example, the platform 1300 can diagnose and treat pediatric cognitive and behavioral conditions associated with developmental delay. A user digital device 110, e.g., a mobile device such as a smartphone, activity monitor, or wearable digital monitor, can record data and metadata related to the patient. Data can be collected based on the patient's interactions with the device, as well as interactions with caregivers and health care professionals, as described above.

[0277] The digital device 110 can be in communication with a personalized medicine system 130 over a communication network 120. The personalized medicine system 130 can include a diagnostic module 132 for providing initial and updated diagnosis of a patient's developing condition, and a therapeutic module 134 for making personalized treatment recommendations in response to the diagnosis of the diagnostic module 132.

[0278] In some cases, the diagnostic module 132 may include a data processing module as described herein. The data processing module may enable the diagnostic module 132 to perform an assessment on a subject with a reduced number of test questions. The data processing module may include a pre-processing module, a training module, and a prediction module as described herein. The data processing module may extract training data from a database or a user, apply one or more transformations to standardize the training data, and pass the standardized training data to the training module. The training module may utilize machine learning algorithms or other algorithms to build and train an assessment model to be used in the diagnostic test based on the standardized training data. After the assessment model is derived from the training data, the model may be used as a predictive tool to assess the subject's risk for one or more disorders, such as cognitive function, developmental improvement, or behavioral, neurological, or mental health disorders. The training data may include data created on a population where the subject patient is not a member of the population. The prediction module may be configured to generate a predicted classification of a given subject's cognitive function (e.g., developmental disorder) by fitting new data to the assessment model built in the training module. The data processing module, as described herein, identifies the most predictive next question based on multiple answers to multiple asked questions, so that a person can be diagnosed or identified as at risk and treated with fewer questions.

[0279] The diagnostic tests (e.g., sets of tests and questions) as generated from the diagnostic module 132 may be provided to the patient or caregiver via the digital device 110. The patient's responses to the diagnostic tests may be received by the diagnostic module 132. The diagnostic module 132 may generate an initial diagnosis based on the patient's responses. For example, the diagnostic module may diagnose autism-related speech delay based on questions asked of the caregiver and tests administered to the patient, such as vocabulary or oral communication tests.

[0280] The diagnostic module can communicate its initial diagnosis to the therapeutic module 134, which suggests an initial treatment to be performed to treat the condition diagnosed using the initial diagnosis. The therapeutic module 134 sends its recommended treatment to the digital device 110, including instructions for the patient and caregiver to perform the recommended treatment in a given time frame. The patient and caregiver can provide feedback to the diagnostic module 132, which can then instruct the data processing module to provide new diagnostic tests and questions to the digital device 110. The diagnostic module 132 then provides an updated diagnosis to the therapeutic module 134, which suggests an updated treatment to be performed by the patient and caregiver as the next step of treatment. Thus, a feedback loop can be formed between the patient and caregiver, the diagnostic module, and the therapeutic module, and the patient can be diagnosed with fewer questions. The feedback can identify the relative levels of effectiveness, compliance, and response resulting from the therapeutic intervention, allowing corrective changes to improve treatment.

[0281] In some cases, the treatment module may rely on the diagnosis module to classify subjects as having different conditions or different severity levels of a condition. Optionally, the treatment module can have an independent prediction or recommendation module to determine the next best treatment or treatment from a list of options. This decision can take into account the assessment from the diagnosis module as well as independently compiled statistics related to the historical probability of some patients responding to some treatments, subdivided by demographics such as gender / age / race / etc. The treatment module can perform the prediction task using simple rules or advanced machine learning techniques. In the case of machine learning, an independent feedback loop occurs, connecting patient treatment outcomes back to the treatment module.

[0282] In some cases, a third party system, such as a healthcare professional's computer system, may be connected to the communications network 120. The healthcare professional or other third party may be alerted to significant deviations from the treatment suggested by the treatment module based on the diagnosis and / or reduced number of questions provided by the diagnostic module. Appropriate further action may then be taken by the third party. For example, the third party system may review and modify the treatment suggested by the treatment module.

[0283] In some cases, a patient may have a response profile that responds to a treatment, and the treatment module may be configured to categorize the response profile based on the subject's initial response. For example, a subject may have a response profile that indicates that the treatment is working or that the treatment is not working. These initial response profiles may be somewhat counterintuitive. For example, fluctuations in symptoms may be an indicator that the treatment is working, even though these fluctuations may include increases and decreases in symptoms relative to baseline. For some treatments, the time when there is a change in symptoms may be delayed.

[0284] Users, such as patients and caregivers, can, for example, download and install apps containing software instructions on the digital device 110. The apps can enable the users to receive instructions from a cloud-based server for diagnostic tests, upload answers to the diagnostic tests, receive treatments from the cloud-based server (e.g., games or interactive content), provide feedback, receive new tests periodically to determine how the treatment is progressing, and receive updated treatments. The apps can be installed on multiple digital devices, such as a first device for the subject to receive digital treatment and a second device for the caregiver to monitor the progress of the treatment. Thus, a feedback loop is created between the user and the cloud-based server (e.g., personalized medicine system 130), and the subject's assessment after the initiation of treatment is used to adjust the treatment to improve response. Experimental Data

[0285] The data processing module as described herein was built on Python 2.7, Anaconda Distribution. The training data used to build and train the assessment model included data generated by the Autism Genetics Resource Exchange (AGRE), which performed an in-home assessment to collect ADI-R and ADOS data from parents and children in the home. The ADI-R includes a parent interview presenting a total of 93 questions, resulting in a diagnosis of autism or not autism. The ADOS includes a semi-structured interview of the child that results in a diagnosis of autism, ASD, or no autism, and the child is administered one of four possible modules based on language level, with each module containing approximately 30 questions. The data included the child's clinical diagnosis derived from the assessment, and in cases where a child had a conflicting ADI-R vs. ADOS diagnosis, a licensed clinical psychologist assigned a consensus diagnosis for the data set for the child in question. The training data contained a total of 3,449 data points, with 3,315 cases (autism or ASD) and 134 controls (non-spectrum). Features assessed in the training data targeted three key domains: language, social communication, and repetitive behaviors.

[0286] A boosted random forest classifier was used to build the evaluation model as described herein. Prior to training the evaluation model on the training data, the training data was preprocessed, thereby standardizing the data and recoding categorical features in a one-hot representation as described herein. Because the training data was biased towards individuals with autism or ASD, sample weights were applied to attribute up to 50 times higher significance to data from non-spectrum individuals compared to data from autism / ASD individuals. The evaluation model was iteratively trained with boosting, whereby the weights of data points were updated after each iteration to increase the significance attributed to misclassified data points, and retrained with the updated significance.

[0287] The trained model was validated using stratified k-fold cross-validation, with k=5. Cross-validation resulted in an accuracy of about 93-96%, where accuracy is defined as the percentage of subjects correctly classified using the model in a binary classification task (autism / non-spectrum). Because the training data contained sample bias, a confusion matrix was calculated to determine how many times the model confused one class (autism or non-spectrum) with another class. The percentage of correctly classified autistic individuals was about 95%, while the percentage of correctly classified non-spectrum individuals was about 76%. However, it should be noted that the model may be adjusted to more accurately fit one class against another, in which case the percentage of correct classifications for each class may change. FIG. 14 shows a receiver operating characteristic (ROC) curve that maps sensitivity versus fallout for an exemplary evaluation model as described herein. The true positive rate (sensitivity) for a diagnosis of autism is mapped on the y-axis, which varies according to the false positive rate (fallout) for the diagnosis, which is mapped on the x-axis. Each of the three curves, labeled "Fold#0", "Fold#1", and "Fold#2", corresponds to a different "fold" of the cross-validation procedure, and for each fold, a portion of the training data was fitted to the evaluation model while varying the predictive confidence threshold at which the dataset must be classified as "autism". As desired or required, the model can be adjusted to increase sensitivity at the expense of an increase in a portion of the fallout, or decrease sensitivity at the expense of a decrease in the fallout, depending on the model's ROC curve.

[0288] The feature recommendation module was configured as described herein, and the expected feature importance for each question was calculated, and the candidate questions were ranked in order of the calculated importance by a call to the server through an application program interface (API). The ability of the feature recommendation module to recommend informational questions was assessed by determining the correlation between the increase in prediction accuracy obtained from answering the recommended questions and the recommendation score of the questions. To calculate the correlation metric, the following steps were performed: (1) The data was split into several folds for cross-validation. (2) Questions that had already been answered were randomly removed from the validation set. (3) An expected feature importance (question recommendation / score) was generated for each question. (4) One of the questions removed in step 2 was revealed, and the relative improvement in prediction accuracy thereafter was measured. (5) The correlation between the relative improvement and the expected feature importance was calculated. The calculated Pearson correlation coefficient was between 0.2 and 0.3, indicating a moderate degree of correlation between the expected feature importance score and the relative improvement. Figure 15 is a scatter plot showing the correlation between predicted feature importance ("predicted informativeness score") and relative improvement ("relative classification improvement") for each question. The scatter plot shows a moderate linear relationship between the two variables, indicating that the feature recommendation module is indeed able to recommend questions that improve prediction accuracy.

[0289] The length of time it took to generate an output using the created prediction module and feature recommendation model was measured. The prediction module took approximately 46 ms to make a prediction of an individual's risk of autism. The feature recommendation module took approximately 41 ms to generate a question recommendation for the individual. These measurements were made by calls to a server through an API, although the calculations may be performed locally, for example.

[0290] Although the assessment models of the data processing modules described with respect to Figures 9-10 were constructed and trained to classify subjects as having or not having autism, as described herein, a similar approach may be used to construct assessment models that are capable of classifying subjects as having one or more of a number of behavioral, neurological or mental health disorders.

[0291] Those skilled in the art can generate and obtain additional data sets to improve the sensitivity and specificity and confidence intervals of the methods and devices disclosed herein to obtain improved results without undue experimentation. Although these measurements were performed with an exemplary data set, the methods and devices may be configured with additional data sets as described herein, and subjects were identified as at risk with a confidence interval of 80% in a clinical setting without undue experimentation. Sensitivity and specificity of 80% or higher in a clinical setting may similarly be obtained by those skilled in the art with the teachings presented herein, for example, using additional data sets, without undue experimentation.

[0292] The additional data set may be obtained from large archival data repositories as described herein, such as the Autism Genetic Resource Exchange (AGRE), Boston Autism Consortium (AC), Simons Foundation, National Database for Autism Research, and the like. Alternatively, or in combination, the additional data set may include mathematically simulated data generated based on the archival data using various simulation algorithms. Alternatively, or in combination, the additional data set may be obtained via crowdsourcing, where subjects self-perform an assessment procedure as described herein and provide data from the assessment. In addition to data from a self-performed assessment, subjects may also provide a clinical diagnosis obtained from a qualified physician, providing a standard of comparison for the assessment procedure.

[0293] Although the detailed description includes many details, these should not be construed as limiting the scope of the disclosure, but merely as illustrating different examples and aspects of the disclosure. It will be understood that the scope of the disclosure includes other embodiments not described in detail above. Various other modifications, changes, and variations that will be apparent to those skilled in the art may be made in the arrangement, operation, and details of the methods and apparatus of the present disclosure presented herein without departing from the spirit and scope of the invention as described herein. For example, one or more aspects, components, or methods of each of the examples as disclosed herein may be combined with others as described herein, and such modifications will be readily apparent to those skilled in the art. For each of the methods disclosed herein, those skilled in the art will appreciate many variations based on the teachings described herein. These steps may be completed in different orders. Steps may be added or deleted. Some of these steps may include sub-steps of other steps. Many of the steps may be repeated as many times as desired, and steps of the methods may be combined with each other.

[0294] While preferred embodiments of the present invention have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are presented by way of example only. Thus, numerous changes, modifications, and substitutions will occur to those skilled in the art without departing from the invention. It will be understood that various alternative forms of the embodiments of the invention described herein may be employed in carrying out the invention. It is intended that the following claims define the scope of the invention, and that methods and structures within the scope of these claims and their equivalents be covered thereby.

Claims

1. 1. A digital therapeutic system for treating a subject with an individualized therapeutic treatment plan, comprising: One or more processors configured to execute instructions contained in the non-transitory computer-readable storage medium to run application software. the application software comprising: a diagnostic module configured to receive subject data for use as input to a first machine learning, classification, artificial intelligence, or statistical modeling algorithm trained on subject population data to output diagnostic data including a risk of the subject for having a behavioral disorder, developmental delay, or neurological dysfunction, wherein the risk comprises at least an 85% confidence interval; and a feature recommendation module configured to generate the subject data, the subject data including features selected from a group of candidate features, the feature recommendation module configured to: (i) estimate a predictive utility of each feature in the group of candidate features for predicting the risk of the behavioral disorder, the developmental delay, or the neurological dysfunction; (ii) select the feature from the group of candidate features based on the estimated predictive utility; and (iii) obtain a feature value from the subject based on the feature from the group of candidate features, the feature value being used to update the estimated predictive utility of each feature in the group of candidate features; a treatment module configured to receive the diagnostic data for use as an input to a second machine learning, classifier, artificial intelligence, or statistical modeling algorithm for outputting the individualized therapeutic treatment plan for the subject, the second machine learning, classifier, artificial intelligence, or statistical modeling algorithm being trained on at least a portion of the subject population data to determine and output the individualized therapeutic treatment plan for the subject, the individualized therapeutic treatment plan including a digital therapeutic for treating the behavioral disorder, the developmental delay, or the neurological dysfunction, the digital therapeutic including an interactive session with a digital therapeutic device, during which the subject's interaction with the digital therapeutic device is used to update the subject data to generate updated subject data; and Including, The diagnostic module is configured to receive the updated subject data collected from the subject's interactions with the digital therapeutic device during the interactive session and generate updated diagnostic data based on the updated subject data; and the therapeutic module is configured to receive the updated diagnostic data and output an updated individualized treatment plan for the subject in response to the diagnostic data and the updated diagnostic data; The behavioral disorder, developmental delay, or neurological dysfunction comprises autism, autism spectrum disorder, attention deficit disorder, attention deficit hyperactivity disorder, or speech and learning disabilities.

2. The system described in claim 1, wherein the subject population data does not include the subject.

3. The system of claim 1 , wherein the subject population data comprises at least 10,000 treatment profiles.

4. 1. A system for diagnosing and treating a subject, the system comprising: a diagnostic module configured to receive subject data for use as input to a first machine learning, classification, artificial intelligence, or statistical modeling algorithm trained on subject population data to output diagnostic data comprising a risk of the subject for having a behavioral disorder, developmental delay, or neurological dysfunction in the subject, the risk comprising at least an 85% confidence interval; and a feature recommendation module configured to generate the subject data, the subject data including features selected from a group of candidate features, the feature recommendation module configured to estimate a predictive utility of each feature in the group of candidate features for predicting the risk of the behavioral disorder, the developmental delay, or the neurological dysfunction, and to select the features from the group of candidate features based on the estimated predictive utility, feature values ​​are obtained from the subject based on the features from the group of candidate features, the feature values ​​being used to update the estimated predictive utility of each feature in the group of candidate features; (1) receiving the diagnostic data for use as an input to a second machine learning, classifier, artificial intelligence, or statistical modeling algorithm for outputting an individualized therapeutic treatment plan including a digital therapeutic for the subject based at least on the diagnostic data; and (2) providing the individualized therapeutic treatment plan including the digital therapeutic for the subject to treat the behavioral disorder, the developmental delay, or the neurological dysfunction, the digital therapeutic including an interactive session with a digital therapeutic device, during which the subject's interaction with the digital therapeutic device is used to update the subject data to generate updated subject data. wherein the diagnostic module and the therapeutic module are arranged to use a feedback loop to update the individualized therapeutic treatment plan in response to updated diagnostic data generated by the diagnostic module based on the updated subject data collected from the subject's interactions with the digital therapeutic device during the interactive session; The behavioral disorder, developmental delay, or neurological dysfunction comprises autism, autism spectrum disorder, attention deficit disorder, attention deficit hyperactivity disorder, or speech and learning disability.

5. 1. A digital therapeutic system for treating a subject with an individualized therapeutic treatment plan, the system comprising: A processor configured to execute instructions contained in a non-transitory computer-readable storage medium to execute application software. The application software comprises: a diagnostic module for receiving subject data for use as input to a first machine learning, classification, artificial intelligence, or statistical modeling algorithm trained on subject population data to output diagnostic data including the subject's risk of having a behavioral disorder, developmental delay, or neurological dysfunction in the subject, the risk comprising at least an 85% confidence interval; and a feature recommendation module configured to generate the subject data, the subject data including features selected from a group of candidate features, the feature recommendation module configured to estimate a predictive utility of each feature in the group of candidate features for predicting the risk of the behavioral disorder, the developmental delay, or the neurological dysfunction, and to select the features from the group of candidate features based on the estimated predictive utility, feature values ​​are obtained from the subject based on the features from the group of candidate features, the feature values ​​being used to update the estimated predictive utility of each feature in the group of candidate features; a treatment module for receiving the diagnostic data for use as an input to a second machine learning, classifier, artificial intelligence, or statistical modeling algorithm for outputting the individualized therapeutic treatment plan for the subject, the second machine learning, classifier, artificial intelligence, or statistical modeling algorithm being trained on at least a portion of the subject population data to determine and output the individualized therapeutic treatment plan for the subject, the individualized therapeutic treatment plan including a digital therapeutic for treating the behavioral disorder, the developmental delay, or the neurological dysfunction, the digital therapeutic including an interactive session with a digital therapeutic device, during which the subject's interaction with the digital therapeutic device is used to update the subject data to generate updated subject data; and the diagnostic module configured to receive the updated subject data and generate updated diagnostic data based on the updated subject data; and the therapeutic module configured to receive the updated diagnostic data and output an updated individualized treatment plan for the subject in response to the updated diagnostic data; The behavioral disorder, developmental delay, or neurological dysfunction comprises autism, autism spectrum disorder, attention deficit disorder, attention deficit hyperactivity disorder, or speech and learning disability.

6. 6. The system of claim 5, wherein the digital therapeutic comprises instructions, feedback, activities, or interactions provided to the subject or caregiver.

7. 7. The system of claim 6, wherein the digital therapeutic is delivered using a mobile device.

8. The system of claim 5 , wherein the diagnostic data and the individualized therapeutic treatment plan are provided to a third party system.

9. The system of claim 8 , wherein the third party system comprises a healthcare professional's computer system or a treatment delivery system.

10. The system of claim 5 , wherein the updated subject data includes relative levels of efficacy, compliance, and response resulting from the individualized therapeutic treatment plan.

11. The system of claim 5 , wherein the subject data includes at least one of subject and caregiver video, audio, responses to questions or activities, and active or passive data streams from user interaction with activities, games, or software features of the system.

12. 6. The system of claim 5, wherein the diagnostic module is further configured for use by an adult to provide data for evaluation of a child or minor, the evaluation including data for use as input to the first machine learning, classification, artificial intelligence, or statistical modeling algorithm.

13. 6. The system of claim 5, wherein the diagnostic module is further configured for use by a caregiver or family member to provide data for evaluation of the subject, the evaluation including data for use as input into the first machine learning, classification, artificial intelligence, or statistical modeling algorithm.

14. 1. A method for treating a subject with an individualized therapeutic treatment plan, the method being implemented in a computer system comprising a processor, the method comprising: and estimating a predictive utility of each feature in a group of candidate features for predicting risk of behavioral disorder, developmental delay, or neurological dysfunction, the risk comprising at least an 85% confidence interval; the processor selecting features from the set of candidate features based on the estimated predictive utility to generate subject data, feature values ​​being obtained from the subject based on the features from the set of candidate features, the feature values ​​being used to update the estimated predictive utility of each feature in the set of candidate features; receiving the subject data for use as input to a first machine learning, classification, artificial intelligence, or statistical modeling algorithm trained on subject population data to output diagnostic data comprising the subject's risk of having a behavioral disorder, developmental delay, or neurological dysfunction; providing the diagnostic data for use as input to a second machine learning, classifier, artificial intelligence, or statistical modeling algorithm; the processor outputs the individualized therapeutic treatment plan for the subject, wherein the second machine learning, classifier, artificial intelligence, or statistical modeling algorithm is trained on at least a portion of the subject population data to determine and output the individualized therapeutic treatment plan for the subject; providing the individualized therapeutic treatment plan, including a digital therapeutic, to the subject for treating the behavioral disorder, the developmental delay, or the neurological dysfunction, the digital therapeutic including an interactive session with a digital therapeutic device, during which the subject's interaction with the digital therapeutic device is used to update the subject data to generate updated subject data; wherein the updated subject data is used to generate updated diagnostic data, and an updated individualized treatment plan for the subject is output in response to the updated diagnostic data; The method, wherein said behavioral disorder, said developmental delay, or said neurological dysfunction comprises autism, autism spectrum, attention deficit disorder, attention deficit hyperactivity disorder, or speech and learning disability.

15. 15. The method of claim 14, wherein the digital therapeutic comprises instructions, feedback, activities, or interactions provided to the subject or caregiver.

16. 16. The method of claim 15, wherein the digital therapeutic is delivered using a mobile device.

17. The method of claim 14, further comprising the processor providing the diagnostic data and the individualized therapeutic treatment plan to a third party system.

18. The method of claim 17 , wherein the third party system comprises a healthcare professional's computer system or a treatment delivery system.

19. 15. The method of claim 14, wherein the updated subject data includes relative levels of efficacy, compliance, and response resulting from the individualized therapeutic treatment plan.

20. 15. The method of claim 14, wherein the subject data includes at least one of subject and caregiver video, audio, responses to questions or activities, and active or passive data streams from user interaction with activities, games, or software features.

21. 15. The method of claim 14, wherein the diagnostic data further includes data describing an evaluation of a child or minor, the data describing the evaluation of a child or minor prepared by an adult, the evaluation including data for use as input to the second machine learning, classification, artificial intelligence, or statistical modeling algorithm.

22. 1. A therapeutic system for treating a subject with an individualized therapeutic treatment plan, the system comprising: A processor configured to execute instructions contained in a non-transitory computer-readable storage medium to execute application software. the application software comprising: a diagnostic module for receiving object data and outputting diagnostic data relating to the object; a feature recommendation module configured to generate the subject data, the subject data including features selected from a group of candidate features, the feature recommendation module configured to estimate a predictive utility of each feature in the group of candidate features for predicting that the subject is at risk of a behavioral disorder, developmental delay, or neurological dysfunction, and to select the feature from the group of candidate features based on the estimated predictive utility, the diagnostic data including the risk of the behavioral disorder, the developmental delay, or the neurological dysfunction, and feature values ​​obtained from the subject based on the features from the group of candidate features, the feature values ​​being used to update the estimated predictive utility of each feature in the group of candidate features, the risk comprising at least an 85% confidence interval; a therapy module for receiving the diagnostic data and outputting the personalized therapeutic treatment plan for the subject, the personalized therapeutic treatment plan including a digital therapeutic for treating the behavioral disorder, the developmental delay, or the neurological dysfunction, the digital therapeutic including an interactive session with a digital therapeutic device, during which the subject's interaction with the digital therapeutic device is used to update the subject data to generate updated subject data; the diagnostic module is configured to receive the updated subject data and generate updated diagnostic data based on the updated subject data; and the therapeutic module is configured to receive the updated diagnostic data and output an updated individualized treatment plan for the subject in response to the diagnostic data and the updated diagnostic data; The behavioral disorder, developmental delay, or neurological dysfunction comprises autism, autism spectrum disorder, attention deficit disorder, attention deficit hyperactivity disorder, or speech and learning disability.

23. 23. The system of claim 22, wherein the updated subject data includes relative levels of efficacy, compliance, and response resulting from the individualized therapeutic treatment plan.

24. 23. The system of claim 22, wherein the personalized therapeutic treatment plan comprises a digital therapeutic.

25. 25. The system of claim 24, wherein the digital therapeutic comprises instructions, feedback, activities, or interactions provided to the subject or caregiver.

26. 26. The system of claim 25, wherein the digital therapeutic is delivered using a mobile device.

27. 23. The system of claim 22, wherein the diagnostic data and the individualized therapeutic treatment plan are provided to a third party system.

28. 30. The system of claim 27, wherein the third party system comprises a healthcare professional's computer system or a treatment delivery system.

29. 23. The system of claim 22, wherein the diagnostic module includes a diagnostic machine learning classifier trained on a subject population and the therapeutic module includes a therapeutic machine learning classifier trained on at least a portion of the subject population, and the diagnostic module is configured to provide feedback to the therapeutic module based on execution of the individualized therapeutic treatment plan.

30. 23. The system of claim 22, wherein the subject data includes at least one of subject and caregiver video, audio, responses to questions or activities, and active or passive data streams from user interaction with activities, games, or software features of the system.

31. 23. The system of claim 22, wherein the diagnostic module further comprises receiving data describing an evaluation of a child or minor, the data describing the evaluation of a child or minor prepared by an adult, the evaluation being subject data for the child or minor.

32. 1. A method for treating a subject with an individualized therapeutic treatment regimen, the method comprising: a processor estimating a predictive utility of each feature in a group of candidate features for predicting risk of behavioral disorder, developmental delay, or neurological dysfunction, said risk comprising at least an 85% confidence interval; the processor selecting features from the set of candidate features based on the estimated predictive utility to generate subject data, feature values ​​being obtained from the subject based on the features from the set of candidate features, the feature values ​​being used to update the estimated predictive utility of each feature in the set of candidate features; receiving, by the processor, the subject data for use as input to a first machine learning, classification, artificial intelligence, or statistical modeling algorithm trained on subject population data; the processor outputting diagnostic data including a risk of the subject for having a behavioral disorder, developmental delay, or neurological dysfunction for the subject; providing the diagnostic data for use as input to a second machine learning, classifier, artificial intelligence, or statistical modeling algorithm trained on at least a portion of the subject population data to output the individualized therapeutic treatment plan for the subject, the individualized therapeutic treatment plan including a digital therapeutic for treating the behavioral disorder, the developmental delay, or the neurological dysfunction, the digital therapeutic including an interactive session with a digital therapeutic device, during which the subject's interaction with the digital therapeutic device is used to update the subject data to generate updated subject data; generating updated diagnostic data based on the updated subject data; and the processor outputting an updated individualized treatment plan for the subject in response to the diagnostic data and the updated diagnostic data. Including, The method, wherein said behavioral disorder, said developmental delay, or said neurological dysfunction comprises autism, autism spectrum, attention deficit disorder, attention deficit hyperactivity disorder, or speech and learning disability.

33. 33. The method of claim 32, wherein the updated subject data includes relative levels of efficacy, compliance, and response resulting from the individualized therapeutic treatment plan.

34. 33. The method of claim 32, wherein the digital therapeutic comprises instructions, feedback, activities, or interactions provided to the subject or caregiver.

35. 35. The method of claim 34, wherein the digital therapeutic is delivered using a mobile device.

36. The method of claim 32, further comprising the processor providing the diagnostic data and the individualized therapeutic treatment plan to a third party system.

37. 37. The method of claim 36, wherein the third party system comprises a healthcare professional's computer system or a treatment delivery system.

38. 33. The method of claim 32, wherein the subject data includes at least one of subject and caregiver video, audio, responses to questions or activities, and active or passive data streams from user interaction with activities, games, or software features.

39. 33. The method of claim 32, wherein the diagnostic data includes data describing an evaluation of a child or minor, the data describing the evaluation of a child or minor prepared by an adult, and the evaluation is an input to the first machine learning, classification, artificial intelligence, or statistical modeling algorithm.

Citation Information

Patent Citations

  • Cognitive dysfunction danger computing device, cognitive dysfunction danger computing system, and program

    JP2011255106A

  • Determination system, determination method, and determination program

    JP2015228202A

  • A system and method for improving the workflow of neurologists regarding Alzheimer's disease.

    JP2015513157A