Systems and methods for digital assessment

Abstract

Provided are systems for digitally assessing a subject. Aspects of the systems include an upload agent, a transfer agent, a processing agent and an assessment agent. Also provided are methods, e.g., that employ the systems described herein to digitally assess a subject, as well as non-transitory computer readable storage media.

Description

[0001] SYSTEMS AND METHODS FOR DIGITAL ASSESSMENT

[0002] CROSS-REFERENCE TO RELATED APPLICATION The application claims the benefit of United States Provisional Patent Application Serial No. 63 / 748,365 filed January 22, 2025, the disclosure of which application is incorporated herein by reference in its entirety.

[0003] INTRODUCTION

[0004] Over the past decade, data science and data collection has transformed many aspects of health care. Advances in machine learning, large-scale data analysis, and electronic health records have enabled clinicians to make more informed decisions and uncover patterns that were previously difficult to detect. Despite these successes, the integration of data science into everyday clinical practice and clinical research remains uneven and incomplete. Many health care systems still struggle to translate data-driven insights into real-time, actionable tools that address current health care issues.

[0005] In particular, long wait times to see neurologists or other specialists have become an increasingly common barrier to timely medical care. As demand for specialty care outpaces provider availability, patients endure prolonged uncertainty and worsening symptoms. Delayed diagnoses and treatments result in worse outcomes for patients with potentially serious conditions. These delays highlight a critical weakness in the current healthcare system: the lack of consistent, standardized methods for assessing and prioritizing patients based on disease risk and severity.

[0006] SUMMARY

[0007] Assessment tools capable of diagnosing and monitoring diseases, as well as streamlining referrals for clinical evaluation are needed to ensure that patients receive appropriate care as efficiently as possible. Inventors of the present disclosure realized a way to utilize home videos (e.g., videos of a subject not taken in a clinical setting) as a means to digitally assess patients. As such, assessment is not constrained by clinician availability, and thus patients can beassessed with no wait time. Similarly, this digital assessment tool can reduce research costs associated with reimbursing participants and allows participants to complete assessments at a time that is most convenient for them. Further, as variability is introduced when assessment is done by different human clinicians, the present digital assessment tools allow for standardization of assessment, thereby improving consistency and accuracy. In contrast to the broad, subjective assessments performed by human clinicians, the digital assessment also addresses the need for granular research outcomes capable of detecting subtle changes in response to novel therapies.

[0008] Provided are systems for digitally assessing a subject. Aspects of the systems include an upload agent, a transfer agent, a processing agent and an assessment agent. Also provided are methods, e.g., that employ the systems described herein to digitally assess a subject, as well as non-transitory computer readable storage media.

[0009] BRIEF DESCRIPTION OF THE FIGURES

[0010] The invention may be best understood from the following detailed description when read in conjunction with the accompanying drawings. Included in the drawings are the following figures:

[0011] FIGS. 1A-1C depict flow diagrams of a system for digitally assessing a subject according to certain embodiments of the disclosure.

[0012] FIG. 2 depicts a functional block diagram of a computer system according to certain embodiments of the disclosure.

[0013] FIG. 3 depicts a general architecture of an example computing device according to certain embodiments of the disclosure.

[0014] FIG. 4 depicts a flow diagram of a method for digitally assessing a subject according to certain embodiments of the disclosure.

[0015] FIG. 5 depicts a flow diagram of a method for digitally assessing a subject where the subject performs a speech task according to certain embodiments of the disclosure.FIG. 6 depicts a flow diagram of a method for digitally assessing a subject where the subject performs a facial task according to certain embodiments of the disclosure.

[0016] FIG. 7 depicts a flow diagram of a method for digitally assessing a subject where the subject performs a hand task according to certain embodiments of the disclosure.

[0017] FIG. 8 depicts a flow diagram of a method for digitally assessing a subject where the subject performs a gait task according to certain embodiments of the disclosure.

[0018] FIG. 9 depicts an example of employing a system of the present disclosure to provide a neurological classification based on a speech task.

[0019] FIG. 10 depicts an example of employing a system of the present disclosure to provide a neurological diagnosis based on a facial task.

[0020] FIG. 11 depicts an example of employing a system of the present disclosure to evaluate treatment effectiveness based on a hand task.

[0021] FIG. 12 depicts an example of employing a system of the present disclosure to monitor disability progression based on a gait task.

[0022] DETAILED DESCRIPTION

[0023] As reviewed above, provided are systems for digitally assessing a subject. Aspects of the systems include an upload agent, a transfer agent, a processing agent and an assessment agent. Also provided are methods, e.g., that employ the systems described herein to digitally assess a subject, as well as non-transitory computer readable storage media.

[0024] Before the present invention is described in greater detail, it is to be understood that this invention is not limited to particular embodiments described, as such may, of course, vary. It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting, since the scope of the present invention will be limited only by the appended claims.Where a range of values is provided, it is understood that each intervening value, to the tenth of the unit of the lower limit unless the context clearly dictates otherwise, between the upper and lower limit of that range and any other stated or intervening value in that stated range, is encompassed within the invention. The upper and lower limits of these smaller ranges may independently be included in the smaller ranges and are also encompassed within the invention, subject to any specifically excluded limit in the stated range. Where the stated range includes one or both of the limits, ranges excluding either or both of those included limits are also included in the invention.

[0025] Certain ranges are presented herein with numerical values being preceded by the term “about.” The term “about” is used herein to provide literal support for the exact number that it precedes, as well as a number that is near to or approximately the number that the term precedes. In determining whether a number is near to or approximately a specifically recited number, the near or approximating unrecited number may be a number which, in the context in which it is presented, provides the substantial equivalent of the specifically recited number.

[0026] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Although any methods and materials similar or equivalent to those described herein can also be used in the practice or testing of the present invention, representative illustrative methods and materials are now described.

[0027] All publications and patents cited in this specification are herein incorporated by reference as if each individual publication or patent were specifically and individually indicated to be incorporated by reference and are incorporated herein by reference to disclose and describe the methods and / or materials in connection with which the publications are cited. The citation of any publication is for its disclosure prior to the filing date and should not be construed as an admission that the present invention is not entitled to antedate such publication by virtue of prior invention. Further, the dates of publication providedmay be different from the actual publication dates which may need to be independently confirmed.

[0028] It is noted that, as used herein and in the appended claims, the singular forms “a”, “an”, and “the” include plural referents unless the context clearly dictates otherwise. It is further noted that the claims may be drafted to exclude any optional element. As such, this statement is intended to serve as antecedent basis for use of such exclusive terminology as “solely,” “only” and the like in connection with the recitation of claim elements, or use of a “negative” limitation.

[0029] As will be apparent to those of skill in the art upon reading this disclosure, each of the individual embodiments described and illustrated herein has discrete components and features which may be readily separated from or combined with the features of any of the other several embodiments without departing from the scope or spirit of the present invention. Any recited method can be carried out in the order of events recited or in any other order which is logically possible.

[0030] While the system and method may be described for the sake of grammatical fluidity with functional explanations, it is to be expressly understood that the claims, unless expressly formulated under 35 U.S.C. § 112, are not to be construed as necessarily limited in any way by the construction of “means” or “steps” limitations, but are to be accorded the full scope of the meaning and equivalents of the definition provided by the claims under the judicial doctrine of equivalents, and in the case where the claims are expressly formulated under 35 U.S.C. § 112 are to be accorded full statutory equivalents under 35 U.S.C. § 112.

[0031] SYSTEMS FOR DIGITALLY ASSESSING A SUBJECT

[0032] As reviewed above, provided are systems for digitally assessing a subject. In embodiments, the system includes an upload agent configured to receive digital information related to a subject and store the digital information in a database, a transfer agent configured to scan the database for the digital information, standardize the digital information and transfer the digital information to a processing database, a processing agent configured to process the digitalinformation by applying a computer-implemented model to identify one or more features and extract one or more biomarkers from the digital information based on the one or more features, and an assessment agent configured to assess the subject based on the one or more extracted biomarkers. As described in detail herein, by “agent” it is meant, for example, a computer-implemented agent capable of perceiving its environment through inputs, making decisions based on some internal process, and acting on the environment through outputs in order to achieve one or more goals. In general, agents are capable of performing these tasks autonomously without continuous human control or interaction. As described in detail herein, an agent may comprise one or more models, such as one or more large language models. Systems for digitally assessing a subject may be used for any relevant digital assessment.

[0033] FIG. 1A illustrates a flow diagram of system 100A according to embodiments of the present invention. As shown in FIG. 1A, system 100A includes upload agent 101A. Upload agent 101A is configured to receive digital information related to a subject and store the digital information in database 110A. System 100A further includes transfer agent 102A. Transfer agent 102A is configured to scan database HOAforthe digital information, standardize the digital information and transfer the digital information to processing database 111 A. System 100A further includes processing agent 103A. Processing agent 103A is configured to process the digital information (e.g., standardized digital information stored in processing database 111 A) by applying a computer-implemented model to identify one or more features and extract one or more biomarkers from the digital information based on the one or more features.

[0034] System 100A further includes assessment agent 104A. Assessment agent 104A is configured to assess the subject based on the one or more extracted biomarkers.

[0035] Upload Agent

[0036] As reviewed above, the system includes an upload agent configured to receive digital information related to a subject and store the digital information ina database. “Digital information” may include any digital information related to the subject. For example, digital information related to the subject includes, but is not limited to, videos, voice recordings, electronic health records, pictures, subject-provided information or a combination thereof. In some cases, the digital information includes one or more of: a video and a voice recording. In some cases, the digital information includes a plurality of videos and / or voice recordings. In some cases, the digital information includes a plurality of videos. In some cases, the digital information includes one or more photographs. In some cases, the digital information may include one or more writings, such as written descriptions, such as electronic medical record information.

[0037] In some cases, the digital information related to the subject includes digital information of the subject or digital information about the subject. For example, in embodiments where the digital information includes videos, the videos may be videos of the subject. In other words, the videos may depict or show the subject. In embodiments where the digital information includes voice recordings, the voice recordings may be voice recordings of the subject speaking.

[0038] In some embodiments, the digital information includes digital information of the subject performing a task. In some cases, the digital information includes one or more videos of the subject performing a task. Tasks may include any relevant task for digitally assessing the subject. For example, in some embodiments, the task is selected from a speech task, a facial task, a hand task and a gait task. In some embodiments, the digital information includes a video of the subject completing a speech task, a facial task, a hand task, or a gait task. In some embodiments, the digital information includes one or more videos of the subject completing one or more of: a speech task, a facial task, a hand task, or a gait task. In some embodiments, the speech task and the facial task are the same (i.e., the same task is used as the speech task and facial task).

[0039] By “speech task” it is meant a task involving speech (i.e., speaking out loud). In such cases, the subject may be digitally assessed based on the subject’s speech. For example, a speech task may include the subject verbally describing an image out loud (i.e., a spoken description of the image). In anotherexample, a speech task may include the subject verbally describing their routine (e.g., morning routine) out loud (i.e., a spoken description of the subject’s routine). In some embodiments, the speech task includes the subject describing an image or the subject describing their routine. In some embodiments, the digital information includes a voice recording of the subject completing the speech task. In some embodiments, the digital information includes a video of the subject completing the speech task.

[0040] By “facial task” it is meant a task involving the face of the subject. In other words, when the subject completes the facial task, the face (or a part of the face such as a facial feature) of the subject moves or changes positions. For example, a face task may include the subject verbally describing their morning routine out loud. When the subject verbally speaks out loud, the face (e.g., lips, cheeks, etc.) of the subject moves. In some embodiments, the facial task includes the subject describing an image or the subject describing their routine. In some embodiments, the digital information includes a video of the subject completing a facial task.

[0041] By “hand task” it is meant a task involving a hand and / or the wrist of the subject. In other words, when the subject completes the hand task, the hand (or a part of the hand, such as a finger) and / or wrist of the subject moves or changes positions. For example, a hand task may include the subject buttoning an article of clothing (e.g., a shirt, jacket, etc.). In another example, a hand task may include the subject performing a finger tapping test (e.g., Halstead Finger Tapping Test). Finger tapping tests may include, e.g., repeatedly tapping the index finger and thumb together or repeatedly tapping a finger against a surface. In some embodiments, the hand task includes the subject buttoning a shirt or the subject performing a finger tapping test. In some embodiments, the digital information includes a video of the subject completing a hand task.

[0042] By “gait task” it is meant a task involving the gait of the subject. In some cases, the gait task includes the subject walking (e.g., walking away from a camera, walking towards a camera, walking in front of a camera, etc.). In other words, when the subject completes the gait task (e.g., the subject walks), thebody (or a part of the body, such as the legs, hips, ankles, knees, feet, etc.) of the subject moves or changes positions. In some embodiments, the digital information includes a video of the subject completing a gait task.

[0043] The digital information may include one or more videos (e.g., one video, two videos, three videos, four videos, five videos, six videos, seven videos, eight videos, nine videos, ten videos, etc.) related to the subject, including, e.g., two or more videos, three or more videos, four or more videos, five or more videos, six or more videos, seven or more videos, eight or more videos, nine or more videos and ten or more videos related to the subject. In some embodiments, the upload agent is configured to receive one or more videos related to a subject and store the one or more videos in a database.

[0044] In some embodiments, the upload agent is configured to receive digital information from the subject. In other words, the subject themself uploads the digital information and the upload agent is configured to receive such digital information. In some embodiments, the digital information (e.g., the video) is selfrecorded by the subject (i.e., a “selfie”). In some cases, the digital information is recorded in a non-clinical environment (e.g., at home).

[0045] In some embodiments, the digital information includes digital information about the subject over time (i.e., digital information collected over a period of time). The period of time may be 1 day or more, including, e.g., 1 week or more, 2 weeks or more, 3 weeks or more, 1 month or more, 2 months or more, 3 months or more, 4 months or more, 5 months or more, 6 months or more, 7 months or more, 8 months or more, 9 months or more, 10 months or more, 11 months or more, 1 year or more, 1.5 years or more, 2 years or more, 3 years or more, 4 years or more, 5 years or more and 10 years or more. Periods of time may range from 1 day to 10 years, including, e.g., 1 week to 1 month, 1 week to 1 year, 1 week to 2 years, 1 week to 5 years, 1 month to 6 months, 1 month to 1 year, 1 month to 2 years, 1 month to 5 years, 6 months to 1 year, 6 months to 1.5 years, 6 months to 2 years, 6 months to 5 years, 1 year to 2 years, 1 year to 5 years and 1 year to 10 years. In some cases, the period of time is 6 months or longer. In some cases, the period of time is 1 year or longer. In some cases, theperiod of time is 2 years or longer. The digital information may include digital information collected at two or more time points (e.g., two time points, three time points, four time points, five time points, six time points, seven time points, eight time points, nine time points, ten time points, 20 time points, 30 time points, 40 time points, 50 time points, etc.) within the period of time, including, e.g., three or more time points, four or more time points, five or more time points, six or more time points, seven or more time points, eight or more time points, nine or more time points, ten or more time points, 20 or more time points, 30 or more time points, 40 or more time points and 50 or more time points within the period of time. For example, in embodiments where the digital information is collected at two time points within the period of time, the digital information may be collected at a first time point and a second time point.

[0046] “Subjects,” “ individuals” and “patients” may be used interchangeably. In some cases, the subject is a mammal (e.g., human). In some cases, the subject is a human. Humans may be of any ethnicity, age, gender or other physiological or demographic characteristics. In some cases, the human is an adult (i.e. , 18 years or older). In some cases, the human is a juvenile (i.e. , less than 18 years old). In some cases, the human is geriatric (e.g., 65 years or older). In some cases, the subject has been diagnosed with a disease, treated for a disease and / or at risk for a disease. In some cases, the subject has not been diagnosed with and / or treated for a disease. Diseases may be any disease of interest, such as a neurological disease including, but not limited to, Alzheimer’s disease, Parkinson’s disease, amyotrophic lateral sclerosis (ALS), Huntington’s disease, multiple sclerosis (MS), dementia, spinal muscular atrophy (SMA), frontotemporal dementia (FTD), essential tremor (ET), stroke, myasthenia gravis, muscular dystrophy, peripheral neuropathy, Guillain-Barre syndrome, Bell’s palsy, multiple system atrophy, tauopathy, brain or spinal cord injury, and prion disease.

[0047] As reviewed above, in some cases, the digital information includes electronic health records. Electronic health records may include one or more of: demographics (birth year, gender, race and ethnicity), clinical concepts (conditions, drug exposures, abnormal measures), and visit-related features (ageat prediction, first visit age, years in the electronic health records). In some embodiments, electronic health record data may cover the entire patient health history through signs, symptoms, medical diagnosis and evaluation or aspects or subsets thereof.

[0048] As reviewed above, in some cases, the digital information includes subject-provided information. By “subject-provided” information it is meant other information provided by the subject that does not include a video, a voice recording, a picture or an electronic health record. For example, when uploading a video to the upload agent, the subject may designate or label what task is completed in the video. This designation or label may be considered subject-provided information. As an example, the subject may upload a video showing a gait task to a field labeled gait task . In other words, designation that the video shows a gait task is subject-provided information.

[0049] Upload agents may include any agent capable of receiving digital information and then storing the digital information in a database. In other words, the upload agent has electronic data capture (EDC) capability. In some cases, the upload agent is compliant with protected health information laws, regulations or rules, including the Health Insurance Portability and Accountability Act (HIPAA) or the California Confidentiality of Medical Information Act (CMIA) or the General Data Protection Regulation (GDPR). Upload agents include, but are not limited to, REDCap, Qualtrics, Castor EDC, Medrio and OpenClinica.

[0050] Transfer Agent

[0051] As reviewed above, the system includes a transfer agent configured to scan the database for the digital information, standardize the digital information and transfer the digital information to a processing database. By processing database, it is meant a database capable of storing the standardized digital information (e.g., standardized digital information that is ready for processing by the processing agent). The database or processing database may be any convenient database for storing digital information. In some cases, the database is a video storage database. Such databases are known in the art. Databasesinclude, but are not limited to MinlO, Amazon Web Services (AWS), Garage, SeaweedFS, Ceph S3, etc.

[0052] In embodiments, the transfer agent is configured to scan the database for digital information. In some cases, scanning the database for digital information includes scanning the database for new digital information. By “new digital information” it is meant digital information that has not yet been processed by the processing agent and / or digital information that is not present in the processing database. In some cases, when scanning the database for digital information, if the transfer agent determines that there is no new digital information present, the transfer agent is configured to then rescan the database at a later time. In some cases, the transfer agent is configured to rescan the database at a scheduled time interval (e g., once a minute, once an hour, once a day, every two days, once a week, once a month, etc.).

[0053] In some cases, when scanning the database for digital information, if the transfer agent determines that there is new digital information present, the transfer agent is configured to then standardize the digital information and transfer the digital information to a processing database. In some cases, standardizing the digital information includes one or more of: generating a standardized filename and generating a standardized format. In some cases, the transfer agent is configured to pull identification (ID), date and / or task information to generate a standardized filename. In some cases, the transfer agent is configured to convert all videos to .mp4 files. In some cases, the transfer agent is configured to transfer the standardized digital information to the processing database.

[0054] FIG. 1B illustrates a flow diagram of transfer agent 102B according to embodiments of the present disclosure. As shown in the example depicted in FIG. 1B, transfer agent 102B is configured to scan database 11 OB for digital information. If there is no new digital information, transfer agent 102B is configured to rescan the database at a scheduled time interval. If there is new digital information, transfer agent 102B is configured to standardize the newdigital information. Transfer agent 102B is configured to then transfer the digital information (e.g., standardized digital information) to processing database 111B.

[0055] Processing Agent

[0056] As reviewed above, the system includes a processing agent configured to process the digital information by applying a computer-implemented model to identify one or more features and extract one or more biomarkers from the digital information based on the one or more features. Suitable computer-implemented models include, but are not limited to those discussed below.

[0057] By “feature” it is meant a feature that can be visually or audibly identified in the digital information. In some cases, the processing agent is configured to identify one or more features (e g., one feature, two features, three features, four features, five features, six features, seven features, eight features, nine features, ten features, etc.) including, e.g., two or more features, three or more features, four or more features, five or more features, six or more features, seven or more features, eight or more features, nine or more features and ten or more features. Such features may include speech features, facial features, hand features, and gait features, or a combination thereof.

[0058] By “biomarker” it is meant a metric that can be calculated and / or determined from tracking the feature (e.g., tracking the feature over time) in the digital information. In some cases, the processing agent is configured to extract one or more biomarkers (e.g., one biomarker, two biomarkers, three biomarkers, four biomarkers, five biomarkers, six biomarkers, seven biomarkers, eight biomarkers, nine biomarkers, ten biomarkers, etc.) including, e.g., two or more biomarkers, three or more biomarkers, four or more biomarkers, five or more biomarkers, six or more biomarkers, seven or more biomarkers, eight or more biomarkers, nine or more biomarkers and ten or more biomarkers. Such biomarkers may include speech biomarkers, facial biomarkers, hand biomarkers, and gait biomarkers, or a combination thereof.

[0059] In some embodiments, the processing agent is further configured to select which computer-implemented model to apply based on a task performed by thesubject in the digital information. In some cases, the digital information includes subject-provided information indicating what task (or tasks) is performed in the digital information. The computer-implemented model may be selected based on this subject-provided information (i.e. , indicating what task is performed in the digital information). For example, a subject may indicate that their digital information includes a video of a gait task. As such, the processing agent is configured to apply a computer-implemented model relevant for analyzing a gait task. In some cases, the processing agent is configured to apply a computer-implemented model (e.g., large language model) to determine what task (or tasks) are performed in the digital information.

[0060] Tasks may include any relevant task for digitally assessing the subject. In some embodiments, the task is selected from a speech task, a facial task, a hand task and a gait task. Speech tasks, facial tasks, hand tasks and gait tasks include those discussed above.

[0061] In some embodiments, the task is a speech task. In some embodiments where the task is a speech task, the processing agent is configured to process the digital information by applying a computer-implemented speech analysis model to identify one or more speech features and extract one or more speech biomarkers from the digital information based on the one or more speech features. Computer-implemented speech analysis models include, e.g., OpenAI Whisper, CLASP++ and models described in Vonk, et al. Neurology 104(9):e213556 (2025).

[0062] By “speech feature” it is meant a feature relating to the speech of a subject. In some embodiments, the one or more speech features are selected from one or more of: an acoustic feature, a linguistic feature, a disfluency feature and a speech content feature. Acoustic features include features related to speech acoustics such as pitch, loudness, timing, resonance, articulation and slurring. Linguistic features include features related to speech linguistics such as grammar, vocabulary and sentence structure. Disfluency features include features related to speech disfluency such as repetitions, repeats, restarts and fillers. In some cases, disfluency features are identified from a transcript of thevideo. In some cases, identifying one or more disfluency features includes annotating the transcript with disfluency symbols to identify repetitions, repeats, restarts and fillers. Speech content features include features related to the content of the speech (e.g., the meaning of the speech).

[0063] By “speech biomarker” it is meant a biomarker calculated and / or determined from tracking one or more speech features. In some embodiments, the one or more speech biomarkers are selected from one or more of: an acoustic biomarker, a linguistic biomarker, a disfluency biomarker and a speech content biomarker.

[0064] In some embodiments, when the task is a speech task, the processing agent is further configured to generate a speech transcription from the digital information. In some embodiments, the processing agent is further configured to process the speech transcription by annotating the transcript to identify one or more speech features and / or applying a computer-implemented model to identify one or more speech features based on content in the speech transcription. In some embodiments, the processing agent is configured to process audio from a video or a voice recording to identify acoustic features.

[0065] In some embodiments, the task is a facial task. In some embodiments where the task is a facial task, the processing agent is configured to process the digital information by applying a computer-implemented facial analysis model to identify one or more facial features and extract one or more facial biomarkers from the digital information based on the one or more facial features. Computer-implemented facial analysis models include, e.g., OpenFace.

[0066] By “facial feature” it is meant a feature relating to the facial movement and / or facial position of a subject. In some embodiments, the one or more facial features are selected from one or more of: a facial landmark and a facial action unit. Facial landmarks include the location and / or position of parts of the face. For example, the nose may be a facial landmark or the outer end of the eyebrow may be a facial landmark. A facial action unit is an anatomically based movement of the face produced by the contraction of one or more facial muscles. Facial action units are used in the facial action coding system (FACS) to objectivelydescribe and measure facial expressions. For more details regarding FACS, see, e.g., Clark et al., Front Psychol. 11:920 (2020); Hamm et al., J Neurosci Methods.

[0067] 200(2):237-256 (2011), etc.

[0068] By “facial biomarker” it is meant a biomarker calculated and / or determined from tracking one or more facial features. In some embodiments, the one or more facial biomarkers are selected from one or more of: a facial asymmetry biomarker, a blinking biomarker, a frowning biomarker and a smiling biomarker. For example, one or more facial landmarks (e.g., a left corner of the lips and a right comer of the lips) may be used to determine the degree of facial asymmetry when smiling.

[0069] In some embodiments, when the task is a facial task, the processing agent is further configured to do one or more of: stop processing a video if facial features are not identified in a threshold percentage of frames in the video, remove extraneous activities from a video, standardize frame rate of a video, calculate metrics associated with muscle weakness or paralysis, calculate summary statistics based on the one or more facial features and identify metrics associated with neurological disease diagnosis.

[0070] In some embodiments, the task is a hand task. In some embodiments where the task is a hand task, the processing agent is configured to process the digital information by applying a computer-implemented hand analysis model to identify one or more hand features and extract one or more hand biomarkers from the digital information based on the one or more hand features. Computer-implemented hand analysis models include, e.g., MediaPipe Hand.

[0071] By “hand feature” it is meant a feature relating to the hand movement and / or hand position of a subject. In some embodiments, the one or more hand features are selected from one or more of: a hand landmark and a wrist landmark. Hand landmarks include the location and / or position of parts of the hand (e.g., the position of a finger, such as the index finger). Wrist landmarks include the location and / or position of the wrist or a part of the wrist.

[0072] By “hand biomarker” it is meant a biomarker calculated and / or determined from tracking one or more hand features. In some embodiments, the one or morehand biomarkers are selected from one or more of: a finger tapping frequency biomarker, a finger tapping speed biomarker, a finger tapping amplitude biomarker, a movement complexity biomarker and a path length biomarker.

[0073] In some embodiments, when the task is a hand task, the processing agent is further configured to do one or more of: stop processing a video if hand features are not identified in a threshold percentage of frames in the video, remove extraneous activities from a video, standardize frame rate of a video, calculate metrics associated with motor function or coordination and calculate metrics associated with hand dexterity.

[0074] In some embodiments, the task is a gait task. In some embodiments where the task is a gait task, the processing agent is configured to process the digital information by applying a computer-implemented gait analysis model to identify one or more gait features and extract one or more gait biomarkers from the digital information based on the one or more gait features. Computer-implemented gait analysis models include, e.g., MediaPipe Pose and Ultralytics.

[0075] By “gait feature” it is meant a feature relating to the positioning and / or location of a body part that moves when walking. In some embodiments, the one or more gait features are selected from one or more of: a hip landmark, a heel landmark and an ankle landmark.

[0076] By “gait biomarker” it is meant a biomarker calculated and / or determined from tracking one or more gait features. In some embodiments, the one or more gait biomarkers are selected from one or more of: a walking speed biomarker, a stride time biomarker, a stride width biomarker, a gait variation biomarker and a gait asymmetry biomarker.

[0077] In some embodiments, when the task is a gait task, the processing agent is further configured to do one or more of: stop processing a video if gait features are not identified in a threshold percentage of frames in the video, remove extraneous activities from a video, standardize frame rate of a video and calculate metrics associated with walking impairment and ambulatory

[0078] dysfunction.FIG. 10 illustrates a flow diagram of processing agent 103C according to embodiments of the present disclosure. As shown in the example depicted in FIG. 1C, processing agent 103C is configured to process digital information (e.g., from processing database 111C). Processing agent 103C is configured to apply one or more computer-implemented models to the digital information. For example, as shown in FIG. 1C, processing agent 103C is configured to: apply a computer-implemented speech analysis model 1200 to identify one or more speech features and extract one or more speech biomarkers from the digital information based on the one or more speech features, apply a computer-implemented facial analysis model 121 C to identify one or more facial features and extract one or more facial biomarkers from the digital information based on the one or more facial features, apply a computer-implemented hand analysis model 122C to identify one or more hand features and extract one or more hand biomarkers from the digital information based on the one or more hand features, and apply a computer-implemented gait analysis model 123C to identify one or more gait features and extract one or more gait biomarkers from the digital information based on the one or more gait features. Processing agent 103C may be configured to apply computer-implemented models (e.g., computer implemented models 120C, 1210, 122C and 123C) sequentially or simultaneously. Extracted biomarkers (e.g., speech biomarkers, facial biomarkers, hand biomarkers, gait biomarkers) may then be used individually or in combination by assessment agent 104C to assess the subject.

[0079] Assessment Agent

[0080] As reviewed above, the system includes an assessment agent configured to assess the subject based on the one or more extracted biomarkers.

[0081] Biomarkers of interest include those discussed above. In some embodiments, the assessment agent is configured to assess the subject based on one or more of: a speech biomarker, a facial biomarker, a hand biomarker and a gait biomarker. In some embodiments, the assessment agent is configured to assess the subject based on a speech biomarker, a facial biomarker, a hand biomarker and a gaitbiomarker. The assessment agent may be configured to perform an assessment (e.g., diagnosing or screening a subject for a disease) based on a single biomarker or a combination of any number of biomarkers (e.g., biomarkers relevant for diagnosing a disease). For example, as multiple sclerosis (MS) causes nerve damage that affects speech muscles leading to voice changes, one or more acoustic biomarkers may be used to assess if a subject has MS or assess the progression of MS in a subject or assess the effect of a treatment, such as a medication, on a subject with MS.

[0082] The subject may be assessed based on one or more biomarkers (e.g., one biomarker, two biomarkers, three biomarkers, four biomarkers, five biomarkers, six biomarkers, seven biomarkers, eight biomarkers, nine biomarkers, ten biomarkers, etc.) including, e.g., two or more biomarkers, three or more biomarkers, four or more biomarkers, five or more biomarkers, six or more biomarkers, seven or more biomarkers, eight or more biomarkers, nine or more biomarkers and ten or more biomarkers. In some embodiments, the assessment agent is configured to assess the subject based on a combination of two or more extracted biomarkers. In some embodiments, the assessment agent is configured to assess the subject based on a combination of three or more extracted biomarkers. In some embodiments, the assessment agent is configured to assess the subject based on a combination of four or more extracted biomarkers.

[0083] In some embodiments, the assessment agent is further configured to assess whether the subject has a disease or disability, assess progression of a disease or disability and / or assess efficacy of a therapy or a treatment. In some cases, the assessment agent is configured to assess whether the subject has a disease or disability based on the one or more extracted biomarkers. In some cases, the assessment agent is configured to assess progression of a disease or disability based on the one or more extracted biomarkers. In some cases, the assessment agent is configured to assess efficacy of a therapy or treatment based on based on the one or more extracted biomarkers. In some cases, a user (e.g., a clinician, researcher) may choose which biomarkers are most relevant tothe goal of the assessment and / or instruct the assessment agent to assess the subject based on particular biomarkers. For example, in instances where certain acoustic features may be associated with multiple measures (e.g., mood, cognition, fatigue, etc.) and multiple conditions (frontotemporal dementia, multiple sclerosis, etc.), a clinician could choose the speech biomarkers that are more salient with regard to multiple sclerosis.

[0084] Further Agents

[0085] Systems of the disclosure may further include any additional agents, e.g., agents configured for a desired functionality. In some embodiments, the system further includes a visualization agent. In some embodiments, the visualization agent is configured to display one or more biomarkers or display a change in one or more biomarkers overtime. In some embodiments, the visualization agent is configured to display one or more biomarkers and display a change in one or more biomarkers over time. Biomarkers include those discussed above. Time periods relevant for changes in one or more biomarkers over time include those discussed above.

[0086] Models

[0087] In certain embodiments, one or more of the agents (e.g., processing agent, etc.) is a computer-implemented model or employs a computer-implemented model. Computer-implemented models of interest include any commercially available and / or publicly available computer-implemented models. Computer-implemented models of interest include, but are not limited to, OpenAI Whisper, OpenFace, MediaPipe Hand, MediaPipe Pose and Ultralytics.

[0088] In some instances, the computer-implemented models comply with protected health information laws, regulations or rules, including the Health Insurance Portability and Accountability Act (HIPAA) within the United States of America and / or the California Confidentiality of Medical Information Act (CMIA) in California and / or the General Data Protection Regulation (GDPR), Regulation (EU) 2016 / 679 of the European Parliament and of the Council of 27 April 2016within the European Union. Additionally, in some instances, the LLM is static or does not learn from queried data or prompts. This non-learning property may also assist in complying with applicable laws (e.g., HIPAA) in a particular jurisdiction (e.g., nation, state, province, department, parish, county, borough, city, precinct, or territory).

[0089] In some embodiments, the computer-implemented model is a language model (LM). In some embodiments, the language model is a large language model (LLM). LLMs include LLMs that are trained, broadly, on general language data, as well as LLMs trained on specialist-level information. In various instances, the LLM is a generative pre-trained transformer (GPT) model, including, but not limited to, GPT-3, GPT-3.5, GPT-3.5-turbo, GPT-4o, GPT-4 and / or GPT-5.

[0090] In some embodiments, the computer implemented model (e.g., LLM) is a trained computer-implemented model (e.g., trained LLM). The computer-implemented model (e.g., LLM) may be trained using any convenient technique. In some cases, the computer-implemented model (e.g., LLM) is trained using an unsupervised learning technique, a semi-supervised learning technique, a supervised learning technique, a parallel training technique, a round robin training technique, an attention-based training technique and combinations thereof, as each such technique is known in the art. In some cases, the LLM is trained using a self-supervised training technique, i.e. , in which a model learns to predict a part of its input data using other parts of the same data as a form of supervision.

[0091] Further details regarding aspects of the disclosure are discussed in US Patent Nos. 11,989,507, 12,148,421, 12,199,936, 12,182,506, 12,032,919, 12,231,456, 12,067,039, 12,051,205, the disclosures of each of which are incorporated herein by reference.

[0092] Further Aspects of the Systems and Non-Transitory Computer-Readable Medium The systems described in connection with the embodiments disclosed herein can be implemented as electronic hardware, computer software, orcombinations of both. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. The described functionality can be implemented in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosure.

[0093] As reviewed above, aspects of the disclosure include systems for digitally assessing a subject. In some embodiments, the system includes one or more processors including memory operably coupled to the one or more processors, where the memory includes instructions stored thereon, which when executed by the processor, cause the processor to instantiate one or more of the agents, an output device configured to present output to the user, and / or an input device configured to receive input from the user.

[0094] Non-transitory physical computer readable storage media of the present disclosure include, but are not limited to, disks (e.g., magnetic or optical disks), solid-state storage drives, cards, tapes, drums, punched cards, barcodes, and magnetic ink characters and other physical medium that may be used for storing representations, instructions, and / or the like.

[0095] In some embodiments, the non-transitory computer-readable storage medium includes instructions stored thereon that cause the system to perform methods described herein (e.g., methods such as those described below and / or methods employing systems described above). In some embodiments, the non-transitory computer readable storage medium includes instructions stored thereon, where the instructions include algorithm for obtaining digital information about a subject, algorithm for processing the digital information via a processing agent to extract one or more biomarkers, algorithm for digitally assessing the subject via an assessment agent based on the one or more extracted biomarkers and algorithm for outputting the assessment to a user. In some embodiments, the non-transitory computer readable storage medium further includes instructions that include algorithm for obtaining the digital information via an upload agent and / or a transfer agent. In some embodiments, the non-transitory computerreadable storage medium further includes instructions that include algorithm for employing any of the systems described above to digitally assess the subject.

[0096] In certain embodiments, the non-transitory computer-readable medium includes instructions stored thereon that cause the system to obtain a digital information about a subject. In some cases, the digital information is obtained via a network connection to a remote server. Such servers may be located within the same building, complex, campus, or compound or may be located at a separate location, e.g., a different geographic location. The network connection may include one or more pieces of hardware, such as a router, modem, cable, ethernet cord, etc. to complete the connection.

[0097] In some instances, the non-transitory computer-readable storage medium includes instructions stored thereon that cause the system to employ a computer-implemented model to assess a subject. The computer-implemented model (e.g., LLM) or other aspects of embodiments may be accessed locally (e.g., a computing device co-located with a user) or may be maintained remotely on a server. For example, a server may have higher computing power or processing power that can allow faster processing for generating and evaluating a response, or a server may have a faster connection to the source of trusted information, e.g., a database server.

[0098] A system of the present disclosure may include a logic subsystem. The logic subsystem may include one or more processors configured to execute software instructions. Additionally or alternatively, the logic subsystem may include one or more hardware or firmware logic machines configured to execute hardware or firmware instructions. The processors of the logic subsystem may be single-core or multi-core, and the programs executed thereon may be configured for sequential, parallel or distributed processing. The processors of the logic subsystem may be one or more graphics processing units (GPUs). The logic subsystem may include individual components that are distributed among two or more devices, which can be remotely located and / or configured for coordinated processing. Aspects of the logic subsystem may be virtualized and executed byremotely accessible networked computing devices configured in a cloudcomputing configuration.

[0099] The various illustrative steps, components, and computing systems (such as devices, databases, interfaces, and engines) described in connection with the embodiments disclosed herein can be implemented or performed by a machine, such as a general purpose processor, a graphics processor unit, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor can be a microprocessor, but in the alternative, the processor can be a controller, microcontroller, or state machine, combinations of the same, or the like. A processor can also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Although described herein primarily with respect to digital technology, a processor can also include primarily analog components. A computing environment can include any type of computer system, including, but not limited to, a computer system based on a microprocessor, a graphics processor unit, a mainframe computer, a digital signal processor, a portable computing device, a personal organizer, a device controller, and a computational engine within an appliance, to name a few.

[0100] The steps of a method, process, or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module, engine, and associated databases can reside in memory resources such as in RAM memory, FRAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of non-transitory computer-readable storage medium, media, or physical computer storage known in the art. An external storage medium can be coupled to the processor such that the processor canread information from, and write information to, the storage medium. In the alternative, the storage medium can be integral to the processor. The processor and the storage medium can reside in an ASIC. The ASIC can reside in a user terminal. In the alternative, the processor and the storage medium can reside as discrete components in a user terminal.

[0101] Some embodiments may further comprise a display device, e.g., for displaying biomarkers, changes in biomarkers and / or results of, or related to, the assessment of the subject. Any convenient display device, such as a liquid crystal display (LCD), light-emitting diode (LED) display, plasma (PDP) display, quantum dot (QLED) display or cathode ray tube display device. The processor and / or memory may be operably connected to the display device, for example, via a wired, such as a Universal Serial Bus (USB) connection, or wireless connection, such as a Bluetooth connection.

[0102] In some instances, the systems further include one or more computers for complete automation or partial automation of the methods described herein. In some embodiments, systems include a computer having a computer readable storage medium with a computer program stored thereon.

[0103] In some embodiments, the system includes an input module, a processing module and an output module. The subject systems may include both hardware and software components, where the hardware components may take the form of one or more platforms, e.g., in the form of servers, such that the functional elements, i.e., those elements of the system that carry out specific tasks (such as managing input and output of information, processing information, etc.) of the system may be carried out by the execution of software applications on and across the one or more computer platforms represented of the system.

[0104] Systems may include a display and operator input device. Operator input devices may, for example, be a keyboard, mouse, or the like. The processing module includes a processor which has access to a memory having instructions stored thereon for performing the steps of the subject methods. The processing module may include an operating system, a graphical user interface (GUI) controller, a system memory, memory storage devices, and input-outputcontrollers, cache memory, a data backup unit, and many other devices. The processor may be a commercially available processor or it may be one of other processors that are or will become available. The processor executes the operating system and the operating system interfaces with firmware and hardware in a well-known manner, and facilitates the processor in coordinating and executing the functions of various computer programs that may be written in a variety of programming languages, such as Java, Perl, C++, other high level or low level languages, as well as combinations thereof, as is known in the art. The operating system, typically in cooperation with the processor, coordinates and executes functions of the other components of the computer. The operating system also provides scheduling, input-output control, file and data management, memory management, and communication control and related services, all in accordance with known techniques. The processor may be any suitable analog or digital system. In some embodiments, processors include analog electronics. In some embodiments, the processor includes analog electronics which provide feedback control, such as for example negative feedback control.

[0105] The system memory may be any of a variety of known or future memory storage devices. Examples include any commonly available random access memory (RAM), magnetic medium such as a resident hard disk or tape, an optical medium such as a read and write compact disc, flash memory devices, or other memory storage device. The memory storage device may be any of a variety of known or future devices, including a compact disk drive, a tape drive, a removable hard disk drive, or a diskette drive. Such types of memory storage devices typically read from, and / or write to, a program storage medium (not shown) such as, respectively, a compact disk, magnetic tape, removable hard disk, or floppy diskette. Any of these program storage media, or others now in use or that may later be developed, may be considered a computer program product. As will be appreciated, these program storage media typically store a computer software program and / or data. Computer software programs, also called computer control logic, typically are stored in system memory and / or the program storage device used in conjunction with the memory storage device.In some embodiments, a computer program product is described comprising a computer usable medium having control logic (computer software program, including program code) stored therein. The control logic, when executed by the processor the computer, causes the processor to perform functions described herein. In other embodiments, some functions are implemented primarily in hardware using, for example, a hardware state machine. Implementation of the hardware state machine so as to perform the functions described herein will be apparent to those skilled in the relevant arts.

[0106] Memory may be any suitable device in which the processor can store and retrieve data, such as magnetic, optical, or solid-state storage devices (including magnetic or optical disks or tape or RAM, or any other suitable device, either fixed or portable). The processor may include a general-purpose digital microprocessor suitably programmed from a computer readable medium carrying necessary program code. Programming can be provided remotely to processor through a communication channel, or previously saved in a computer program product such as memory or some other portable or fixed computer readable storage medium using any of those devices in connection with memory. For example, a magnetic or optical disk may carry the programming, and can be read by a disk writer / reader. Systems of the invention also include programming, e.g., in the form of computer program products, algorithms for use in practicing the methods as described above. Programming according to the present invention can be recorded on computer readable media, e.g., any medium that can be read and accessed directly by a computer. Such media include, but are not limited to: magnetic storage media, such as floppy discs, hard disc storage medium, and magnetic tape; optical storage media such as CD-ROM; electrical storage media such as RAM and ROM; portable flash drive; and hybrids of these categories such as magnetic / optical storage media.

[0107] The processor may also have access to a communication channel to communicate with a user at a remote location. By remote location is meant the user is not directly in contact with the system and relays input information to an input manager from an external device, such as a computer connected to a WideArea Network (“WAN”), telephone network, satellite network, or any other suitable communication channel, including a mobile telephone (i.e. , smartphone).

[0108] In some embodiments, systems according to the present disclosure may be configured to include a communication interface. In some embodiments, the communication interface includes a receiver and / or transmitter for communicating with a network and / or another device. The communication interface can be configured for wired or wireless communication, including, but not limited to, radio frequency (RF) communication (e.g., Radio-Frequency Identification (RFID), Zigbee communication protocols, WiFi, infrared, wireless Universal Serial Bus (USB), Ultra Wide Band (UWB), Bluetooth® communication protocols, and cellular communication, such as code division multiple access (CDMA) or Global System for Mobile communications (GSM).

[0109] In one embodiment, the communication interface is configured to include one or more communication ports, e.g., physical ports or interfaces such as a USB port, an RS-232 port, or any other suitable electrical connection port to allow data communication between the subject systems and other external devices such as a computer terminal (for example, at a physician’s office or in hospital environment) that is configured for similar complementary data communication.

[0110] In one embodiment, the communication interface is configured for infrared communication, Bluetooth® communication, or any other suitable wireless communication protocol to enable the subject systems to communicate with other devices such as computer terminals and / or networks, communication enabled mobile telephones, personal digital assistants, or any other communication devices which the user may use in conjunction with other devices.

[0111] In one embodiment, the communication interface is configured to provide a connection for data transfer utilizing Internet Protocol (IP) through a cell phone network, Short Message Service (SMS), wireless connection to a personal computer (PC) on a Local Area Network (LAN) which is connected to the internet, or WiFi connection to the internet at a WiFi hotspot.In one embodiment, the subject systems are configured to wirelessly communicate with a server device via the communication interface, e.g., using a common standard such as 802.11 or Bluetooth® RF protocol, or an IrDA infrared protocol. The server device may be another portable device, such as a smart phone, Personal Digital Assistant (PDA) or notebook computer; or a larger device such as a desktop computer, appliance, etc. In some embodiments, the server device has a display, such as a liquid crystal display (LCD), as well as an input device, such as buttons, a keyboard, mouse or touch-screen.

[0112] In some embodiments, the communication interface is configured to automatically or semi-automatically communicate data stored in the subject systems, e.g., in an optional data storage unit, with a network or server device using one or more of the communication protocols and / or mechanisms described herein.

[0113] Output controllers may include controllers for any of a variety of known display devices for presenting information to a user, whether a human or a machine, whether local or remote. If one of the display devices provides visual information, this information typically may be logically and / or physically organized as an array of picture elements. A graphical user interface (GUI) controller may include any of a variety of known or future software programs for providing graphical input and output interfaces between the system and a user, and for processing user inputs. The functional elements of the computer may communicate with each other via system bus. Some of these communications may be accomplished in alternative embodiments using network or other types of remote communications. The output manager may also provide information generated by the processing module to a user at a remote location, e.g., over the Internet, phone or satellite network, in accordance with known techniques. The presentation of data by the output manager may be implemented in accordance with a variety of known techniques. As some examples, data may include SQL, HTML or XML documents, email or other files, or data in other forms. The data may include Internet URL addresses so that a user may retrieve additional SQL, HTML, XML, or other documents or data from remote sources. The one or moreplatforms present in the subject systems may be any type of known computer platform or a type to be developed in the future, and they may be of a class of computer commonly referred to as servers. However, they may also be a mainframe computer, a workstation, or other computer type, such as a laptop computer. They may be connected via any known or future type of cabling or other communication system including wireless systems, either networked or otherwise. They may be co-located or they may be physically separated.

[0114] Various operating systems may be employed on any of the computer platforms, possibly depending on the type and / or make of computer platform chosen.

[0115] Appropriate operating systems include Windows, iOS, Oracle Solaris, Linux, IBM i, Unix, and others.

[0116] FIG. 2 shows a functional block diagram for one example of a computer system 200 for practicing methods of the present invention, i.e., a processor operably connected to memory, 202, for digitally assessing a subject. A processor and memory 202 can be configured to implement a variety of processes for digitally assessing a subject.

[0117] An apparatus 212 may be a database, and processor 202 may be operably connected to such database to acquire digital information, e.g., digital information related a subject. A data communication channel can be included between the apparatus 212 and the processor 202. Digital information can be provided to the processor 202 via the data communication channel. The processor 202 may further be configured such that a computer-implemented model (e.g., LLM) can be applied to the digital information.

[0118] The processor 202 can be configured to provide a graphical display, such as one or more graphs or plots illustrating biomarkers, changes in biomarkers and / or an assessment to a display device 206. The processor 202 can be further configured to display data on the display device 206. The display device 206 can be implemented as a monitor, a tablet computer, a smartphone, or other electronic device configured to present graphical interfaces.The processor 202 can be configured to receive adjustments from a first input device. The first input device can be implemented as a mouse 210, or the first device can be implemented as the keyboard 208 or other means for providing an input signal to the processor 202 such as a touchscreen, a stylus, an optical detector, or a voice recognition system. Some input devices can include multiple inputting functions. In such implementations, the inputting functions can each be considered an input device. For example, as shown in FIG. 2, the mouse 210 can include a right mouse button and a left mouse button, each of which can generate a triggering event. Such triggering event can cause the processor 202 to alter the manner in which the data is displayed, which portions of the data is actually displayed on the display device 206, and / or provide input to further processing.

[0119] The processor 202 can be connected to a storage device 204. The storage device 204 can be configured to receive and store digital information from the processor 202.

[0120] The display device 206 can be further configured to alter the information presented according to input received from the processor 202 in conjunction with input from the apparatus 212, the storage device 204, the keyboard 208, and / or the mouse 210. In some implementations, the processor 202 can generate a user interface for use with digitally assessing a subject.

[0121] FIG. 3 depicts a general architecture of an example computing device 300 according to certain embodiments. The general architecture of the computing device 300 includes an arrangement of computer hardware and software components. The computing device 300 may include many more (or fewer) elements than those shown in FIG. 3. It is not necessary, however, that all of these generally conventional elements be shown in order to provide an enabling disclosure. As illustrated, the computing device 300 includes a processing unit 310, a network interface 320, a computer readable medium 330, an input / output device interface 340, a display 350, and an input device 360, all of which may communicate with one another by way of a communication bus. The network interface 320 may provide connectivity to one or more networks or computingsystems. The processing unit 310 may thus receive information and instructions from other computing systems or services via a network. The processing unit 310 may also communicate to and from memory 370 and further provide output information for an optional display 350 via the input / output device interface 340. The input / output device interface 340 may also accept input from the optional input device 360, such as a keyboard, mouse, digital pen, microphone, touch screen, gesture recognition system, voice recognition system, gamepad, accelerometer, gyroscope, or other input device.

[0122] The memory 370 may contain computer program instructions (grouped as modules or components in some embodiments) that the processing unit 310 executes in order to implement one or more embodiments. The memory 370 generally includes RAM, ROM and / or other persistent, auxiliary or non-transitory computer-readable media. The memory 370 may store an operating system 372 that provides computer program instructions for use by the processing unit 310 in the general administration and operation of the computing device 300. The memory 370 may further include computer program instructions and other information for implementing aspects of the present disclosure.

[0123] For example, in one embodiment, the memory 370 includes an obtaining digital information about a subject module 374, a processing the digital information via a processing agent to extract one or more biomarkers module 376, an assessing the subject via an assessment agent based on the one or more extracted biomarkers module 378 and an outputting the assessment to a user module 380.

[0124] METHODS FOR DIGITALLY ASSESSING A SUBJECT

[0125] Aspects of the present disclosure further include methods of digitally assessing a subject. In some embodiments, the method is a computer-implemented method. In some embodiments, the computer-implemented method of digitally assessing a subject includes obtaining digital information about the subject, processing the digital information via a processing agent to extract one or more biomarkers, assessing the subject via an assessment agent based onthe one or more extracted biomarkers and outputting the assessment to a user. A user includes, e.g., a clinician, a researcher, etc. In some embodiments, the method further includes clinically assessing or clinically evaluating the patient based on the digital assessment. In some embodiments, the method further includes referring a subject to a clinician for further assessment based on the digital assessment.

[0126] In some embodiments, the digital information is obtained via an upload agent and / or a transfer agent. Agents (e.g., processing agents, assessment agents, upload agents, transfer agents, etc.) include those discussed above. In some embodiments, the method includes employing any of the systems discussed above to digitally assess the subject.

[0127] FIG. 4 illustrates flow diagram 400 for digitally assessing a subject according to embodiments of the present invention. Flow diagram 400 starts at step 401. Step 401 includes obtaining digital information about the subject. Upon completing step 401, the process next moves to step 402. Step 402 includes processing the digital information via a processing agent to extract one or more biomarkers. Upon completing step 402, the process next moves to step 403. Step 403 includes assessing the subject via an assessment agent based on the one or more extracted biomarkers. Upon completing step 403, the process next moves to step 404. Step 404 includes outputting the assessment to a user.

[0128] Methods of the present disclosure may be used for any digital assessment including, but not limited to, detecting a disease or disability, diagnosing a disease or disability, monitoring a disease or disability, predicting progression of a disease or disability, measuring efficacy of a treatment (e.g., a treatment for a neurological disease), and determining whether a subject is neurologically normal or abnormal. In some cases, the disease is a neurological disease.

[0129] In some embodiments, the method is a method of determining whether a subject is neurologically normal or abnormal, where the method includes applying any of the methods described above (e.g., by employing any of the systems discussed above) to determine whether the subject is neurologically normal or abnormal.In some embodiments, the method is a method of diagnosing a subject as having a neurological disease, where the method includes applying any of the methods described above (e.g., by employing any of the systems discussed above) to diagnose whether the subject has the neurological disease.

[0130] In some embodiments, the method is a method of monitoring a neurological disease in a subject, where the method includes applying any of the methods described above (e.g., by employing any of the systems discussed above) to monitor the neurological disease in the subject.

[0131] In some embodiments, the method is a method of measuring efficacy of a therapy in a subject, where the method includes applying any of the methods described above (e.g., by employing any of the systems discussed above) to measure the efficacy of the therapy in the subject, where the digital information includes digital information about the subject prior to the therapy and digital information about the subject after the therapy has been administered.

[0132] In some embodiments, the method includes digitally assessing a subject over a period of time. Suitable time periods include those discussed above. Such methods of digitally assessing a subject over a period of time may include applying any of the methods described above (e.g., by employing any of the systems discussed above) at multiple time points within a period of time. In some cases, the method includes further assessing the subject based on the results of the assessments at multiple time points. In some embodiments, the method of digitally assessing a subject over a period of time may include applying any of the methods described above (e.g., by employing any of the systems discussed above) at a first time point and a second time point and further assessing the subject based on the results of the assessment at the first time point and the second time point.

[0133] In some embodiments, the method includes digitally assessing a population of subjects. Such methods of digitally assessing a population of subjects may include applying any of the methods described above (e.g., by employing any of the systems discussed above) to digitally assess each subject of the population of subjects.Populations of subjects (e.g., human subjects) may include any of the subjects discussed above. Subjects of the population may be of any ethnicity, age, gender or other physiological or demographic characteristics. In some cases, subjects of the population may share one or more particular ethnicity, age, gender or other physiological or demographic characteristics. In some cases, subjects of the population have been diagnosed with a disease, treated for a disease and / or are at risk for a disease. In some cases, subjects of the population have not been diagnosed with and / or treated for a disease. Diseases (e.g., neurological diseases) include those discussed above.

[0134] Embodiments

[0135] FIG. 5 illustrates flow diagram 500 for digitally assessing a subject according to embodiments of the invention where the digital information includes a self-recorded video of the subject completing a speech task. Flow diagram 500 starts at step 501. Step 501 includes obtaining a self-recorded video of a subject completing a speech task (e.g., via an upload agent and a transfer agent). The speech task may include, e.g., the subject describing an image or a subject describing their morning routine. In some cases, the digital information includes two or more self-recorded videos of the subject completing a speech task (e.g., a first video of the subject describing an image and a second video of the subject describing their morning routine). Upon completing step 501, the process next moves to step 502. Step 502 includes processing the video of the subject completing the speech task via a processing agent. Processing the video may include sub-step 502a of generating a speech transcription from the video (e.g., via OpenAI Whisper) followed by sub-step 502c of applying a computer-implemented model to the speech transcription to identify disfluency features and sub-step 502d of applying a computer-implemented model (e.g., an LLM) to identify speech content features. Processing the video may further include substep 502b of applying a computer-implemented model (e.g., CLASP++) to identify acoustic and linguistic speech features. The processing agent is further configured to extract speech biomarkers from the identified disfluency features,speech content features, acoustic features and linguistic features. Upon completing step 502, the process next moves to step 503. Step 503 includes assessing the subject via an assessment agent based on the one or more extracted speech biomarkers. Upon completing step 503, the process next moves to step 504. Step 504 includes outputting the assessment (i.e. , an assessment based on the speech task) to a user.

[0136] FIG. 6 illustrates flow diagram 600 for digitally assessing a subject according to embodiments of the invention where the digital information includes a self-recorded video of the subject completing a facial task. Flow diagram 600 starts at step 601. Step 601 includes obtaining a self-recorded video of a subject completing a facial task (e.g., via an upload agent and a transfer agent). The facial task may include, e.g., the subject describing an image or a subject describing their morning routine. In some cases, the digital information includes two or more self-recorded videos of the subject completing a facial task (e.g., a first video of the subject describing an image and a second video of the subject describing their morning routine). Upon completing step 601, the process next moves to step 602. Step 602 includes processing the video of the subject completing the facial task via a processing agent. Processing the video may include sub-step 602a of applying a computer-implemented model (e.g., OpenFace) to identify facial landmarks and / or facial action units. As shown in sub-step 602b, if facial landmarks and / or facial action units are not identified in a threshold number of frames, the video is not analyzed. For example, if facial landmarks are identified in less than a threshold number of frames (e.g., identified in < 85% of frames), the video is not analyzed. However, if facial landmarks and / or facial action units are identified in a threshold number of frames (e.g., identified in > 85% of frames), the processing step continues at substep 602c where facial landmarks and / or facial action units are used to automatically remove extraneous activities (e.g., adjusting the camera before beginning the task). Next, at sub-step 602d, the processing agent is configured to extract facial biomarkers from the facial landmarks and facial action units. For example, a facial asymmetry biomarker may be extracted based on the faciallandmarks by, e.g., calculating the ratio of left vs right facial landmark movements in the video. Ratios further from one indicate greater facial asymmetry. Other facial biomarkers such as a blinking biomarker, a frowning biomarker and a smiling biomarker may be extracted based on the facial action units. For example, a blinking biomarker may be extracted by calculating one or more of: the average blinking rate, the maximum blinking rate, the average length of the blink, and variance in blinking. In another example, a frowning biomarker may be determined from metrics related to “brow lowerer” and “lip corner depressor” facial actions units. In another example, a smiling biomarker may be determined from metrics related to “cheek raiser” and “lip puller” action units. Upon completing step 602, the process next moves to step 603. Step 603 includes assessing the subject via an assessment agent based on the one or more extracted facial biomarkers (e.g., facial asymmetry biomarker, blinking biomarker, frowning biomarker, smiling biomarker, etc.). For example, a facial asymmetry biomarker may be used to assess muscle weakness and / or paralysis. In other examples, a blinking biomarker, a frowning biomarker and / or a smiling biomarker may be used to assess neurological disease that affect mood. Upon completing step 603, the process next moves to step 604. Step 604 includes outputting the assessment (i.e. , an assessment based on the facial task) to a user.

[0137] FIG. 7 illustrates flow diagram 700 for digitally assessing a subject according to embodiments of the invention where the digital information includes a self-recorded video of the subject completing a hand task. Flow diagram 700 starts at step 701. Step 701 includes obtaining a self-recorded video of a subject completing a hand task (e.g., via an upload agent and a transfer agent). The hand task may include, e.g., the subject buttoning their shirt or the subject performing a finger tapping task. In some cases, the digital information includes two or more self-recorded videos of the subject completing a hand task (e.g., a first video of the subject buttoning their shirt and a second video of the subject performing a finger tapping task). Upon completing step 701, the process next moves to step 702. Step 702 includes processing the video of the subjectcompleting the hand task via a processing agent. Processing the video may include sub-step 702a of applying a computer-implemented model (e.g., MediaPipe Hand) to identify hand landmarks and / or wrist landmarks. As shown in sub-step 702b, if a threshold number of hand landmarks and / or wrist landmarks are not identified (e.g., if < 10 landmarks identified), the video is not analyzed. However, if a threshold number of hand landmarks and / or wrist landmarks are identified (e.g., if > 10 landmarks identified), the processing step continues at sub-step 702c where hand landmarks and / or wrist landmarks are used to automatically remove extraneous activities (e.g., adjusting the camera before beginning the task). Next, at sub-step 702d, the processing agent is configured to a desired frame rate to seconds to standardize varying frame rates. Next, at sub-step 702e, the processing agent is configured to extract hand biomarkers from the hand landmarks (e.g., index finger landmarks, thumb landmarks) and / or wrist landmarks. For example, when processing a video in which the subject completes a finger tapping task, a finger tapping frequency biomarker, a finger tapping speed biomarker and / or a finger tapping amplitude biomarker may be extracted. A finger tapping frequency biomarker may be extracted by calculating the number of finger taps, divided by the seconds between first and last finger tap. A finger tapping speed biomarker may be extracted by calculating the median pixels traveled per second for each landmark. A finger tapping amplitude biomarker may be extracted by identifying the local maxima of the distance between the thumb and index finger and calculating the median amplitude across all finger taps. In another example, when processing a video in which the subject buttons their shirt, a movement complexity biomarker and / or a path length biomarker may be extracted. A movement complexity biomarker may be extracted by calculating the number of local peaks with a minimum peak prominence of 0.05. A path length biomarker may be extracted by calculating the sum of the magnitude of the vectors created in 2D space. Upon completing step 702, the process next moves to step 703. Step 703 includes assessing the subject via an assessment agent based on the one or more extracted hand biomarkers. For example, hand biomarkers may be used to assess motorfunction, coordination and dexterity of the hands, as well as diseases (e.g., neurological diseases) that affect motor function, coordination and dexterity of the hands. Upon completing step 703, the process next moves to step 704. Step 704 includes outputting the assessment (i.e., an assessment based on the hand task) to a user.

[0138] FIG. 8 illustrates flow diagram 800 for digitally assessing a subject according to embodiments of the invention where the digital information includes a self-recorded video of the subject completing a gait task. Flow diagram 800 starts at step 801. Step 801 includes obtaining a self-recorded video of a subject completing a gait task (e.g., via an upload agent and a transfer agent). The gait task may include, e.g., the subject walking away from the camera or the subject walking towards the camera. Upon completing step 801, the process next moves to step 802. Step 802 includes processing the video of the subject completing the gait task via a processing agent. Processing the video may include sub-step 802a of applying a computer-implemented model (e.g., MediaPipe Pose, Ultralytics) to identify hip landmarks, heel landmarks and ankle landmarks. As shown in sub-step 802b, if walking segments are not identified in a threshold number of frames in the video, the video is not analyzed. However, if walking segments are identified in a threshold number of frames in the video, the processing step continues at sub-step 802c where hip landmarks, heel landmarks and ankle landmarks are used to automatically identify segments in the video in which the subject is walking toward and / or away from the camera. Next, at sub-step 802d, the processing agent is configured to convert frame rate to seconds to standardize varying frame rates. Next, at sub-step 802e, the processing agent is configured to extract gait biomarkers from the hip landmarks, heel landmarks and ankle landmarks. For example, a walking speed biomarker, a stride time biomarker, a stride width biomarker, a gait variation biomarker and / or a gait asymmetry biomarker may be extracted based on the hip landmarks, heel landmarks and ankle landmarks. A walking speed biomarker may be extracted by: 1 ) smoothing vertical (Y) pixel position data of right and left hip and ankle landmarks, 2) at each frame, calculating the average vertical distance betweenright and left hip to ankle, 3) for each second, calculating the change in average pixel height from the first to the last frame, divided by pixel height at first frame, and 4) calculating the median change in pixel height across the entire video. A stride time biomarker may be extracted by: 1) filling the gaps in right and left ankle vertical (Y) position data using linear interpolation, 2) calculating the vertical distance between right and left ankle, and 3) calculating the time between each ankle vertical distance local minima and local maximum. Astride width biomarker may be extracted by: 1) filling the gaps in right and left heel vertical (Y) and horizontal (X) positions using linear interpolation and 2) at frames when vertical distance between heels is zero (both feet on ground), calculating the horizontal distance between heels. A gait variation biomarker and / or a gait asymmetry biomarker may be extracted by calculating the coefficient of variation across all strides and the ratio of values from right to left strides for each spatiotemporal gait parameter (stride time, stride width, etc.). Upon completing step 802, the process next moves to step 803. Step 803 includes assessing the subject via an assessment agent based on the one or more extracted gait biomarkers. For example, gait biomarkers may be used to assess walking impairment and ambulatory dysfunction, as well as diseases (e.g., neurological diseases) that affect walking impairment and ambulatory dysfunction. Upon completing step 803, the process next moves to step 804. Step 804 includes outputting the assessment (i.e. , an assessment based on the gait task) to a user.

[0139] UTILITY

[0140] The methods and systems described herein find use in a variety of applications where it is desirable to digitally assess a subject. The present systems enable standardization of the assessment of subjects. For example, digital assessment decreases variation and inconsistencies compared to current assessments that are carried out by a variety of different clinicians.

[0141] As the present methods and systems are capable of utilizing digital information (e.g., videos) of the subject taken at home (or at locations that are not a clinical setting) and / or filmed by themselves (e.g., “selfie” videos), thepresent methods and systems allow for increased accessibility to medical assessments. Further, patients in need of medical assessment are not hampered by long wait times and / or expensive costs associated with seeing a medical provider. Similarly, in clinical research, quantitative measures that are sensitive to change will enable better precision, earlier findings, and reduced costs.

[0142] The present systems and methods enable digital assessments including, but not limited to: 1) classifying a subject as neurologically normal or abnormal for diagnostic and monitoring purposes, 2) classifying a subject by neurological diagnosis for diagnostic and monitoring purposes, and 3) identifying and quantifying trait severity (i.e. , mood, cognition, gait, dexterity, balance, voice, speech, fatigue, strength) for diagnostic purposes, monitoring purposes, and to evaluate response to pharmacologic and / or rehabilitation interventions. For example, and not by way of limitation, the present systems and methods may be used to differentiate between frontotemporal dementia (FTD) subtypes and FTD from healthy controls using linguistic and acoustic features, monitor changes in fatigue in people with MS using pitch, pause duration, and utterance duration, assess changes in muscle weakness in people with stroke and multiple sclerosis using facial asymmetry, classify by neurological diagnosis using facial action units, monitor disability progression in people with Parkinson’s Disease using finger tapping speed and frequency biomarkers, assess if a novel drug improves motor control in people with MS using path length and movement smoothness biomarkers, assess efficacy of post-stroke gait intervention using gait asymmetry biomarkers, and monitor disability progression and ambulatory dysfunction using biomarkers relating to gait variability and walking speed. The present methods and systems provide a way to streamline clinical evaluation of high priority patients, share results through visualization of digital metric values and summarize messages to clinical and / or research teams.

[0143] The following is offered by way of illustration and not by way of limitation.EXPERIMENTAL

[0144] Example 1: Neurological classification based on a speech task

[0145] The present systems and methods can be used to classify a subject as neurologically normal or abnormal using combinations of digital metrics for diagnostic and monitoring purposes. In this example, a classification can be made between individuals with frontotemporal dementia (FTD) and healthy controls by using linguistic and acoustic speech features.

[0146] As shown in FIG. 9, Patient A and Patient B each upload a video of themselves completing a speech task. The videos are processed by the system to identify speech features that can then be used to extract speech biomarkers. For example, extracting speech biomarkers can include calculating content units (e.g., a speech content biomarker), lexical frequency (e.g., a linguistic biomarker), and familiarity speech features (e.g., a disfluency biomarker). Based on these quantitative speech biomarkers, the subject is assessed as either likely having FTD or not having FTD. In the example shown in FIG. 9, Patient A is assessed as having a probability of FTD that is > 0.5. As such, Patient A’s clinician is notified to evaluate Patient A for FTD. On the other hand, Patient B is assessed as having a probability of FTD that is < 0.5, and thus no action is needed for Patient B.

[0147] Example 2: Neurological diagnosis based on a facial task

[0148] The present systems and methods can be used to classify a subject as 1 ) neurologically normal or abnormal using combinations of digital metrics for diagnostic and monitoring purposes and 2) classify a subject by neurological diagnosis using combinations of digital metrics for diagnostic and monitoring purposes. In this example, a classification can be made between subjects with multiple sclerosis (MS), frontotemporal dementia (FTD), Parkinson’s disease (PD), or healthy control (HC) using facial action units.

[0149] As shown in FIG. 10, Patient C and Patient D each upload a video of themselves completing a facial task. The videos are processed by the system to identify facial features that can then be used to extract facial biomarkers. Forexample, extracting facial biomarkers can include calculating a smiling biomarker, a frowning biomarker and a blinking biomarker based on facial action units. Facial action units related to smiling include AU06 “cheek raiser” and AU 12 “lip corner puller”, facial action units related to frowning include AU04 “brow lowerer” and AU15 “lip corner depressor”, and facial action units related to blinking include AU45 “blink”. Based on these quantitative facial biomarkers, the subject is first classified as either being normal neurologically (i.e. , “healthy control”) or abnormal neurologically. Next, subjects classified as abnormal neurologically are diagnosed as having MS, FTD or PD. A decision tree created through partition analysis may be used to classify participants by neurological diagnosis. In the example shown in FIG. 10, Patient C is assessed as having PD. As such, Patient C’s clinician is notified to evaluate Patient C for PD. On the other hand, Patient D is assessed as being neurologically normal (i.e., healthy control), and thus no action is needed for Patient D.

[0150] Example 3: Evaluate treatment effectiveness based on a hand task

[0151] The present systems and methods can be used to identify and quantify trait severity (i.e., mood, cognition, gait, dexterity, balance, voice, speech, fatigue, strength) using combinations of digital metrics for diagnostic purposes, monitoring purposes, and to evaluate response to pharmacologic and rehabilitation interventions in the home setting. In this example, it is assessed whether a novel drug improves motor control in people with multiple sclerosis (MS) using biomarkers related to path length and movement smoothness.

[0152] As shown in FIG. 11, Patient E and Patient F each upload videos of themselves completing a hand task prior to taking a drug (“baseline visit”) and after taking a drug (“final visit”). The videos are processed by the system to identify hand features that can then be used to extract hand biomarkers. For example, extracting hand biomarkers can include calculating a path length biomarker and a movement complexity biomarker (e.g., a biomarker quantifying movement smoothness). The change in these hand biomarkers before taking the drug and after taking the drug are calculated. These changes in hand biomarkersare used to determine whether the subject is responding to the drug or not. In the example shown in FIG. 11, Patient E shows improved movement smoothness (i.e. , a large change in the movement complexity biomarker) and is thus assessed as positively responding to the drug. On the other hand, Patient F shows minimal improvement or no improvement in movement smoothness (i.e., a negligent change or no change in the movement complexity biomarker) and is thus assessed as being a non-responder for the drug.

[0153] Example 4: Monitor disability progression based on a gait task

[0154] The present systems and methods can be used to identify and quantify trait severity (i.e., mood, cognition, gait, dexterity, balance, voice, speech, fatigue, strength) using combinations of digital metrics for diagnostic purposes, monitoring purposes, and to evaluate response to pharmacologic and rehabilitation interventions in the home setting. In this example, disability progression and ambulatory dysfunction is monitored using biomarkers related to gait variability and walking speed.

[0155] As shown in FIG. 12, Patient G and Patient H each upload videos of themselves completing gait tasks over a period of time. For example, Patient G and Patient H upload videos of a gait task in 2009, 2017 and 2024. The videos are processed by the system to identify gait features that can then be used to extract gait biomarkers. For example, extracting gait biomarkers can include calculating a walking speed biomarker and gait variation biomarker. Such gait biomarkers can be determined at routine intervals as part of the standard care with a neurologist. At each time point, the walking speed biomarker and gait variation biomarker for the subject can be compared to biomarker data from other subjects in a population (e.g., subjects of the same age, subjects with the same disease, etc.). In the example shown in FIG. 12, Patient G shows increased gait variability over time (i.e., an increase in the gait variation biomarker) relative to the population of other subjects, and thus is considered to be at “fall-risk”. As such, a message can be shared with the subject’s clinician to review next steps for high fall risk (e.g., refer to a neurological physical therapist (PT), schedule foran eye exam, etc.). On the other hand, Patient H shows minimal increase or no increase in in gait variability over time relative to the population of other subjects (e.g., the gait variation biomarker stays within the normal range) and is thus assessed to require no change in care plan.

[0156] The below items disclose various aspects of the invention. Each of the aspects described below can be combined with other aspects and embodiments disclosed elsewhere herein, including the claims, where the combinations are clearly compatible. Certain aspects include:

[0157] Aspect 1. A system for digitally assessing a subject, the system comprising:

[0158] an upload agent configured to:

[0159] receive digital information related to a subject; and

[0160] store the digital information in a database;

[0161] a transfer agent configured to:

[0162] scan the database for the digital information;

[0163] standardize the digital information; and

[0164] transfer the digital information to a processing database;

[0165] a processing agent configured to:

[0166] process the digital information by applying a computer-implemented model to identify one or more features; and

[0167] extract one or more biomarkers from the digital information based on the one or more features; and

[0168] an assessment agent configured to:

[0169] assess the subject based on the one or more extracted biomarkers. Aspect 2. The system of Aspect 1 , wherein the processing agent is further configured to select which computer-implemented model to apply based on a task performed by the subject in the digital information.

[0170] Aspect 3. The system of Aspect 2, wherein the task is selected from: a speech task;

[0171] a facial task;a hand task; and

[0172] a gait task.

[0173] Aspect 4. The system of Aspects 2 or 3, wherein:

[0174] when the task is a speech task, the processing agent is configured to: process the digital information by applying a computer-implemented speech analysis model to identify one or more speech features; and

[0175] extract one or more speech biomarkers from the digital information based on the one or more speech features;

[0176] when the task is a facial task, the processing agent is configured to:

[0177] process the digital information by applying a computer-implemented facial analysis model to identify one or more facial features; and

[0178] extract one or more facial biomarkers from the digital information based on the one or more facial features;

[0179] when the task is a hand task, the processing agent is configured to:

[0180] process the digital information by applying a computer-implemented hand analysis model to identify one or more hand features; and

[0181] extract one or more hand biomarkers from the digital information based on the one or more hand features; and

[0182] when the task is a gait task, the processing agent is configured to:

[0183] process the digital information by applying a computer-implemented gait analysis model to identify one or more gait features; and

[0184] extract one or more gait biomarkers from the digital information based on the one or more gait features.

[0185] Aspect 5. The system of Aspect 4, wherein the one or more speech features are selected from one or more of:

[0186] an acoustic feature;

[0187] a linguistic feature;

[0188] a disfluency feature; and

[0189] a speech content feature.Aspect 6. The system of Aspects 4 or 5, wherein when the task is a speech task, the processing agent is further configured to generate a speech transcription from the digital information.

[0190] Aspect 7. The system of Aspect 6, wherein the processing agent is further configured to process the speech transcription by:

[0191] annotating the transcript to identify one or more speech features; and / or applying a computer-implemented model to identify one or more speech features based on content in the speech transcription.

[0192] Aspect 8. The system of any one of Aspects 4 to 7, wherein the one or more speech biomarkers are selected from one or more of:

[0193] an acoustic biomarker;

[0194] a linguistic biomarker;

[0195] a disfluency biomarker; and

[0196] a speech content biomarker.

[0197] Aspect 9. The system of any one of Aspects 4 to 8, wherein the one or more facial features are selected from one or more of:

[0198] a facial landmark; and

[0199] a facial action unit.

[0200] Aspect 10. The system of any one of Aspects 4 to 9, wherein when the task is a facial task, the processing agent is further configured to do one or more of:

[0201] stop processing a video if facial features are not identified in a threshold percentage of frames in the video;

[0202] remove extraneous activities from a video;

[0203] standardize frame rate of a video;

[0204] calculate metrics associated with muscle weakness or paralysis; calculate summary statistics based on the one or more facial features; and identify metrics associated with neurological disease diagnosis.

[0205] Aspect 11. The system of any one of Aspects 4 to 10, wherein the one or more facial biomarkers are selected from one or more of:

[0206] a facial asymmetry biomarker;a blinking biomarker;

[0207] a frowning biomarker; and

[0208] a smiling biomarker.

[0209] Aspect 12. The system of any one of Aspects 4 to 11 , wherein the one or more hand features are selected from one or more of:

[0210] a hand landmark; and

[0211] a wrist landmark.

[0212] Aspect 13. The system of any one of Aspects 4 to 12, wherein when the task is a hand task, the processing agent is further configured to do one or more of:

[0213] stop processing a video if hand features are not identified in a threshold percentage of frames in the video;

[0214] remove extraneous activities from a video;

[0215] standardize frame rate of a video;

[0216] calculate metrics associated with motor function or coordination; and calculate metrics associated with hand dexterity.

[0217] Aspect 14. The system of any one of Aspects 4 to 13, wherein the one or more hand biomarkers are selected from one or more of:

[0218] a finger tapping frequency biomarker;

[0219] a finger tapping speed biomarker;

[0220] a finger tapping amplitude biomarker;

[0221] a movement complexity biomarker; and

[0222] a path length biomarker.

[0223] Aspect 15. The system of any one of Aspects 4 to 14, wherein the one or more gait features are selected from one or more of:

[0224] a hip landmark;

[0225] a heel landmark; and

[0226] an ankle landmark.

[0227] Aspect 16. The system of any one of Aspects 4 to 15, wherein when the task is a gait task, the processing agent is further configured to do one or more of:stop processing a video if gait features are not identified in a threshold percentage of frames in the video;

[0228] remove extraneous activities from a video;

[0229] standardize frame rate of a video; and

[0230] calculate metrics associated with walking impairment and ambulatory dysfunction.

[0231] Aspect 17. The system of any one of Aspects 4 to 16, wherein the one or more gait biomarkers are selected from one or more of:

[0232] a walking speed biomarker;

[0233] a stride time biomarker;

[0234] a stride width biomarker;

[0235] a gait variation biomarker; and

[0236] a gait asymmetry biomarker.

[0237] Aspect 18. The system of any one of the preceding Aspects, wherein the assessment agent is configured to assess the subject based on a combination of two or more extracted biomarkers.

[0238] Aspect 19. The system of any one of the preceding Aspects, wherein the assessment agent is configured to assess the subject based on:

[0239] a speech biomarker;

[0240] a facial biomarker;

[0241] a hand biomarker; and

[0242] a gait biomarker.

[0243] Aspect 20. The system of any one of the preceding Aspects, wherein the assessment agent is further configured to:

[0244] assess whether the subject has a disease or disability;

[0245] assess progression of a disease or disability; and / or

[0246] assess efficacy of a therapy or a treatment.

[0247] Aspect 21. The system of any one of the preceding Aspects, wherein the digital information comprises one or more of: a video and a voice recording.

[0248] Aspect 22. The system of any one of the preceding Aspects, wherein the digital information comprises a plurality of videos or voice recordings.Aspect 23. The system of Aspects 21 or 22, wherein the digital information comprises a voice recording the subject completing a speech task.

[0249] Aspect 24. The system of Aspects 21 or 22, wherein the digital information comprises a video of the subject completing:

[0250] a speech task;

[0251] a facial task;

[0252] a hand task; or

[0253] a gait task.

[0254] Aspect 25. The system of Aspects 23 or 24, wherein:

[0255] the speech task comprises:

[0256] the subject describing an image; or

[0257] the subject describing their routine;

[0258] the facial task comprises:

[0259] the subject describing an image; or

[0260] the subject describing their routine;

[0261] the hand task comprises:

[0262] the subject buttoning a shirt; or

[0263] the subject performing a finger tapping test; and

[0264] the gait task comprises:

[0265] the subject walking.

[0266] Aspect 26. The system of Aspect 25, wherein the speech task and the facial task are the same.

[0267] Aspect 27. The system of any one of the preceding Aspects, wherein the digital information is self-recorded by the subject.

[0268] Aspect 28. The system of any one of the preceding Aspects, wherein the digital information comprises digital information about the subject over time.

[0269] Aspect 29. The system of any one of the preceding Aspects, wherein the system further comprises a visualization agent configured to:

[0270] display one or more biomarkers; or

[0271] display a change in one or more biomarkers over time.Aspect 30. A computer-implemented method of digitally assessing a subject, the method comprising:

[0272] obtaining digital information about the subject;

[0273] processing the digital information via a processing agent to extract one or more biomarkers;

[0274] assessing the subject via an assessment agent based on the one or more extracted biomarkers; and

[0275] outputting the assessment to a user.

[0276] Aspect 31. The method of Aspect 30, wherein the digital information is obtained via an upload agent and / or a transfer agent.

[0277] Aspect 32. A method of determining whether a subject is neurologically normal or abnormal, the method comprising:

[0278] applying the method of Aspect 30 to determine whether the subject is neurologically normal or abnormal.

[0279] Aspect 33. A method of diagnosing a subject as having a neurological disease, the method comprising:

[0280] applying the method of Aspect 30 to diagnose whether the subject has the neurological disease.

[0281] Aspect 34. A method of monitoring a neurological disease in a subject, the method comprising:

[0282] applying the method of Aspect 30 to monitor the neurological disease in the subject.

[0283] Aspect 35. A method of measuring efficacy of a therapy in a subject, the method comprising:

[0284] applying the method of Aspect 30 to measure the efficacy of the therapy in the subject, wherein the digital information includes digital information about the subject prior to the therapy and digital information about the subject after the therapy has been administered.

[0285] Aspect 36. A method of digitally assessing a subject over a period of time, the method comprising:applying the method of Aspect 30 to digitally assess the subject at a first time point and a second time point; and

[0286] further assessing the subject based on the results of the assessment at the first time point and the second time point.

[0287] Aspect 37. A method of digitally assessing a population of subjects, the method comprising:

[0288] applying the method of Aspect 30 to digitally assess each subject of the population of subjects.

[0289] Aspect 38. A non-transitory computer readable storage medium comprising instructions stored thereon, the instructions comprising:

[0290] algorithm for obtaining digital information about a subject;

[0291] algorithm for processing the digital information via a processing agent to extract one or more biomarkers;

[0292] algorithm for assessing the subject via an assessment agent based on the one or more extracted biomarkers; and

[0293] algorithm for outputting the assessment to a user.

[0294] Aspect 39. The non-transitory computer readable storage medium according to Aspect 38, wherein the instructions further comprise:

[0295] algorithm for obtaining the digital information via an upload agent and / or a transfer agent.

[0296] Aspect 40. The non-transitory computer readable storage medium according to Aspect 38, wherein the instructions further comprise:

[0297] algorithm for employing a system according to any one of Aspects 1 to 29 to digitally assess the subject.

[0298] Although the foregoing invention has been described in some detail by way of illustration and example for purposes of clarity of understanding, it is readily apparent to those of ordinary skill in the art in light of the teachings of this invention that certain changes and modifications may be made thereto without departing from the spirit or scope of the appended claims.Accordingly, the preceding merely illustrates the principles of the invention. It will be appreciated that those skilled in the art will be able to devise various arrangements which, although not explicitly described or shown herein, embody the principles of the invention and are included within its spirit and scope. Furthermore, all examples and conditional language recited herein are principally intended to aid the reader in understanding the principles of the invention and the concepts contributed by the inventors to furthering the art and are to be construed as being without limitation to such specifically recited examples and conditions. Moreover, all statements herein reciting principles, aspects, and embodiments of the invention as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents and equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure. Moreover, nothing disclosed herein is intended to be dedicated to the public regardless of whether such disclosure is explicitly recited in the claims.

[0299] The scope of the present invention, therefore, is not intended to be limited to the exemplary embodiments shown and described herein. Rather, the scope and spirit of present invention is embodied by the appended claims. In the claims, 35 U.S.C. § 112(f) or 35 U.S.C. § 112(6) is expressly defined as being invoked for a limitation in the claim only when the exact phrase “means for” or the exact phrase “step for” is recited at the beginning of such limitation in the claim; if such exact phrase is not used in a limitation in the claim, then 35 U.S.C. § 112(f) or 35 U.S.C. § 112(6) is not invoked.

Claims

What is claimed is:

1. A system for digitally assessing a subject, the system comprising:an upload agent configured to:receive digital information related to a subject; andstore the digital information in a database;a transfer agent configured to:scan the database for the digital information;standardize the digital information; andtransfer the digital information to a processing database;a processing agent configured to:process the digital information by applying a computer-implemented model to identify one or more features; andextract one or more biomarkers from the digital information based on the one or more features; andan assessment agent configured to:assess the subject based on the one or more extracted biomarkers.

2. The system of Claim 1 , wherein the processing agent is further configured to select which computer-implemented model to apply based on a task performed by the subject in the digital information.

3. The system of Claim 2, wherein the task is selected from:a speech task;a facial task;a hand task; anda gait task.

4. The system of Claims 2 or 3, wherein:when the task is a speech task, the processing agent is configured to: process the digital information by applying a computer-implemented speech analysis model to identify one or more speech features; andextract one or more speech biomarkers from the digital information based on the one or more speech features;when the task is a facial task, the processing agent is configured to:process the digital information by applying a computer-implemented facial analysis model to identify one or more facial features; and extract one or more facial biomarkers from the digital information based on the one or more facial features;when the task is a hand task, the processing agent is configured to:process the digital information by applying a computer-implemented hand analysis model to identify one or more hand features; and extract one or more hand biomarkers from the digital information based on the one or more hand features; andwhen the task is a gait task, the processing agent is configured to:process the digital information by applying a computer-implemented gait analysis model to identify one or more gait features; andextract one or more gait biomarkers from the digital information based on the one or more gait features.

5. The system of Claim 4, wherein the one or more speech features are selected from one or more of:an acoustic feature;a linguistic feature;a disfluency feature; anda speech content feature.

6. The system of Claims 4 or 5, wherein when the task is a speech task, the processing agent is further configured to generate a speech transcription from the digital information.

7. The system of Claim 6, wherein the processing agent is further configured to process the speech transcription by:annotating the transcript to identify one or more speech features; and / or applying a computer-implemented model to identify one or more speech features based on content in the speech transcription.

8. The system of any one of Claims 4 to 7, wherein the one or more speech biomarkers are selected from one or more of:an acoustic biomarker;a linguistic biomarker;a disfluency biomarker; anda speech content biomarker.

9. The system of any one of Claims 4 to 8, wherein the one or more facial features are selected from one or more of:a facial landmark; anda facial action unit.

10. The system of any one of Claims 4 to 9, wherein when the task is a facial task, the processing agent is further configured to do one or more of:stop processing a video if facial features are not identified in a threshold percentage of frames in the video;remove extraneous activities from a video;standardize frame rate of a video;calculate metrics associated with muscle weakness or paralysis; calculate summary statistics based on the one or more facial features; and identify metrics associated with neurological disease diagnosis.

11. The system of any one of Claims 4 to 10, wherein the one or more facial biomarkers are selected from one or more of:a facial asymmetry biomarker;a blinking biomarker;a frowning biomarker; anda smiling biomarker.

12. The system of any one of Claims 4 to 11 , wherein the one or more hand features are selected from one or more of:a hand landmark; anda wrist landmark.

13. The system of any one of Claims 4 to 12, wherein when the task is a hand task, the processing agent is further configured to do one or more of:stop processing a video if hand features are not identified in a threshold percentage of frames in the video;remove extraneous activities from a video;standardize frame rate of a video;calculate metrics associated with motor function or coordination; and calculate metrics associated with hand dexterity.

14. The system of any one of Claims 4 to 13, wherein the one or more hand biomarkers are selected from one or more of:a finger tapping frequency biomarker;a finger tapping speed biomarker;a finger tapping amplitude biomarker;a movement complexity biomarker; anda path length biomarker.

15. The system of any one of Claims 4 to 14, wherein the one or more gait features are selected from one or more of:a hip landmark;a heel landmark; andan ankle landmark.

16. The system of any one of Claims 4 to 15, wherein when the task is a gait task, the processing agent is further configured to do one or more of:stop processing a video if gait features are not identified in a threshold percentage of frames in the video;remove extraneous activities from a video;standardize frame rate of a video; andcalculate metrics associated with walking impairment and ambulatory dysfunction.

17. The system of any one of Claims 4 to 16, wherein the one or more gait biomarkers are selected from one or more of:a walking speed biomarker;a stride time biomarker;a stride width biomarker;a gait variation biomarker; anda gait asymmetry biomarker.

18. The system of any one of the preceding claims, wherein the assessment agent is configured to assess the subject based on a combination of two or more extracted biomarkers.

19. The system of any one of the preceding claims, wherein the assessment agent is configured to assess the subject based on:a speech biomarker;a facial biomarker;a hand biomarker; anda gait biomarker.

20. The system of any one of the preceding claims, wherein the assessment agent is further configured to:assess whether the subject has a disease or disability;assess progression of a disease or disability; and / orassess efficacy of a therapy or a treatment.

21. The system of any one of the preceding claims, wherein the digital information comprises one or more of: a video and a voice recording.

22. The system of any one of the preceding claims, wherein the digital information comprises a plurality of videos or voice recordings.

23. The system of Claims 21 or 22, wherein the digital information comprises a voice recording the subject completing a speech task.

24. The system of Claims 21 or 22, wherein the digital information comprises a video of the subject completing:a speech task;a facial task;a hand task; ora gait task.

25. The system of Claims 23 or 24, wherein:the speech task comprises:the subject describing an image; orthe subject describing their routine;the facial task comprises:the subject describing an image; orthe subject describing their routine;the hand task comprises:the subject buttoning a shirt; orthe subject performing a finger tapping test; andthe gait task comprises:the subject walking.

26. The system of Claim 25, wherein the speech task and the facial task are the same.

27. The system of any one of the preceding claims, wherein the digital information is self-recorded by the subject.

28. The system of any one of the preceding claims, wherein the digital information comprises digital information about the subject over time.

29. The system of any one of the preceding claims, wherein the system further comprises a visualization agent configured to:display one or more biomarkers; ordisplay a change in one or more biomarkers over time.

30. A computer-implemented method of digitally assessing a subject, the method comprising:obtaining digital information about the subject;processing the digital information via a processing agent to extract one or more biomarkers;assessing the subject via an assessment agent based on the one or more extracted biomarkers; andoutputting the assessment to a user.

31. The method of Claim 30, wherein the digital information is obtained via an upload agent and / or a transfer agent.

32. A method of determining whether a subject is neurologically normal or abnormal, the method comprising:applying the method of Claim 30 to determine whether the subject is neurologically normal or abnormal.

33. A method of diagnosing a subject as having a neurological disease, the method comprising:applying the method of Claim 30 to diagnose whether the subject has the neurological disease.

34. A method of monitoring a neurological disease in a subject, the method comprising:applying the method of Claim 30 to monitor the neurological disease in the subject.

35. A method of measuring efficacy of a therapy in a subject, the method comprising:applying the method of Claim 30 to measure the efficacy of the therapy in the subject, wherein the digital information includes digital information about the subject prior to the therapy and digital information about the subject after the therapy has been administered.

36. A method of digitally assessing a subject over a period of time, the method comprising:applying the method of Claim 30 to digitally assess the subject at a first time point and a second time point; andfurther assessing the subject based on the results of the assessment at the first time point and the second time point.

37. A method of digitally assessing a population of subjects, the method comprising:applying the method of Claim 30 to digitally assess each subject of the population of subjects.

38. A non-transitory computer readable storage medium comprising instructions stored thereon, the instructions comprising:algorithm for obtaining digital information about a subject;algorithm for processing the digital information via a processing agent to extract one or more biomarkers;algorithm for assessing the subject via an assessment agent based on the one or more extracted biomarkers; andalgorithm for outputting the assessment to a user.

39. The non-transitory computer readable storage medium according to Claim 38, wherein the instructions further comprise:algorithm for obtaining the digital information via an upload agent and / or a transfer agent.

40. The non-transitory computer readable storage medium according to Claim 38, wherein the instructions further comprise:algorithm for employing a system according to any one of Claims 1 to 29 to digitally assess the subject.

Images