Movement disorder detection and evaluation method

Abstract

The present disclosure relates to systems and methods for assessing the severity of non-autonomous motion associated with late dyskinesia (TD). The method includes receiving video data of a patient. The video data may include facial movement of the patient. The method includes processing the video data to identify facial landmark data of the patient, which may include applying an image processing technique to the video data to identify facial landmarks on one or more frames of the video data to produce tagged frames. The method may include applying a plurality of trained machine learning models to the facial landmark data to determine one or more motion severity scores based on changes in the patient's facial landmark data, where each motion severity score represents a category of facial motion, and where the one or more motion severity scores represent a category of facial motion. And a total severity score may be generated based on the plurality of motion severity scores.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 444,674, filed on February 10, 2023, the entire contents of which are incorporated herein by reference. Background Art

[0002] Tardive dyskinesia (TD) is a movement disorder that affects the nervous system. For example, TD can cause a series of involuntary, repetitive muscle movements in the face, arms, and legs. Often, people develop TD through prolonged use of certain psychiatric medications. TD is often underdiagnosed because patients are often prescribed psychiatric medications by psychiatrists who monitor their responses to the medications but are not trained to diagnose movement disorders caused by prescribed medications. TD can also be underdiagnosed because trained neurologists may not be aware of the possibility that TD is causing a patient's symptoms.

[0003] For those patients seeking a diagnosis or treatment for TD, it is often recommended that the patient undergo a time-consuming Abnormal Involuntary Movement Scale (AIMS) test. Conventional AIMS testing may require the patient to visit a trained clinician in person, making AIMS testing even more time-consuming. As a result, patients may withdraw from seeking a diagnosis or treatment for TD after not receiving a timely TD diagnosis or after not seeing adequate clinical improvement due to infrequent AIMS testing. Furthermore, the results of the AIMS test may be inconsistent among clinicians because a portion of the test relies, at least in part, on subjective judgments made by the clinician. Summary of the Invention

[0004] The present disclosure relates generally to detecting and assessing movement disorders, and more particularly, to assessing a patient's risk for tardive dyskinesia (TD). The disclosed technology relates to methods and systems (eg, devices) for assessing a patient's risk for TD.

[0005] Methods and systems for assessing the severity of involuntary movements associated with TD may be described herein.

[0006] The method may include receiving video data of a patient. The video data may correspond to facial movements of the patient. The video data may be captured by a device (e.g., a mobile device with a camera). The method may include processing the video data to identify facial landmark data of the patient. The method may also include applying a plurality of trained machine learning models to the facial landmark data to determine a plurality of motion severity scores based on the facial landmark data of the patient (e.g., changes in the facial landmark data of the patient). In some examples, each of the plurality of motion severity scores represents a corresponding category of facial movement (e.g., where each category of facial movement corresponds to a TD symptom). The method may also include processing the plurality of motion severity scores to generate an overall severity score.

[0007] A movement severity score in the plurality of movement severity scores (e.g., each movement severity score) can indicate a severity of an involuntary movement associated with a corresponding category of facial movement. In some examples, the categories of facial movement include one or more of upper facial movement, tongue movement, mouth and lower facial movement, head shaking, and / or head tilt.

[0008] Processing the video data may include applying an image processing technique to the video data to identify facial landmarks on one or more frames of video data to generate labeled frames. The image processing technique may include any combination of a dlib facial recognition method, a Haar Cascade method, a Fisherface method, and / or an elastic graph matching method. In some cases, processing the video data includes identifying facial landmark data corresponding to each of the facial landmarks from the labeled frames. The facial landmark data may include one or more of a landmark identifier, a landmark coordinate location, a frame source identifier, a video source identifier, or a patient identifier.

[0009] In some examples, processing the video data may include segmenting the video data into a plurality of data subsets including video segments of a predefined duration. In some examples, the predefined duration may be less than ten seconds, such as a duration of four seconds. In some cases, the data subsets (e.g., each data subset) may include a video segment that overlaps with at least one other video segment.

[0010] In some examples, the method may include applying a first trained machine learning model to at least a portion of the facial landmark data for each of the data subsets to generate a motion indicator for each of the data subsets, and applying a second trained machine learning model to the motion indicator for each of the data subsets to generate a respective motion severity score for each respective category of facial motion. The motion indicator for a data subset (e.g., each of the data subsets) may be an indicator, such as a binary indicator, indicating whether the respective category of facial motion exists within the data subset. In some cases, the first trained machine learning model is applied to a portion of the facial landmark data that is predetermined to be associated with the respective category of facial motion for each of the data subsets.

[0011] The plurality of motion severity scores may include a first motion severity score associated with a first category of facial motion and a second motion severity score associated with a second category of facial motion, wherein the second category of facial motion is different from the first category of facial motion. In this case, the overall severity score may be based on the first motion severity score and the second motion severity score.

[0012] In some examples, the total severity score corresponds to an Abnormal Involuntary Movement Scale (AIMS) score. In some examples, the total severity score corresponds to a risk of receiving a positive TD diagnosis by a trained physician. In such examples, the method can include generating a notification including a recommendation directing the patient to seek help from a trained clinician when the total severity score is above a predetermined threshold.

[0013] The method may include providing a prompt to the patient to perform the action, for example, by displaying instructions to perform the action (e.g., tilt the head, remain still in a particular orientation, etc.) on a user interface of the electronic device.

[0014] The method may include capturing one or more videos of the patient, wherein the video data of the patient includes one or more videos. For example, in response to one or more generated prompts, the video can be captured by a device (e.g., a mobile device with a camera).

[0015] A plurality of trained machine learning models can be trained using training data comprising facial landmark data extracted from videos of a plurality of patients with a positive TD diagnosis, an Abnormal Involuntary Movement Scale (AIMS) score associated with each of the plurality of patients, and / or a label corresponding to each of the videos indicating the presence of a plurality of different categories of facial movements. The categories of facial movements include upper facial movement, tongue movement, mouth and lower facial movement, head shaking, and / or head tilt.

[0016] The method may include generating a notification indicating the total severity score, and / or displaying the notification on a user interface of the electronic device.

[0017] The method can be used to adjust the medication provided to the patient. For example, the method may include administering a first daily dose of medication associated with TD to the subject after receiving video data, and increasing the daily dose of the medication to a first subsequent daily dose based on the patient's total severity score being above a threshold. The medication may include a vesicular monoamine transporter 2 (VMAT2) inhibitor, for example, wherein the VMAT2 inhibitor includes deutetrabenazine, tetrabenazine, or valbenazine. In some examples, the first daily dose is in accordance with a titration schedule associated with the VMAT2 inhibitor. For example, the first daily dose may be at least about 6 mg / day, the medication may be deutetrabenazine, and the first subsequent daily dose may be at least about 6 mg / day more than the first daily dose.

[0018] The method can be used to update the patient's severity score through multiple iterations. For example, the method can include receiving second video data of the patient, wherein the second video data corresponds to facial movement of the patient occurring at a time after administration of a first subsequent daily dose of the medication. The method can also include processing the second video data to identify second facial landmark data of the patient, and applying a plurality of trained machine learning models to the second facial landmark data to determine a plurality of second movement severity scores based on changes in the second facial landmark data of the patient. The method can also include processing the plurality of second movement severity scores to generate a second total severity score, and increasing the daily dose of the medication to a second subsequent daily dose based on the second total severity score of the patient being above a second threshold.

[0019] Furthermore, in some examples, the method may include administering a first daily amount of a medication associated with a TD to the subject after receiving the video data, receiving second video data of the patient, wherein the second video data corresponds to facial movement of the patient occurring at a time after the patient's medication regimen was adjusted, processing the second video data to identify second facial landmark data of the patient, applying the plurality of trained machine learning models to the second facial landmark data to determine a plurality of second motion severity scores based on changes in the patient's second facial landmark data, and processing the plurality of second motion severity scores to generate a second total severity score. In such examples, the method may also include determining that the total severity score is above a threshold and / or generating a recommendation to adjust the medication's medication regimen based on the patient's total severity score being above the threshold.

[0020] Also described herein are methods and systems for titrating a medication for a patient associated with tardive dyskinesia (TD). In one example, the method includes receiving video data of the patient, wherein the video data corresponds to facial movements of the patient, and processing the video data to identify facial landmark data of the patient. The method may also include applying a plurality of trained machine learning models to the facial landmark data to determine a motion severity score based on the patient's facial landmark data, and increasing a daily dose of a VMAT2 inhibitor based at least in part on the patient's motion severity score being above a threshold. In some cases, the method includes administering a daily dose of a vesicular monoamine transporter 2 (VMAT2) inhibitor to the subject. The VMAT2 inhibitor may include any combination of deuterated tetrabenazine, tetrabenazine, or valbenazine.

[0021] In some cases, the method includes applying multiple trained machine learning models to facial landmark data to determine multiple motion severity scores based on changes in the patient's facial landmark data, wherein each of the multiple motion severity scores represents a respective category of facial motion corresponding to TD symptoms, and processing the multiple motion severity scores to generate an overall severity score. The daily dose of the VMAT2 inhibitor can be increased based on the overall severity score. In some cases, processing the video data includes segmenting the video data into multiple data subsets comprising video segments of predefined duration. In this case, the method can include applying a first trained machine learning model to at least a portion of the facial landmark data for each of the data subsets to generate a motion indicator for each of the data subsets, and applying a second trained machine learning model to the motion indicator for each of the data subsets to generate a respective motion severity score for each respective category of facial motion.

[0022] Described herein are methods and systems for training one or more machine learning models capable of assessing the severity of involuntary movements associated with TD. In some examples, the method may include receiving video data of a patient, wherein the video data corresponds to facial movements of the patient. The method may include processing the video data to identify facial landmark data of the patient, and applying a plurality of trained machine learning models to the facial landmark data to determine a movement severity score based on changes in the facial landmark data of the patient. The plurality of trained machine learning models may be trained using training data comprising facial landmark data extracted from videos of a plurality of patients with a positive TD diagnosis, an Abnormal Involuntary Movement Scale (AIMS) score associated with each of the plurality of patients, and / or a label corresponding to each of the videos indicating the presence of a plurality of different categories of facial movements.

[0023] In some examples, the method may include segmenting the video data into a plurality of data subsets comprising video segments of a predefined duration. In such an example, a first trained machine learning model may be trained based on at least a portion of the facial landmark data for each of the data subsets to generate a motion indicator for each of the data subsets, and a second trained machine learning model may be trained based on the motion indicator for each of the data subsets to generate a respective motion severity score for each respective category of facial motion. The data subsets (e.g., each of the data subsets) may include a video segment that overlaps with at least one other video segment. The facial landmark data may include one or more of a landmark identifier, a landmark coordinate location, a frame source identifier, a video source identifier, and / or a patient identifier.

[0024] Finally, methods and systems for assessing the severity of involuntary movements associated with TD may include receiving video data of a patient (e.g., where the video data corresponds to facial movements of the patient), processing the video data to identify facial landmark data of the patient, and applying multiple trained machine learning models to the facial landmark data to determine a movement severity score based on changes in the patient's facial landmark data.

[0025] The methods described herein can be performed by one or more devices, where, for example, each device can include a memory configured to store instructions and one or more processors configured to read the stored instructions. In one or more instances, the disclosed technology relates to a system for assessing a patient's risk of tardive dyskinesia. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1A A schematic diagram of an example system environment is shown.

[0027] Figure 1B The diagram shows Figure 1A An example block diagram of one or more portions of an example system environment is provided.

[0028] Figure 2 is a flow chart illustrating an example implementation of assessing a patient's risk for tardive dyskinesia.

[0029] Figure 3 An example interface is shown displaying example prompts to a patient.

[0030] Figure 4 is a block diagram of an example computing device.

[0031] Figure 5 is a diagram of an example mask including one or more landmarks associated with a patient. DETAILED DESCRIPTION

[0032] The following discussion omits or only briefly describes conventional features of the treatment or diagnostic system, which are obvious to those skilled in the art. It should be noted that various embodiments are described in detail with reference to the accompanying drawings, in which the same reference numerals represent the same parts and components throughout the several views. Reference to various embodiments does not limit the scope of the appended claims. In addition, any examples set forth in this specification are intended to be non-limiting and only set forth some of the many possible embodiments of the appended claims. In addition, the specific features described herein can be used in combination with the features of other descriptions in each of the various possible combinations and arrangements.

[0033] Unless otherwise specifically defined herein, all terms are to be given their broadest possible interpretation, including the meanings implied in the specification and the meanings understood by those skilled in the art and / or as defined in dictionaries, treatises, etc. It must also be noted that, as used in the specification and the appended claims, the singular forms "a," "an," and "the" include plural referents unless otherwise indicated, and the terms "include" and / or "comprise" when used in this specification specify the presence of the stated features, elements, and / or components. However, this does not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof.

[0034] For the diagnosis and assessment of tardive dyskinesia (TD), the Abnormal Involuntary Movement Scale (AIMS) is the main tool used by clinicians. AIMS is a test included in the combination of projects rated on a 5-point anchor scale, including oral, limb and trunk movements, and the global severity of movement judged by the examiner, and yes-no questions about dental health (e.g., rating on a binary scale). In order to assess the severity of each project assessed on the anchor scale, the clinician instructs the patient to carry out a series of specific activities, such as instructing the patient to stick out their tongue, bend and stretch the patient's arm etc. The 5-point anchor scale is used to indicate the severity of the involuntary movements of the patient's body. For example, a movement severity score of zero indicates no involuntary movement, a movement severity score of one indicates minimal or normal involuntary movement, a movement severity score of two indicates mild involuntary movements, a movement severity score of three indicates moderate involuntary movements, and a movement severity score of four indicates severe involuntary movements.

[0035] The AIMS should only be administered by appropriately trained clinicians. TD can often be underdiagnosed or improperly diagnosed because it develops in patients who have been prescribed certain psychiatric medications for an extended period by clinicians (e.g., psychiatrists or other medical professionals) who are generally not certified to administer the AIMS. Therefore, access to an accurate TD diagnosis can be difficult. Even when TD has been diagnosed by a certified clinician, the complexity of the AIMS test can prevent TD patients from obtaining regular AIMS testing to track and record treatment progress, which can lead to frustration in treatment and subsequent treatment abandonment. Furthermore, because the AIMS involves implicit subjective determinations, AIMS scores can vary even between trained clinicians.

[0036] Therefore, there is a need for a diagnostic tool for easily and consistently assessing patients' risk and symptoms of TD.

[0037] Figure 1A A schematic diagram of a system environment (or "environment") 100 for implementing a TD diagnostic system as described herein is shown. As illustrated, the environment 100 includes one or more server devices 102 connected to one or more of a device 106 (e.g., a clinician device), a device 108 (e.g., a patient device), and a database 110 via a network 112. Figure 1B Shown Shown Figure 1A 1 is a block diagram of an example of one or more portions of an example system environment 100.

[0038] like Figure 1A As shown, server device(s) 102, device 106, device 108, and database 110 may communicate with each other via a network 112. Network 112 may include any suitable network through which computing devices may communicate. Network 112 may include wired and / or wireless communication networks. Example wireless communication networks may include one or more types of radio frequency (RF) communication signals using one or more wireless communication protocols, such as a cellular communication protocol, a wireless local area network (WLAN) or WiFi communication protocol, and / or another wireless communication protocol. Although Figure 1A The components of environment 100 are shown communicating via network 112, but it is understood that the components of environment 100 can communicate directly with each other, eg, bypassing network 112. For example, device 108 can communicate directly with server device 102.

[0039] Server device 102 can generate, receive, analyze, store and / or transmit digital data, for example patient's video data.In one or more cases, server device 102 can communicate with device 106 and device 108.For example, server device 102 can send data to device 106, including video data or other information, such as motion severity score and / or total severity score.In another example, server device 102 can receive input from a user (for example, patient) or from a clinician via device 106 via device 108.In some cases, (one or more) server devices 102 may include a distributed server set, wherein (one or more) server devices 102 include a plurality of server devices distributed across network 112.In some examples, (one or more) distributed server devices 102 may be located at the same location or different physical locations.In other cases, (one or more) server devices 102 may include a content server, an application server, a communication server, a web hosting server or another type of server.

[0040] In one or more cases, (one or more) server devices 102, devices 106 and / or devices 108 may include an assessment system 104 or a portion thereof. The assessment system 104 can analyze the patient's video data and identify the patient's landmarks. In addition, the assessment system 104 can apply one or more trained machine learning models (such as assessment model 114) to the video data and the identified landmarks to determine the spatial arrangement and time displacement of the identified landmarks. In some cases, the assessment system 104 can identify the landmarks before applying the trained machine learning model to the identified landmarks. For example, the assessment system 104 can utilize one or more image processing techniques (e.g., Haar cascade technology, integral projection technology, Fisher face technology, elastic graph matching technology, radial basis function network recognition technology, hidden Markov model technology and other similar technologies for facial recognition).

[0041] The assessment system 104 can apply a trained machine learning model to video data (e.g., corresponding to a video of the patient or a series of videos of the patient) to determine the movement of landmark positions over a period of time. After determining the spatial arrangement and temporal displacement of the identified landmarks, the trained machine learning model (such as assessment models 113a-113e, 114a-114e) can determine one or more motion severity scores indicating the severity of the involuntary movements of the patient's body. Each motion severity score can be associated with a different (e.g., unique) movement of the patient, such as upper facial movement, tongue movement, mouth and lower facial movement, head shaking and / or head tilt. In some cases, based at least on the motion severity score, the assessment system 104 generates a total severity score indicating the risk or severity of the movement disorder being analyzed. The motion severity score and / or the total severity score can be on the scale used for the AIMS test, or can be on a different scale (e.g., 1 to 10, A to F, etc.).

[0042] It should be noted that the assessment system 104 and processes described herein relate to determining evidence of TD. However, it should be noted that the processes described herein for detecting TD are exemplary in nature and are not limited to detecting only TD. Therefore, it should be understood that the processes described herein can be used to detect and provide treatment for other movement disorders (e.g., Huntington's disease).

[0043] The assessment system 104, or one or more portions thereof, residing on the device 106 or device 108, can allow for on-device analysis to assess a patient's risk for a movement disorder, such as, but not limited to, TD. The assessment system 104, or one or more portions thereof, residing on the device 106 or device 108, can allow the respective device, such as the device 106 or device 108, to assess a patient's risk for TD or the severity of a patient's TD. In one or more instances, the assessment system 104, or one or more portions thereof, can reside on the server device(s) 102, such that the device 106 and / or device 108 can offload certain risk analysis tasks to be performed at the server device(s) 102 and / or request information from the server device(s) 102 to enable the device 106 and / or device 108 to perform as described herein.

[0044] In one or more cases, the evaluation system 104 operates on a central server (such as (one or more) server devices 102) and can be utilized by one or more computing electronic devices (such as devices 106 and 108) via an application downloaded from the central server or a third-party application store and executed on the one or more computing electronic devices. In one or more cases, the evaluation system 104 can be a software-based program downloaded from a central server (such as (one or more) server devices 102) and installed on one or more computing electronic devices (such as devices 106 and 108). In one or more cases, the evaluation system 104 can be provided as a software service by a third-party cloud service provider (not shown). In one or more cases, the evaluation system 104 can be pre-installed on one or more computing electronic devices as software and / or firmware. In one or more cases, the evaluation system 104 can be installed on one or more computing electronic devices via an external storage device (such as a universal serial bus (USB) flash drive).

[0045] In one or more instances, device 108 is an electronic computing device, such as a desktop computer, laptop computer, tablet computer, personal digital assistant (PDA), smartphone, thin client, or any other electronic device or computing system capable of communicating with server device(s) 102 over network 112. Device 108 can be a client of server device(s) 102. Device 108 can be configured with a camera (e.g., Figure 3 ) or another type of imaging device suitable for acquiring images and video (e.g., a video of the patient performing actions in response to prompts from the assessment system 104). In other cases, the device 108 can be any suitable type of mobile device capable of running a mobile application, including a smartphone, a tablet, a candy bar tablet, or any type of device running a mobile operating system. For example, the device 108 can be a mobile device operated by a user (such as a patient) and capable of connecting to a network (such as the network 112) to transmit captured video of the patient. In other cases, the device 108 can be any wearable electronic device capable of sending, receiving, and processing data, such as a head-mounted display, a smartwatch, etc.

[0046] In one or more cases, the device 108 may include a user interface for providing an end user with the ability to interact with the assessment system 104. The user interface refers to the information (such as graphics, text, and sound) presented to the user by the assessment system 104, as well as the control sequences employed by the user to control the assessment system 104 and respond to prompts generated by the assessment system 104. The user interface may be, for example, a keyboard that allows the user to enter text, a camera that can recognize user gestures and / or objects, a touch screen that accepts input from the user via touching a body part and / or a stylus, etc. The user may access the assessment system 104 through the user interface to enable the assessment system 104 to operate on the user's device (such as the device 108).

[0047] In one or more cases, device 106 includes one or more of the same or similar features as discussed with respect to device 108. Therefore, a description of these features will not be repeated. In other embodiments, device 106 may include a system including a digital camera or other motion recording device in communication with a separate electronic computing device.

[0048] Figure 1B Shown Shown Figure 1A 10. A block diagram of an example of one or more portions of an example system environment 100 is provided. The database 110 is configured to store information such as captured video data 112, facial landmark data 118, motion severity data 122, risk data 124, patient information, and the like. Additionally, the database 110 may be configured to store trained machine learning models, such as, but not limited to, assessment models 113a-113e, 114a-114e. It should be noted that the database 110 may store information for more than one patient. It should also be noted that the information stored within the database 110 may be tagged with an identifier that will be associated with each piece of data associated with the patient. Additionally, the information stored within the database 110 may be stored as anonymized data. As described herein, the information stored within the database 110 may be located on the database 110, on the server device(s) 102, or distributed across the database 110, the server device(s) 102, and other data repositories and servers. Additionally, while Figure 1B The assessment models 113a - 113e , 114a - 114e are shown as being located on the database 110 , but it should be understood that the assessment models 113a - 113e , 114a - 114e may also be located on the assessment system 104 or distributed across the database 110 and the assessment system 104 .

[0049] In one or more embodiments, the assessment system 104 includes a data preprocessor 116, features of which may be implemented in hardware and / or software. The data preprocessor 116 may be configured to receive a raw signal, such as video data 112, and process the raw signal to remove invalid data, outliers, and the like. For example, the data preprocessor 116 may be configured to receive a raw signal representing video data 112 captured by a camera, such as a camera of the device 106 or 108. The data preprocessor 116 may be configured to process the raw video data 112 to determine facial landmark data 118, which, in some examples, may be used to identify the patient's landmarks. In one or more embodiments, the data preprocessor 116 may process the raw signal in an online mode, i.e., using one or more data buffers to process the raw signal as it is received, for example, from a camera of the device 108. In one or more other embodiments, the data preprocessor 116 may process the raw signal in an offline mode, i.e., processing the raw signal retrieved from a data repository, such as the database 110. For situations where the server device(s) 102 utilize a distributed computing system, the data pre-processor 116 utilizes the distributed computing components of the server device(s) 102 to process the raw signals in parallel.

[0050] In one or more instances, the assessment system 104 includes an assessment engine 120 that can be implemented in hardware. In one or more other instances, the assessment engine 120 can be implemented as a memory in a tangible, non-transitory memory such as a memory (e.g., Figure 4 404)) maintained in the memory 404)) of the executable program, which can be executed by one or more processors (e.g., such as Figure 4 As further described herein, the assessment engine 120 may be configured to implement the assessment models 113a-113e, 114a-114e to generate motion severity data 122 and risk data 124 and determine the patient's risk of TD or the severity of TD symptoms.

[0051] The first trained machine learning models 113a-113e can be configured to determine whether a particular category of facial motion associated with TD (e.g., upper facial motion, tongue motion, mouth and lower facial motion, head shaking, and head tilt) is present within the video data. In some examples, the first trained machine learning models 113a-113e can determine whether the category of facial motion is associated with TD based on the facial landmark data 118. In some examples, the output of each of the first trained machine learning models 113a-113e can be a binary indication of whether a particular category of facial motion associated with TD is present within the video clip. In other examples, the output of each of the first trained machine learning models 113a-113e can include a percentage probability of whether a particular category of facial motion associated with TD is present within the video clip. In this way, the first trained machine learning models 113a-113e can generate a motion indicator (e.g., such as a numerical indication) that indicates whether a particular category of facial motion associated with TD is present within the video clip. Thus, the evaluation system 104 may apply the trained machine learning models 113a-113e to a reduced amount of data, namely, facial landmark data 118 associated with the facial movement category corresponding to the particular algorithm 113a-113e.

[0052] The assessment engine 120 can apply the assessment model 114a-114e to the motion indicator generated by the assessment model 113a-113e to generate motion severity data. Motion severity data (e.g., a motion severity score) can indicate the severity of the motion. In some examples, the motion severity data can be specific to each of a plurality of different categories of facial motion. The motion severity score can be scored on a scale indicating the severity of the involuntary motion of the patient's body. In one example, the scale corresponds to the 0-4 scale used in the AIMS test. For example, a motion severity score of zero can indicate no involuntary motion, a motion severity score of one can indicate minimal or normal involuntary motion, a motion severity score of two can indicate mild involuntary motion, a motion severity score of three can indicate moderate involuntary motion, and a motion severity score of four can indicate severe involuntary motion. However, it is contemplated that other scales (0-1, 0-10, 0-100, etc.) can be utilized.

[0053] The assessment system 104, and preferably the assessment engine 120, can apply first trained machine learning models 113a-113e to facial landmark data within the video data to generate motion indicators for facial motion categories. Each of the first trained machine learning models 113a-113e can be unique for a particular category of facial motion (e.g., model 113a for eye blink analysis, model 113b for head tilt analysis, etc.). The assessment system 104, and preferably the assessment engine 120, can apply corresponding second trained machine learning models 114a-114e (e.g., model 114a for eye blink analysis, model 114b for head tilt analysis, etc.) to the motion indicators generated by the corresponding first trained machine learning models 113a-113e to generate motion severity data based on the entire assessment session for a particular patient. Thus, the assessment engine can utilize two trained machine learning models unique to each category of facial movement (e.g., models 113a, 114a for blink analysis, models 113b, 114b for head tilt analysis, etc.) to generate movement severity data for the patient.

[0054] Figure 2 is a flow chart illustrating an example process 200 for assessing a patient's risk of TD. An assessment system (such as assessment system 104) can perform one or more steps of process 200. Although primarily described in the context of a process performed by assessment system 104, in some examples, one or more of devices 106 and / or devices 108 can perform any combination of the steps of process 200 (e.g., in combination with assessment system 104). The risk of TD can be assessed in various locations. For example, a patient can perform an assessment at home (e.g., via device 108). In another example, a patient can perform an assessment at a clinician's office (e.g., via device 106). In one or more instances, to access assessment system 104, a patient can log in to an assessment portal via, for example, but not limited to, a telemedicine application. It should be noted that in the examples provided herein, a patient can perform an assessment via device 108. However, it should be understood that one, more, or all of the processes discussed herein for performing an assessment via device 108 can also be implemented via device 106.

[0055] The assessment system 104 may generate one or more preliminary prompts to a corresponding device (e.g., device 108) to prepare the patient to begin the assessment (201). The device 108 may display the preliminary prompts on a user interface of the device 108 (e.g., via user interface 302). For example, one prompt may instruct the patient to remove any objects (e.g., gum or candy) from the patient's mouth. In another example, another prompt may instruct the patient to begin the assessment while sitting in a hard, sturdy, and armless chair. In another example, another prompt may instruct the patient to remove their shoes and socks. In yet another example, another prompt may instruct the patient to position the device 108 so that the device 108 is in a specific position and the camera 304 of the device 108 can capture video data of the patient. For example, the prompt may specify that the camera 304 of the device 108 should be placed at a specific distance from the patient and / or at a specific height relative to the patient. Additionally, the prompt may specify that the camera 304 of the device 108 should be placed in a stable, stationary position. One example of a prompt may be a notification generated via a graphical user interface (GUI) on a display device of one or more devices described herein, such as a notification provided via a display of device 106 and / or device 108 .

[0056] The prompt to perform the action (202) is preferably generated by the assessment system 104. For example, the assessment system 104 may provide the prompt to the device 108, and the device 108 may display the prompt on the user interface. For example, the displayed prompt 306 may instruct the patient to "Please open your mouth for 30 seconds," e.g., Figure 3 In one or more instances, user interface 302 can display a timer (e.g., timer 307) that displays the time corresponding to the displayed prompt (e.g., displayed prompt 306). When capturing video data of the patient, the timer can provide an indication of the length of time remaining to complete the prompted instruction. For example, when the camera begins capturing video of the patient with their mouth open, the timer can begin counting down from a predefined time period (e.g., 30 seconds).

[0057] The assessment system 104 may generate a prompt to perform a scripted action. The prompt for the scripted action may include, for example, but not limited to, instructions for the patient to perform a series of actions. For example, the prompt may instruct the patient to open their mouth for a first period of time, briefly close their mouth upon expiration of the first period of time, and then open their mouth for a second period of time. In another example, the prompt may instruct the patient to open their mouth and stick out their tongue twice. In other examples, the prompt for the scripted action may include, for example, but not limited to, instructions for the patient to perform a single activity.

[0058] In addition to or as an alternative to providing prompts to perform scripted actions, the assessment system 104 can generate prompts to perform non-scripted actions. Prompts for non-scripted actions can include, for example, but not limited to, instructions for the patient to gaze at the display screen of the device 108 for a period of time.

[0059] Furthermore, in other examples, the prompt may be for the patient to remain still (eg, not move) for a duration, such as when the patient holds their head, face, and / or mouth in a particular orientation.

[0060] Video data captured during the assessment session is received, such as by the assessment system 104 (204). For example, the assessment system 104 may capture video data during the assessment session via a camera of the device 108 (e.g., camera 304). For example, during the assessment session, the camera may record video of the patient resting or performing body movements in response to prompts displayed on a user interface (e.g., user interface 302). The device 108 may provide the captured video as video data to the assessment system 104. In one or more instances, the device 108 may begin recording the video at the start of the assessment session. In one or more other instances, the device 108 may begin recording the video in response to receiving a prompt from the assessment system 104. In one or more instances, the assessment system 104 may store the video data (e.g., video data 112) in the database 110. The video data may be captured by a camera (e.g., the camera of the device 108) that is focused on a specific area of ​​the patient's body. For example, the camera is configured to capture video data of an area of ​​the patient's face.

[0061] In one or more instances, the video data is processed to identify landmarks of the patient (206), such as by the assessment system 104. For example, the assessment system 104, such as the data pre-processor 116, can extract and analyze video data, such as the video data 112, to identify landmarks of the patient (e.g., facial landmark data and / or other body landmarks, such as landmarks of the hands, feet, or torso) by utilizing one or more image processing techniques (e.g., dlib facial recognition, Haar cascade techniques, Fisherface methods, elastic graph matching techniques, and other similar techniques for facial recognition).

[0062] In the example of dlib facial recognition, the evaluation system 104 may receive video data of a patient, and the video data may include multiple frames of the patient's face. The evaluation system 104 may include a detector for identifying faces within each frame of the video data, a shape predictor for identifying landmarks 117 within the video data (e.g., to accurately locate faces), and / or a facial recognition model (e.g., such as dlib facial recognition). The evaluation system 104 may map the frames of video data including images of the patient's face into a multidimensional vector space (e.g., a 128-dimensional vector space). In some examples, the evaluation system 104 may be configured to identify a bounding box of the patient's face (e.g., identifying a set of landmarks 117 associated with the patient's face within each frame of the video data). The evaluation system 104 may perform facial recognition by mapping the landmarks associated with the patient's face into the multidimensional vector space and then comparing the Euclidean distance of the identified landmarks to a distance threshold (e.g., a distance threshold of 0.6) to ensure accurate and consistent recognition of the patient's face across the frames of video data. In some examples, the evaluation system 104 may determine an accuracy close to 100% (eg, 99.38%) on standard labeled faces in the Face Recognition in the Wild (LFW) benchmark.

[0063] refer to Figure 5 In some embodiments, the data pre-processor 116 may extract and analyze the video data 112 and output a landmark 117 comprising a plurality of unique, uniquely labeled landmarks (e.g., as an example, Figure 51-60 in the video data). In this case, the data pre-processor 116 can extract and analyze the video data 112 of each video frame to determine the mask 115 and, likewise, determine the patient's constituent landmarks 117. Such landmarks 117 can correspond to physical landmarks on the patient's face, such as the human subject's facial contour, eyes, nose, lips, tongue, nasion, inion, lateral canthus, external auditory meatus (e.g., ear attachment point), one or more preauricular points, etc. For example, the points representing the landmarks 117 can be indicated as coordinates expressed in pixels and / or pixel gradients corresponding to the predicted region or landmark (e.g., X and Y coordinates for a standard facial dataset, or X, Y, and Z coordinates for a three-dimensional facial dataset). Using the image processing techniques described above, the assessment system 104 (e.g., the data pre-processor 116) can extract facial landmark data 118 corresponding to each of the landmarks 117 identified in one or more frames of video data 112 and store it as a data array and / or a data table. For example, the facial landmark data 118 corresponding to a particular landmark 117 can be in the form of a data array 119. The facial landmark data 118 can include a landmark identifier, a landmark coordinate location, a frame source identifier, a video source identifier, a patient (e.g., anonymous) identifier, etc. The assessment system 104 can store the patient's identified landmarks 117 as facial landmark data 118 in the database 110. Finally, it should be understood that Figure 5 The landmarks 117 shown in FIG. 1 are a subset of the actual number of landmarks that may be recognized by the data pre-processor 116 of the evaluation system 104 .

[0064] It should be noted that the examples described herein relate to the assessment system 104 processing captured video data 112 during an assessment session. However, it should be understood that in other examples, the assessment system 104 can obtain captured video data 112 during an assessment session and store the video data 112 in the database 110. Furthermore, after providing one or more prompts to the patient and capturing a video of the patient performing body movements in response to the corresponding prompts, the assessment system 104 can retrieve the stored video data 112 and process the video data 112 after the patient completes the examination procedure.

[0065] The assessment system 104 may analyze the landmarks 117 to determine whether the action performed by the patient corresponds to a prompt (e.g., the prompt generated at 202) (208). In one or more cases, the assessment system 104 may be preprogrammed to associate one or more of the identified landmarks 117 with each prompt. For example, the assessment system 104 may be preprogrammed to associate the prompt "Please open your mouth for 30 seconds" with a first subset of landmarks (e.g., landmarks labeled 4-14) and to associate the prompt "Please open your mouth and stick out your tongue" with a second subset of landmarks 118 (e.g., landmarks labeled 49-60). In some cases, for example, by associating a particular subset of the identified landmarks 117 with a prompt, the assessment system 104 may analyze those landmarks 117 to determine whether the action performed corresponds to the prompt (208). For example, in response to providing the prompt "Please open your mouth for 30 seconds" to the patient, the assessment system 104 may analyze the first subset of landmarks 117 to determine the spatial arrangement and temporal shifts in those particular landmarks 117.

[0066] The assessment system 104 can determine that the performed action does not correspond to a cue based on movement in the spatial arrangement of corresponding landmarks 117. For example, the assessment system 104 can determine that no landmarks 117 associated with the patient's cheeks, jaw, lips, and / or tongue moved in the captured video data 112 (i.e., the patient did not open the patient's mouth). In other cases, the assessment system 104 can determine that the performed action does not correspond to a cue if the positional displacement of the spatial arrangement of corresponding landmarks 117 does not exceed a threshold. For example, the assessment system 104 can detect slight movement of the patient's lips and jaw based on the spatial arrangement and displacement of associated landmarks 117 in a series of frames from the captured video data 112. However, the assessment system 104 can determine that the spatial arrangement and displacement of the associated landmarks 117 did not shift by more than a predetermined threshold (i.e., the patient's mouth was not open enough to analyze the indication of the landmarks associated with the patient's tongue). In this way, the assessment system 104 can determine that the performed action does not correspond to the provided cue.

[0067] The assessment system 104 can determine that the action performed corresponds to the provided prompt by applying one or more classifiers associated with the prompted action to at least a portion of the temporal shift of the identified landmarks. In some cases, each classifier can correspond to one or more orofacial movements of the patient. The assessment system 104 can select one or more orofacial movements of the corresponding classifier and compare the selected orofacial movements with the temporal shift of the identified landmarks. In one or more cases, the assessment system 104 can determine that the patient performed the correct action based on the temporal shift of the identified landmarks being correctly associated with the selected orofacial movements.

[0068] For the case where it is determined that the action performed does not correspond to the prompt (208: No), the assessment system 104 provides the same prompt (202) to the device 108, and the device 108 displays the prompt on the user interface 302. It should be noted that in some cases where the assessment system 104 detects that the patient has not performed the action corresponding to the prompt, the assessment system 104 may issue another prompt or instruction for display on the user interface 302, which further guides the patient to perform the action corresponding to the prompt.

[0069] If the positional displacement of the spatial arrangement of the landmarks exceeds a threshold, the assessment system 104 can determine that the action being performed corresponds to the cue. For example, the assessment system 104 can detect movement of the patient's lips and jaw based on the spatial arrangement and displacement of the associated landmarks in the captured video data 112. Additionally, the assessment system 104 can determine that the spatial arrangement and displacement of the associated landmarks exceed a predetermined threshold (e.g., the patient's mouth is open enough to analyze an indication of a landmark associated with the patient's tongue). In this way, the assessment system 104 can determine that the action being performed corresponds to the provided cue.

[0070] In the event that it is determined that the action performed does correspond to a prompt (208: YES), the assessment system 104 preferably makes a determination as to whether the prompts for the assessment session are complete (210). For example, an assessment session for certain movement disorders may include a patient responding to a series of prompts to perform corresponding actions. In response to the patient performing an action corresponding to the provided prompt and the assessment system 104 generating a corresponding movement severity score, the assessment system 104 may determine whether the prompts for the assessment session are complete. Furthermore, it should be understood that in some examples, the system may not determine whether the action performed corresponds to a prompt (e.g., 208 may be omitted from process 200).

[0071] For situations where the prompt for the assessment session is determined to be incomplete (210: No), the assessment system 104 provides another prompt to perform an action, as described with respect to 202. For situations where the prompt for the assessment session is determined to be complete (210: Yes), the assessment system 104 may apply one or more machine learning models to the obtained video data to generate one or more motion severity scores (212). For example, upon completion of the patient assessment, one or more trained machine learning models 113a-113e, 114a-114e are preferably applied by the assessment system 104 to facial landmark data 118 corresponding to landmarks 117 identified in the obtained video data 112 to generate one or more motion severity scores.

[0072] In some examples, the assessment system 104 can separately assess movement severity scores for different categories of facial movements associated with TD symptoms. In such embodiments, the assessment system 104 can apply one or more trained machine learning models to the facial landmark data 118 to determine a first severity score corresponding to a first category of facial movements associated with TD, and apply one or more different trained machine learning models to the facial landmark data 118 to determine a second severity score corresponding to a second category of facial movements associated with TD. For example, unique categories of facial movements can include upper facial movements, tongue movements, mouth and lower facial movements, head shaking, and head tilt, although other unique categories can also be considered.

[0073] For example, upon determining that the prompt for conducting an assessment session is complete, the assessment system 104 may partition the video data 112 and / or corresponding facial landmark data 118 from a particular assessment session into a plurality of data subsets corresponding to video segments of a particular length that is less than the length of the entire assessment session. For example, the assessment system 104 may partition the video data 112 and / or corresponding facial landmark data 118 into subsets corresponding to video segments of 1-30 seconds, 1-20 seconds, or 1-10 seconds. In a particular embodiment, the assessment system 104 may partition the video data 112 and / or corresponding facial landmark data 118 into subsets corresponding to 4-second video segments. Thus, each data subset includes facial landmark data 118 associated with each landmark 117 of a particular segment of the assessment session. In one embodiment, the data subsets include sequential, non-overlapping segments of the video data 112. In another embodiment, the data subsets include sequential, overlapping segments of the video data 112. For example, in one embodiment, each sequential data subset corresponding to a 4-second segment of the assessment session may represent a 2-second overlap of the immediately preceding and subsequent 4-second segments.

[0074] The evaluation engine 120 can then apply a first trained machine learning model (e.g., such as one or more of the evaluation models 113a-113e) to the facial landmark data 118 comprising each data subset. Each of the first trained machine learning models 113a-113e is configured to determine from the facial landmark data 118 corresponding to each data subset whether a particular category of facial motion associated with TD (e.g., upper facial motion, tongue motion, mouth and lower facial motion, head shake, and head tilt) is present within the video segments of the data subset. This determination can be in the form of a motion indicator, such as a numerical indication. In one embodiment, the output of each of the first trained machine learning models 113a-113e can be a binary indication of whether a particular category of facial motion associated with TD is present within the video segments of the data subset. In another embodiment, the output of each of the first trained machine learning models 113a-113e can include a percentage probability of whether a particular category of facial motion associated with TD is present within the video segments of the data subset. In this way, the first trained machine learning models 113a-113e can generate a numerical indication of whether a particular category of facial motion associated with TD is present within the video segment associated with each data subset for the facial landmark data 118 of each data subset. For example, the first trained machine learning model 113a can generate a numerical indication of whether a blink is present for each data subset, the second trained machine learning model 113b can generate a numerical indication of whether a tongue motion is present for each data subset, and so on.

[0075] It should be understood that each of the first trained machine learning models 113a-113e can apply the same or different machine learning algorithms (e.g., any of the machine learning algorithms described herein). For example, the machine learning algorithm used for each of the first trained machine learning models 113a-113e can be selected based on which machine learning algorithm is most suitable for evaluating the particular category of facial movements that the trained machine learning models 113a-113e are configured to detect.

[0076] In one embodiment, all facial landmark data 118 for each of the data subsets can be provided to the evaluation models 113a-113e to determine whether facial motion associated with TD exists. Alternatively, each respective evaluation model 113a-113e can be provided with only the facial landmark data 118 associated with the corresponding category of facial motion. This selective feeding of data to the evaluation models 113a-113e can be done for statistical reasons, such as to avoid overfitting. For this reason, the evaluation system 104 can be pre-programmed with a correlation between each landmark 117 and the specific category (or categories) of facial motion associated with them. Thus, the evaluation system 104 can apply the trained machine learning models 113a-113e to a reduced amount of data, namely, the facial landmark data 118 associated with the facial motion category corresponding to the particular algorithm 113a-113e. For example, for a trained model 113a that analyzes eye blinks, the evaluation system 104 can filter out facial landmark data 118 corresponding to landmarks 117 that are not associated with eye blinks and provide only facial landmark data 118 corresponding to landmarks 117 associated with eye blinks to the trained model 113a. In other cases, the evaluation system 104 can apply the trained machine learning model to all facial landmark data 118 including landmarks identified in the obtained video data.

[0077] After the assessment engine has applied the trained models 113a-113e to the facial landmark data 118, the assessment engine 120 can apply the assessment models 114a-114e to the motion indicators (e.g., numerical indications) corresponding to the respective data subsets generated by the assessment models 113a-113e to generate motion severity data 122. Specifically, the motion severity data includes a motion severity score, i.e., a score corresponding to the severity of the patient's facial motion. In some cases, the assessment system 104, and preferably the assessment engine 120, can apply a first trained machine learning model 113a-113e to the facial landmark data 118 of the data subsets as described above to generate numerical indications of facial motion categories within the particular data subsets, and apply a corresponding second trained machine learning model 114a-114e to each of the numerical indications generated by the first trained machine learning model 113a-113e to generate motion severity data 122 for each of one or more categories of facial motion based on the entire assessment session for the particular patient. Thus, in its analysis of facial landmark data 118 for each category of facial movement for a particular patient, the evaluation engine 120 can utilize two trained machine learning models that are unique for each category of facial movement (e.g., models 113a, 114a for analyzing blinks, models 113b, 114b for analyzing head tilt, etc.).

[0078] The motion severity data 122 (i.e., a motion severity score) for each category of facial movement can indicate the severity of the movement. The motion severity score can be scored on a scale that indicates the severity of the involuntary movement of the patient's body. In one example, the scale corresponds to the 0-4 scale used in the AIMS test. For example, a motion severity score of zero can indicate no involuntary movement. In another instance, a motion severity score of 1 can indicate minimal or normal involuntary movement. In another instance, a motion severity score of 2 can indicate mild involuntary movement. In another instance, a motion severity score of 3 can indicate moderate involuntary movement. In another instance, a motion severity score of four can indicate severe involuntary movement. However, it is contemplated that other scales (0-1, 0-10, 0-100, etc.) can be utilized.

[0079] After each of the second trained machine learning models 114a-114e has generated a corresponding motion severity score for the corresponding category of facial movement, the assessment system 104 can calculate a single total severity score based on each of the motion severity scores. The total severity score can be a composite score that includes each of the motion severity scores and represents the overall severity of the involuntary facial movement for a particular patient. For example, the total severity score can be determined on a finite scale to provide a normalized measure for comparing the severity of involuntary facial movements between patients or for a particular patient at different time points. In one embodiment, the total severity score can represent an average of the motion severity scores generated by the second trained machine learning models 114a-114e. It is also contemplated that the total severity score can include a weighted average and / or other calculation based on the individual motion severity scores. In particular embodiments, the total severity score can correspond to an AIMS score.

[0080] In one or more instances, a notification (214) is preferably generated by the assessment system 104 indicating one or more of the motor severity score and / or the total severity score. In one or more other instances, the assessment system 104 may provide a notification indicating the total severity score to a clinician or prescribing physician for the psychiatric medication. In one or more instances, the assessment system 104 may mark the motor severity score and / or the total severity score as indicating evidence of TD. For example, if the assessment system 104 determines that the total severity score exceeds a threshold, the assessment system 104 determines that there is evidence of TD. In another example, if the assessment system 104 determines that the total severity score exceeds a threshold, the assessment system 104 provides a recommendation directing the patient to seek help from a trained clinician.

[0081] The assessment system 104 can store the motion severity score in the database 110. In addition, the assessment system 104 can store the motion severity score as anonymized data in the database 110. One or more subsequent motion severity scores for the patient, as provided by a clinician, can also be provided to and stored within the database 110, so that subsequent motion severity scores and video data for the patient and the clinician-provided AIMS score can be included within the training data for further training the machine learning model, wherein an exemplary process for training the machine learning model is described below.

[0082] The assessment system 104 can be configured to determine the progression of movement disorders over time. Upon diagnosis of TD, a clinician can prescribe a chemical or biological medication for the treatment of TD. For example, a clinician can prescribe a treatment including deutetrabenazine, valbenazine, tetrabenazine, clonazepam, and / or botulinum toxin.

[0083] Further provided herein are methods that can be used to treat TD. In some examples, the assessment system 104 can be configured to determine the TD total severity score as described above to inform the creation, maintenance, and / or revision of a patient's medical treatment plan, such as administering to a patient in need of a pharmaceutical composition comprising a pharmacologically active agent described herein in an effective amount to treat TD. As used herein, the terms "treatment method" or "therapy" (and their various forms) related to tardive dyskinesia include preventive (e.g., disease prevention), curative, or palliative treatments. As used herein, the term "treatment" includes alleviating or reducing at least one adverse or negative effect or symptom of tardive dyskinesia. The term "administering" means providing a pharmaceutical composition or dosage form (used interchangeably herein) to a patient. As used herein, the terms "compound," "drug," "pharmacologically active agent," "active agent," or "medicament" are used interchangeably herein and refer to one or more compounds or compositions of matter that induce a desired pharmacological and / or physiological effect by local and / or systemic action when administered to a subject (human or animal). Possible pharmacologically active agents may be selected from tetrabenazine, deutetrabenazine, valbenazine, deuterovalbenazine, clonazepam and / or botulinum toxin. A preferred active agent disclosed herein is deutetrabenazine. "Deutetrabenazine" or "deu-TBZ" is a selectively deuterated, stable, non-radioactive isotope form of tetrabenazine in which six hydrogen atoms on two O-linked methyl groups have been replaced with deuterium atoms (i.e., -OCD3 instead of -OCH3 moieties).

[0084] Vesicular monoamine transporter 2 (VMAT2) inhibitors can be used to treat uncontrolled involuntary movements associated with movement disorders such as TD. VMAT2 inhibitors can include, for example, deutetrabenazine (e.g., or XR), tetrabenazine and / or valbenazine. In some examples, the total daily dose of the VMAT2 inhibitor can be administered once a day (qd) or twice a day (bid). In some examples, the total daily dose of the VMAT2 inhibitor is 6 mg, or 12 mg, or 18 mg, or 24 mg, or 30 mg, or 36 mg, or 42 mg, or 48 mg of the VMAT2 inhibitor. The daily dose of the drug may require regular reassessment from a clinician, for example, to update the patient's titration schedule. For example, a patient can receive a package of VMAT2 inhibitor pills with different daily doses, also referred to as a titration kit. For example, VMAT2 inhibitor pills can be obtained with pills of different strengths, such as 6 mg pills, 12 mg pills, and / or 24 mg pills. A clinician can prescribe an initial daily dose of the VMAT2 inhibitor to the patient. The daily dose can be based on the results of the AIMS test, for example, in combination with other factors associated with the patient. In an embodiment, the daily dose can be based at least in part on the total severity score determined by the assessment system 104 as described above. The titration kit may include a supply of VMAT2 inhibitor pills for a predetermined time period (e.g., four weeks). During the entire titration kit, the daily amount of the VMAT2 inhibitor may be gradually increased. For example, the daily amount of the VMAT2 inhibitor may be increased in a systematic, stepwise manner (e.g., 6 mg / day in week 1, 18 mg / day in week 2, 24 mg / day in week 3, 30 mg / day in week 4, etc.).

[0085] Accurate diagnosis and assessment of the patient's TD may require regular re-examination and assessment of the patient's condition (e.g., by using the AIMS test). In one embodiment, the assessment system 104 (e.g., the motion severity score and / or total severity score generated by the assessment system 104) can be used to mark the patient to further assess potential TD diagnosis through training conditions. The patient's clinician can determine the patient's initial daily amount and / or increase the patient's daily amount after receiving and analyzing the latest regular assessment and / or based on a predetermined schedule. The assessment system 104 (e.g., the motion severity score and / or total severity score generated by the assessment system 104) can be used to determine the patient's titration process and / or determine the daily amount (e.g., initial daily amount) that the patient takes during any specific treatment interval. Therefore, the assessment system 104 can be configured to adjust the patient's dosage schedule and / or dosage amount based on the motion severity score generated by the assessment system 104. When using the assessment system 104 as an auxiliary for dosage selection and titration, an objective tool for standardization and acceleration of the TD assessment process is provided for both clinicians and patients. Examples of titration regimens for treating TD are described in PCT Publication No. WO 2016 / 0144901 and U.S. Patent Publication No. US 2016 / 0287574, the entire contents of which are incorporated herein by reference.

[0086] In one or more instances, in conjunction with a regimen for treating TD, the patient may utilize the assessment system 104 to perform multiple assessments (e.g., a second assessment) during a subsequent time period to determine the progression of the movement disorder over time. Figure 2 The second assessment may be performed in the same or similar manner as described in the example process 200 of . Accordingly, the description of these features will not be repeated. As a result of the second assessment, the assessment system 104 may generate a second total severity score. In one or more instances, additional movement severity scores may be obtained as described herein. In one or more instances, the assessment system 104 may compare the second total severity score to the previous total severity score to determine the status of the patient's movement disorder. For example, for instances where the assessment system 104 determines that the second total severity score is less than the previous total severity score, the assessment system 104 may determine that the severity of the movement disorder is regressing. In another instance, for instances where the assessment system 104 determines that the second total severity score is greater than the previous total severity score, the assessment system 104 may determine that the severity of the movement disorder is progressing. In one or more instances, the assessment system 104 may provide a notification to the clinician and / or patient via the device 106 and / or the device 108 suggesting that the currently prescribed treatment regimen be reassessed or adjusted.

[0087] The assessment system 104 can be configured to determine the progression of the severity of the movement disorder and can generate and provide notifications corresponding to the progression of the severity of the movement disorder, for example, in conjunction with a treatment regimen for TD and a subsequent assessment that generates a corresponding total severity score. For example, to initiate treatment for TD, based on the initial total severity score, the patient is administered a total daily dose of 6 mg of the pharmacologically active agent (e.g., deutetrabenazine). At a subsequent time (e.g., one week from the time the patient performs the first assessment and the assessment system 104 generates the initial total severity score), the patient can be administered a total daily dose of 6 mg of the pharmacologically active agent (e.g., deutetrabenazine) in accordance with the treatment regimen for TD. Figure 2 The second assessment is performed and a second total severity score is generated in the same or similar manner as described in the example process 200 of . In one or more cases, additional movement severity scores can be obtained as described herein. In one or more cases, the assessment system 104 or the clinician can compare the second total severity score with the previous total severity score to determine the state of the patient's movement disorder (i.e., the progression of the severity of the movement disorder), determine whether the current treatment is effective or ineffective based on the state of the movement disorder, and / or determine whether the patient has failed to follow the treatment plan. For example, the assessment system 104 and / or the clinician can determine that the second total severity score is greater than the previous total severity score (e.g., the initial total severity score), and therefore the severity of the movement disorder is developing. In addition, the assessment system 104 can provide a notification indicating the possibility that the current treatment is no longer effective based on determining that the severity of the movement disorder is developing (e.g., via displaying a notification on an interface of a device such as device 108). The assessment system 104 can provide the state of the movement disorder so that the clinician can determine whether the treatment plan for TD should be changed. The patient can continue to perform the assessment at regular intervals (e.g., but not limited to weekly intervals) so that the assessment system 104 can generate additional total severity scores and determine the status of the movement disorder. By providing an automated and objective method for repeatedly assessing TD severity and progression, patient compliance can be improved by increasing the efficiency of TD severity assessment and a discrete measure for observing TD improvement during a treatment regimen.

[0088] Machine learning models, such as evaluation models 113a-113e, 114a-114e, can be trained using training data comprising video data from a plurality of patients and a total severity score associated with each of the plurality of patients. In some examples, a plurality of trained machine learning models are trained using training data comprising facial landmark data extracted from videos of a plurality of patients with a positive TD diagnosis, an Abnormal Involuntary Movement Scale (AIMS) score associated with each of the plurality of patients, and / or a label corresponding to each video indicating the presence of a plurality of different categories of facial movements.

[0089] The video data (e.g., video data for each patient) can be divided into a plurality of data subsets corresponding to video segments of a particular length that is less than the length of the overall assessment session. For example, the video data can be divided into subsets corresponding to video segments of 1-30 seconds, 1-20 seconds, or 1-10 seconds. In a particular embodiment, the video data can be divided into subsets corresponding to 4-second video segments. Thus, each data subset can include facial landmark data associated with each landmark. In one embodiment, the data subsets include sequential, non-overlapping segments of the video data. In another embodiment, the data subsets include overlapping segments of the video data. For example, in one embodiment, each sequential data subset corresponding to a 4-second segment of an assessment session can represent a 2-second overlap of the immediately preceding and subsequent 4-second segments.

[0090] The training data may include only a subset of data in which movement of relevant facial landmark data was detected (e.g., by a physician, a technician, and / or using a facial recognition tool). For example, the training data may include video clips (e.g., only video clips) in which specific categories of facial motion associated with TD (e.g., upper facial motion, tongue motion, mouth and lower facial motion, head shaking, and head tilt) are present within the video clips. Furthermore, in some examples, the training data may include (e.g., only include) facial landmark data associated with the corresponding category of facial motion from each video clip. Data may be selectively fed into the training data for statistical reasons, such as to avoid overfitting. The training data may include correlations between each landmark and the specific category (or categories) of facial motion associated with it. Furthermore, in some examples, the training data may include a reduced amount of data, i.e., facial landmark data associated with the category of facial motion. For example, training data used to analyze eye blinks may not include (e.g., have been filtered out) facial landmark data corresponding to landmarks not associated with eye blinks, and only facial landmark data corresponding to landmarks associated with eye blinks may be provided to the patient.

[0091] The total severity score and / or motion severity score of each category of motion can be assigned to training data. In some examples, the motion severity score of training data can include a 5-point anchor score corresponding to the specific project of AIMS. In other examples, the motion severity score can use different scales (e.g., one to ten, AZ, etc.). The motion severity score of training data can be assessed on a scale identical or different to the motion severity score determined by the assessment model 113a-113e, 114a-114e trained by the assessment system 104. The motion severity score and / or total risk score of training data may have been manually performed by a physician (e.g., based on the patient video data recorded in response to one or more prompts). Like this, training data can include the corresponding motion severity score and / or total severity score (e.g., AIMS score) of each patient determined by the trained clinician from multiple patients and by the trained clinician, including each component rating / answer of each score. Specifically, training data can include landmarks (e.g., face, hands, and feet) identified in each patient's video data, and the motion severity score, for example, associated with the landmarks. Furthermore, in some examples, the training data may be associated with one or more labels characterizing the video, such as the number of blinks performed by the patient during the video.

[0092] The machine learning model may include any combination of algorithms. For example, the machine learning model may include a gradient boost decision tree, a random forest algorithm, a logarithmic regression model, and / or an XGBoost algorithm. In an example, a gradient boost decision tree may combine weak learners to minimize the loss function. For example, a regression tree may be used to generate splits and / or real values ​​to be added together. Weak learners may be constrained to, for example, a maximum number of layers, a maximum number of nodes, a maximum number of splits, etc. Trees may be added to the machine learning algorithm one at a time and / or existing trees may remain unchanged. When adding a tree, a gradient descent process may be used to minimize the loss. For example, additional trees may be added to reduce the loss (e.g., following the gradient). In an example, additional trees may be given parameters, and those parameters may be modified to reduce the loss. In an example, the XGBoost algorithm may include an implementation of a gradient boost decision tree that may be designed for speed and / or performance.

[0093] The XGBoost algorithm can (e.g., automatically) handle missing data values, support parallelization and / or continuous training of tree construction. Gradient boosting tree technology can produce a prediction model in the form of a set (e.g., multiple learning algorithms) of basic prediction models, which is a decision tree (e.g., a tree-like model of a decision and its possible consequences). The XGBoost algorithm can iteratively build a single strong learner model by using an optimization algorithm to minimize some suitable loss functions (e.g., a function of the difference between the estimated value and the true value of a data instance). The optimization algorithm can use a training set of known values ​​of a response variable (e.g., AIMS score) and the corresponding values ​​of its predictors (e.g., training data patient landmarks) to minimize the expected value of the loss function. The learning process can continuously fit new models to provide a more accurate estimate of the response variable.

[0094] Although primarily described in the context of the XGBoost algorithm, other machine learning models can be used, such as unsupervised learning methods (e.g., clustering methods such as k-means or c-means clustering methods) and / or supervised learning methods (e.g., gradient boosting decision trees). As an example, the evaluation system 104 can use a gradient descent or stochastic gradient descent learning method.

[0095] Supervised learning methods can use labeled training data to train a machine learning algorithm. As the training data is received, supervised learning methods can adjust weights until the machine learning algorithm is appropriately weighted. Supervised learning methods can use a loss function to measure the accuracy of the machine learning algorithm. Supervised learning methods can continue adjusting weights until the error decreases below a predetermined threshold.

[0096] Figure 4 is a block diagram illustrating an example computing device 400. One or more computing devices, such as computing device 400, may implement one or more features for assessing a patient's risk of TD, as described herein. For example, computing device 400 may be Figure 1A 106, device 108, and / or server device(s) 102 are shown as examples. Computing device 400 may include a processor 402, a memory 404, a storage device 406, an I / O interface 408, and / or a communication interface 410, which may be communicatively coupled via a communication infrastructure 412. It should be understood that computing device 400 may include more than Figure 4 For example, a computing device may include a camera (eg, when implemented as device 108).

[0097] The processor 402 may include hardware for executing instructions, such as instructions constituting a computer program. In an example, to execute instructions for dynamically modifying a workflow, the processor 402 may retrieve (or fetch) instructions from an internal register, an internal cache, memory 404, or a storage device 406, and decode and execute the instructions.

[0098] The memory 404 may be a volatile or non-volatile memory for storing data, metadata, computer-readable or machine-readable instructions, and / or programs for execution by the processor to perform the operations described herein. The storage device 406 may include a storage device, such as a hard disk, a flash drive, or other digital storage device, for storing data or instructions for performing the methods described herein.

[0099] The memory 404 may include a computer-readable storage medium or a machine-readable storage medium that maintains one or more computer-executable instructions for performing as described herein. For example, the memory 404 may include computer-executable instructions or machine-readable instructions that include one or more portions of the processes described herein. The processor 402 of the device 400 may access instructions from the memory to be executed so that the processor 402 of the device 400 operates as described herein. The memory 404 may include computer-executable instructions for executing configuration software. For example, the computer-executable instructions may be executed to partially and / or fully perform one or more processes as described herein. In addition, the memory 404 may have stored thereon one or more settings and / or control parameters associated with the device 400.

[0100] The I / O interface 408 can allow a user to provide input to the computing device 400, receive output from the computing device 400, and / or otherwise transmit data to and receive data from the computing device 400. The I / O interface 408 may include a mouse, a keypad or keyboard, a touch screen, a camera, an optical scanner, a network interface, a modem, other known I / O devices, or a combination of such I / O interfaces. The I / O interface 408 may include one or more devices for presenting output to the user, including but not limited to a graphics engine, a display (e.g., a display screen), one or more output drivers (e.g., a display driver), one or more audio speakers, and one or more audio drivers. The I / O interface 408 can be configured to provide graphical data to the display for presentation to the user. The graphical data can represent one or more graphical user interfaces and / or any other graphical content (e.g., any combination of the prompts described herein).

[0101] Communication interface 410 may include hardware, software, or both. In any case, communication interface 410 may provide one or more interfaces for communication (e.g., packet-based communication) between computing device 400 and one or more other computing devices or networks. Communication may be wired or wireless. By way of example and not limitation, communication interface 410 may include a network interface controller (NIC) or network adapter for communicating with an Ethernet or other wired network, or a wireless NIC (WNIC) or wireless adapter for communicating with a wireless network such as Wi-Fi.

[0102] In addition, the communication interface 410 can facilitate communication with various types of wired or wireless networks. The communication interface 410 can also facilitate communication using various communication protocols. The communication infrastructure 412 can also include hardware, software, or both that couple the components of the computing device 400 to each other. For example, the communication interface 410 can use one or more networks and / or protocols to enable multiple computing devices connected by a particular infrastructure to communicate with each other to perform one or more aspects of the processes described herein.

[0103] In addition to what has been described herein, the methods and systems may also be implemented in a computer program, software, or firmware incorporated into one or more computer-readable media for execution, for example, by a computer or processor. Examples of computer-readable media include electronic signals (transmitted via a wired or wireless connection) and tangible / non-transitory computer-readable storage media. Examples of tangible / non-transitory computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), removable disks, and optical media such as CD-ROM disks and digital versatile disks (DVDs).

[0104] Although the present disclosure has been described in terms of certain embodiments and generally associated methods, variations and substitutions of the embodiments and methods will be apparent to those skilled in the art. Therefore, the above description of the exemplary embodiments does not constrain the present disclosure. Other variations, substitutions, and modifications are also possible without departing from the spirit and scope of the present disclosure.

Claims

1. A method for assessing the severity of involuntary movements associated with tardive dyskinesia (TD), the method comprising: receiving video data of a patient, wherein the video data corresponds to facial movements of the patient; processing the video data to identify facial landmark data of the patient; applying a plurality of trained machine learning models to the facial landmark data to determine a plurality of motion severity scores based on the facial landmark data of the patient, wherein each of the plurality of motion severity scores represents a respective category of facial motion; and The plurality of motion severity scores are processed to generate an overall severity score.

2. The method according to claim 1, wherein A movement severity score of the plurality of movement severity scores indicates a severity of involuntary movement associated with a corresponding category of facial movement.

3. The method according to claim 1, wherein The categories of facial movements include one or more of upper facial movements, tongue movements, mouth and lower facial movements, head shaking, and head tilting.

4. The method according to claim 1, wherein Processing the video data includes applying image processing techniques to the video data to identify facial landmarks on one or more frames of video data to generate labeled frames.

5. The method according to claim 4, wherein The image processing technology includes the dlib face recognition method, the Haar cascade method, the Fisherface method or the elastic graph matching method.

6. The method according to claim 5, wherein: Processing the video data further includes identifying the facial landmark data corresponding to each of the facial landmarks from the tagged frame.

7. The method according to claim 6, wherein: The facial landmark data includes one or more of a landmark identifier, a landmark coordinate location, a frame source identifier, a video source identifier, or a patient identifier.

8. The method according to claim 6, wherein: Processing the video data further includes segmenting the video data into a plurality of data subsets comprising video segments of predefined duration.

9. The method according to claim 8, wherein The predefined duration is four seconds.

10. The method according to claim 8, wherein Each of the data subsets includes a video segment that overlaps with at least one other video segment.

11. The method according to claim 1, wherein Processing the video data includes segmenting the video data into a plurality of data subsets comprising video segments of predefined duration; as well as Wherein, applying the multiple trained machine learning models further includes: applying a first trained machine learning model to at least a portion of the facial landmark data for each of the data subsets to generate a motion indicator for each of the data subsets; and A second trained machine learning model is applied to the motion indicators for each of the data subsets to generate a respective motion severity score for each respective category of facial motion.

12. The method according to claim 11, wherein The motion indicator for each of the data subsets is a binary indicator indicating whether the corresponding category of facial motion is present within the data subset.

13. The method according to claim 12, wherein: The first trained machine learning model is applied to a portion of the facial landmark data predetermined to be associated with a respective category of facial motion for each of the data subsets.

14. The method according to claim 1, wherein The plurality of motion severity scores include a first motion severity score associated with a first category of facial motion and a second motion severity score associated with a second category of facial motion different from the first category of facial motion, wherein the total severity score is based on the first motion severity score and the second motion severity score.

15. The method according to claim 1, wherein The total severity score corresponds to the Abnormal Involuntary Movement Scale (AIMS) score.

16. The method according to claim 1, further comprising: Prompts are provided to the patient to perform an action.

17. The method according to claim 16, further comprising: The prompt is provided to the patient by displaying instructions for performing the action on a user interface of the electronic device.

18. The method of claim 1, further comprising: One or more videos of the patient are captured, wherein the video data of the patient includes the one or more videos.

19. The method according to claim 1, wherein The total severity score corresponds to the risk of receiving a positive TD diagnosis by a trained physician.

20. The method according to claim 19, further comprising: A notification is generated, wherein the notification includes a recommendation to direct the patient to a trained clinician when the total severity score is above a predetermined threshold.

21. The method according to claim 1, wherein The plurality of trained machine learning models are trained using training data comprising facial landmark data extracted from videos from a plurality of patients with a positive TD diagnosis, an Abnormal Involuntary Movement Scale (AIMS) score associated with each of the plurality of patients, and a label corresponding to each of the videos indicating the presence of a plurality of different categories of facial movements.

22. The method according to claim 21, wherein The categories of facial movements include upper facial movements, tongue movements, mouth and lower facial movements, head shaking, and head tilting.

23. The method of claim 1, further comprising generating a notification indicating the total severity score.

24. The method of claim 23, further comprising displaying the notification on a user interface of the electronic device.

25. The method of claim 1, further comprising: After receiving the video data, administering to the subject a first daily amount of a medication associated with TD; as well as Based on the patient's total severity score being above a threshold, the daily amount of the medication is increased to a first subsequent daily amount.

26. The method according to claim 25, wherein Such drugs include vesicular monoamine transporter 2 (VMAT2) inhibitors.

27. The method according to claim 26, wherein The VMAT2 inhibitors include deutetrabenazine, tetrabenazine or valbenazine.

28. The method according to claim 26, wherein The first daily amount is in accordance with a titration schedule associated with the VMAT2 inhibitor.

29. The method according to claim 25, wherein The first daily amount is at least about 6 mg / day, and the drug is deutetrabenazine, and the first subsequent daily amount is at least about 6 mg / day greater than the first daily amount.

30. The method of claim 25, further comprising: receiving second video data of the patient, wherein the second video data corresponds to facial movement of the patient occurring at a time after administration of the first subsequent daily amount of the medication; processing the second video data to identify second facial landmark data for the patient; and applying the plurality of trained machine learning models to the second facial landmark data to determine a plurality of second motion severity scores based on changes in the second facial landmark data of the patient; processing the plurality of second motion severity scores to generate a second overall severity score; and Based on the patient's second total severity score being above a second threshold, the daily amount of the medication is increased to a second subsequent daily amount.

31. The method of claim 1 , further comprising: After receiving the video data, administering to the subject a first daily amount of a medication associated with TD; receiving second video data of the patient, wherein the second video data corresponds to facial movement of the patient occurring at a time after the patient's medication regimen was adjusted; processing the second video data to identify second facial landmark data of the patient; applying the plurality of trained machine learning models to the second facial landmark data to determine a plurality of second motion severity scores based on changes in the second facial landmark data of the patient; and The plurality of second motion severity scores are processed to generate a second overall severity score.

32. The method of claim 1, further comprising: determining that the total severity score is above a threshold; as well as A recommendation is generated to adjust a dosing regimen of a medication for the patient based on the total severity score of the patient being above the threshold.

33. A method for titrating a medication in a patient associated with tardive dyskinesia (TD), the method comprising: administering to the subject a daily amount of a vesicular monoamine transporter 2 (VMAT2) inhibitor; receiving video data of the patient, wherein the video data corresponds to facial movements of the patient; processing the video data to identify facial landmark data of the patient; applying a plurality of trained machine learning models to the facial landmark data to determine a motion severity score based on the facial landmark data of the patient; and Based at least in part on the patient's motor severity score being above a threshold, increasing the daily amount of the VMAT2 inhibitor.

34. The method according to claim 33, wherein The VMAT2 inhibitors include deutetrabenazine, tetrabenazine or valbenazine.

35. The method of claim 33, wherein: Applying the plurality of trained machine learning models includes: applying the plurality of trained machine learning models to the facial landmark data to determine a plurality of movement severity scores based on changes in the facial landmark data of the patient, wherein each of the plurality of movement severity scores represents a respective category of facial movement corresponding to a TD symptom; and processing the plurality of motion severity scores to generate an overall severity score; wherein the daily amount of the VMAT2 inhibitor is increased based on the total severity score.

36. The method according to claim 35, wherein Processing the video data includes segmenting the video data into a plurality of data subsets comprising video segments of predefined duration; as well as Wherein, applying the multiple trained machine learning models further includes: applying a first trained machine learning model to at least a portion of the facial landmark data for each of the data subsets to generate a motion indicator for each of the data subsets; and A second trained machine learning model is applied to the motion indicators for each of the data subsets to generate a respective motion severity score for each respective category of facial motion.

37. A method for assessing the severity of involuntary movements associated with tardive dyskinesia (TD), the method comprising: receiving video data of the patient, wherein the video data corresponds to facial movements of the patient; processing the video data to identify facial landmark data of the patient; and applying a plurality of trained machine learning models to the facial landmark data to determine a motion severity score based on changes in the facial landmark data for the patient, The plurality of trained machine learning models are trained using training data comprising facial landmark data extracted from videos of a plurality of patients with a positive TD diagnosis, an Abnormal Involuntary Movement Scale (AIMS) score associated with each of the plurality of patients, and labels corresponding to each of the videos indicating the presence of a plurality of different categories of facial movements.

38. The method of claim 37, further comprising: segmenting the video data into a plurality of data subsets comprising video segments of predefined duration; as well as The multiple trained machine learning models include: a first trained machine learning model trained based on at least a portion of the facial landmark data for each of the data subsets to generate a motion indicator for each of the data subsets; and A second trained machine learning model is trained based on the motion indicators for each of the data subsets to generate a respective motion severity score for each respective category of facial motion.

39. The method according to claim 38, wherein Each of the data subsets includes a video segment that overlaps with at least one other video segment.

40. The method of claim 37, wherein The facial landmark data includes one or more of a landmark identifier, a landmark coordinate location, a frame source identifier, a video source identifier, or a patient identifier.

41. A system for assessing the severity of involuntary movements associated with tardive dyskinesia (TD), the system comprising: One or more processors and memory, wherein the one or more processors and memory are configured to: receiving video data of a patient, wherein the video data corresponds to facial movements of the patient; processing the video data to identify facial landmark data of the patient; applying a plurality of trained machine learning models to the facial landmark data to determine a plurality of motion severity scores based on the facial landmark data of the patient, wherein each of the plurality of motion severity scores represents a respective category of facial motion; and The plurality of motion severity scores are processed to generate an overall severity score.

42. The system of claim 41, wherein: A movement severity score of the plurality of movement severity scores indicates a severity of involuntary movement associated with a corresponding category of facial movement.

43. The system of claim 41, wherein: The categories of facial movements include one or more of upper facial movements, tongue movements, mouth and lower facial movements, head shaking, and head tilting.

44. The system of claim 41, wherein: To process the video data, the one or more processors and memory are configured to apply image processing techniques to the video data to identify facial landmarks on one or more frames of video data to generate labeled frames.

45. The system of claim 44, wherein: The image processing technology includes the dlib face recognition method, the Haar cascade method, the Fisherface method or the elastic graph matching method.

46. ​​The system of claim 45, wherein: To process the video data, the one or more processors and memory are configured to identify the facial landmark data corresponding to each of the facial landmarks from the tagged frame.

47. The system of claim 46, wherein: The facial landmark data includes one or more of a landmark identifier, a landmark coordinate location, a frame source identifier, a video source identifier, or a patient identifier.

48. The system of claim 46, wherein: To process the video data, the one or more processors and memory are configured to segment the video data into a plurality of data subsets comprising video segments of predefined duration.

49. The system of claim 48, wherein The predefined duration is four seconds.

50. The system of claim 48, wherein Each of the data subsets includes a video segment that overlaps with at least one other video segment.

51. The system of claim 41, wherein: To process the video data, the one or more processors and memory are configured to segment the video data into a plurality of data subsets comprising video segments of predefined duration; as well as In order to apply the plurality of trained machine learning models, the one or more processors and memory are configured to: applying a first trained machine learning model to at least a portion of the facial landmark data for each of the data subsets to generate a motion indicator for each of the data subsets; as well as A second trained machine learning model is applied to the motion indicators for each of the data subsets to generate a respective motion severity score for each respective category of facial motion.

52. The system of claim 51, wherein: The motion indicator for each of the data subsets is a binary indicator indicating whether the corresponding category of facial motion is present within the data subset.

53. The system of claim 52, wherein: The first trained machine learning model is applied to a portion of the facial landmark data predetermined to be associated with a respective category of facial motion for each of the data subsets.

54. The system of claim 41, wherein: The plurality of motion severity scores include a first motion severity score associated with a first category of facial motion and a second motion severity score associated with a second category of facial motion different from the first category of facial motion, wherein the total severity score is based on the first motion severity score and the second motion severity score.

55. The system of claim 41, wherein The total severity score corresponds to the Abnormal Involuntary Movement Scale (AIMS) score.

56. The method of claim 41, wherein The one or more processors and memory are configured to: Prompts are provided to the patient to perform an action.

57. The system of claim 56, wherein: The one or more processors and memory are configured to: The prompt is provided to the patient by displaying instructions for performing the action on a user interface of the electronic device.

58. The system of claim 41, wherein: The one or more processors and memory are configured to: One or more videos of the patient are captured, wherein the video data of the patient includes the one or more videos.

59. The system of claim 41, wherein: The total severity score corresponds to the risk of receiving a positive TD diagnosis by a trained physician.

60. The system of claim 59, wherein: The one or more processors and memory are configured to: A notification is generated, wherein the notification includes a recommendation to direct the patient to a trained clinician when the total severity score is above a predetermined threshold.

61. The system of claim 41, wherein: The plurality of trained machine learning models are trained using training data comprising facial landmark data extracted from videos from a plurality of patients with a positive TD diagnosis, an Abnormal Involuntary Movement Scale (AIMS) score associated with each of the plurality of patients, and a label corresponding to each of the videos indicating the presence of a plurality of different categories of facial movements.

62. The system of claim 61, wherein: The categories of facial movements include upper facial movements, tongue movements, mouth and lower facial movements, head shaking, and head tilting.

63. The system of claim 41, wherein: The one or more processors and memory are configured to generate a notification indicating the total severity score.

64. The system of claim 63, wherein: The one or more processors and memory are configured to display the notification on a user interface of the electronic device.

65. The method of claim 41, wherein The one or more processors and memory are configured to: determining that the total severity score is above a threshold; and A recommendation is generated to adjust a dosing regimen of a medication for the patient based on the total severity score of the patient being above the threshold.

66. A method for assessing the severity of involuntary movements associated with tardive dyskinesia (TD), the method comprising: receiving video data of a patient, wherein the video data corresponds to facial movements of the patient; processing the video data to identify facial landmark data of the patient; and A plurality of trained machine learning models are applied to the facial landmark data to determine a motion severity score based on changes in the facial landmark data of the patient.

Images