Parkinson's Disease Detection Based on User Devices

Abstract

A method and a user device for determining a Unified Parkinson's Disease Rating Scale (UPDRS) value associated with a user of the user device, including obtaining video data associated with the movement of a user's body part. Using a model and the video data associated with the movement of the user's body part to determine the UPDRS value. Providing the UPDRS value to allow for the evaluation of the user based on the UPDRS value.

Description

[0001] Cross - Reference to Related Applications

[0002] This application claims priority to U.S. Application No. 16 / 554,006, filed with the United States Patent and Trademark Office on August 28, 2019, the entire disclosure of which is incorporated herein by reference in its entirety. Background Art

[0003] Parkinson's disease is a long - term, progressive central nervous system disease that mainly affects the motor system. Movement - related symptoms gradually begin, usually with almost imperceptible tremors in one hand, which gradually worsen over time. In the early stages of Parkinson's disease, movement - related symptoms such as tremors, shakes, bradykinesia, and stiffness are very common. In the later stages, as the disease progresses, non - motor symptoms may appear. Patients may experience difficulty walking, as well as symptoms related to thinking and behavior, such as sensory difficulties, sleep disorders, and mood problems. In short, movement problems are the main symptoms of Parkinson's disease. The assessment of disease stage and symptoms has always been the main focus in clinical practice. Movement problems can be reflected in other forms of movement disorders, such as the stiffness and lag in the movement of a patient's limbs, which can be determined by analyzing a video capturing the patient's movement. Summary of the Invention

[0004] According to one aspect of the present disclosure, a method for determining a Unified Parkinson's Disease Rating Scale (UPDRS) value associated with a user of a user device, performed by the user device, includes: obtaining, by a processor of the user device, video data associated with the movement of a user's body part; after obtaining the video data associated with the movement of the user's body part, using, by the processor of the user device, a model to determine a UPDRS value based on the video data associated with the movement of the user's body part; and after determining the UPDRS value, providing, by the processor of the user device, the UPDRS value to allow the user to be evaluated based on the UPDRS value.

[0005] According to one aspect of the present disclosure, a user device includes at least one memory configured to store program code; and at least one processor configured to read the program code and operate in accordance with the instructions of the program code, the program code including: an obtaining code configured to cause the at least one processor to obtain video data associated with the movement of a user's body part; a determining code configured to cause the at least one processor, after obtaining the video data associated with the movement of the body part, to determine a UPDRS value based on the video data associated with the movement of the user's body part using a model; and a providing code configured to cause the at least one processor, after determining the UPDRS value, to provide the UPDRS value to allow the user to be evaluated based on the UPDRS value.

[0006] According to some possible implementations, a non-transitory computer-readable medium stores instructions that include one or more instructions which, when executed by one or more processors of a user device for determining a Unified Parkinson's Disease Rating Scale (UPDRS) value associated with a user, cause the one or more processors to: obtain video data associated with the movement of a body part of the user; after obtaining the video data associated with the movement of the body part of the user, use a model to determine the UPDRS value based on the video data associated with the movement of the body part of the user; and after determining the UPDRS value, provide the UPDRS value to allow for the evaluation of the user based on the UPDRS value. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 is an overview diagram of an example implementation described herein;

[0008] Figure 2 is an example environment diagram in which the systems and / or methods described herein can be implemented;

[0009] Figure 3 is Figure 2 a diagram of example components of one or more devices in

[0010] Figure 4 is a flowchart of an example process for determining a Unified Parkinson's Disease Rating Scale (UPDRS) value associated with a user of a user device;

[0011] Figure 5 is a flowchart of an example process for performing image segmentation techniques. DETAILED DESCRIPTION

[0012] In 2015, 6.2 million people were affected by Parkinson's disease, which also led to 117,400 deaths globally. Parkinson's disease is common in people over 60 years old, and about 1% of the population is affected. The average life expectancy after diagnosis is 7 to 14 years. The cost of Parkinson's disease to society is high, and Parkinson's disease reduces the quality of life of patients and their caregivers. Therefore, the present disclosure utilizes the motion sensors of a user device to monitor the movement progression related to Parkinson's disease, thereby helping patients and their caregivers with an easily accessible and easy-to-use health monitoring tool.

[0013] Recent advances in sequence data analysis using deep neural network (DNN) methods have made significant progress. In addition, the popularity of powerful mobile devices has reached a point where most of the public can access applications based on artificial intelligence (AI). The present disclosure will attempt to initially screen for signs of movement-related Parkinson's disease symptoms by allowing the user device to be used as an auxiliary medical diagnostic tool. The present disclosure has the following benefits, among others: assessment of movement-related symptoms of Parkinson's disease; providing multiple categories of motor tests, such as grades 0-4 in the Unified Parkinson's Disease Rating Scale (UPDRS) clinical test; providing an easy-to-install mobile application with no additional hardware requirements for the user device.

[0014] The medical diagnosis auxiliary application disclosed in the present invention is used to regularly and quantitatively measure the disease state of Parkinson's disease patients. Motion sensors such as accelerometers, gyroscope sensors, etc. have been effectively embedded in most modern user devices, such as mobile phones, smart watches, etc.

[0015] Multidimensional motion data can measure the movement of the body attached to the user's device at a sampling rate of 10 to 50 Hz, which is sufficient to quantitatively measure the motor function of Parkinson's patients. In particular, in recent years, advances in DNN and long short-term memory networks (LSTM) have shown promising results in analyzing time series data, making it possible to conduct more accurate AI-based Parkinson's disease motion analysis.

[0016] The present disclosure addresses the following challenges: (1) coordination between misdiagnosis and false positives; and (2) the architecture of the front-end and back-end communicating through an AI-based data analytics processing pipeline.

[0017] The present disclosure solves technical barriers to delivering advanced artificial intelligence-driven technologies to reality and provides practical applications that are accessible and easy to use. The present disclosure separates AI services and client applications into their own independent and loosely coupled components. The separation of AI services and client sensor collection modules allows multiple components to be developed, tested, and deployed in parallel. In addition, with this configuration, the user device can save processing and hardware resources by avoiding the need to perform a large amount of calculations, where the computing power and capabilities of the user device may be limiting factors. Since the user device executing the client application is able to perform motion testing and data collection, the user is able to gain the flexibility to perform testing in a convenient location (e.g., a home care environment).

[0018] The present disclosure is directed to practical applications that allow accurate and real-time testing via user equipment.

[0019] The proposed disclosure provides a preliminary screening tool for the early detection of Parkinson's disease by assessing motor function. Additionally, the present disclosure allows for the periodic monitoring of the Parkinson's disease state for better patient care and management. According to a first scenario of the present disclosure, if needed, healthcare professionals are able to quantitatively and long - term assess Parkinson's disease using the UPDRS scoring system. According to another scenario of the present disclosure, users can periodically perform an assessment of the early signs of motor dysfunction.

[0020] The model is an end - to - end deep learning model using a one - way LSTM - based DNN that sequentially utilizes motion sensor time - series data.

[0021] The application architecture follows Model - View - ViewModel (MVVM), where the data model is different from the above - mentioned DNN model. The DNN model is used for inference during the application lifecycle and provides communication between the data model and the DNN model, both of which will be fed into the viewer model. The viewer model is configured to serve the user interface (UI) through data binding. The DNN model can be stored in the cloud and / or on a local mobile device.

[0022] Motion sensor data is recorded in real - time at an adjustable rate, such as 10hz - 50hz, and can be analyzed locally on the user device or remotely through a cloud service. For example, a score from 0 - 4 will be provided after assessing the sensor data. This score is related to the MDS - UPDRS, which is a Parkinson's motor rating scale for the assessment of the Parkinson's disease state.

[0023] According to an embodiment, the system collects sensor data and analyzes the sensor data to determine the severity of tremors or shakes corresponding to the UPDRS criteria.

[0024] According to one embodiment, the system captures video data of the actions of certain hand movements or the actions of movements of other body parts from a patient under instruction guidance and analyzes the pattern of the movement to determine the severity of the movement disorder of an individual movement corresponding to the UPDRS scale for a given movement.

[0025] According to an embodiment, the system captures sensor data from a patient under instruction guidance and captures video data of the actions of certain hand movements or the actions of movements of other body parts. The captured sensor data and / or video data are analyzed to determine the severity of the movement disorder of an individual movement corresponding to the UPDRS scale for a given movement. Additionally, the system can use all or selected movements to predict the overall UPDRS score to determine the stage of Parkinson's disease of the patient.

[0026] According to one embodiment, the system uses a model and a subset of movements analyzed from video or sensor data to predict the overall UPDRS score or the stage of Parkinson's disease.

[0027] According to one embodiment, the sensor data is recorded by the built-in gyroscope sensor and accelerometer of the user device under the guidance of predefined gestures and movements.

[0028] Quantitative and continuous monitoring of Parkinson's disease has always been a challenge for Parkinson's disease caregivers. This disclosure allows for patient management using easily accessible and easy-to-use applications, making health monitoring possible and convenient.

[0029] This disclosure provides insights for improved treatment plans and ultimately improves the quality of life of Parkinson's disease patients. This disclosure provides an AI-based mobile application that users can utilize to evaluate motor function for pre-screening of Parkinson's disease. The success of the DNN model can further expand the application to facilitate comprehensive diagnosis. The MVVM design of the application can be used to provide a better DNN model, ultimately making the mobile device a reliable and easy-to-use health monitoring tool capable of filling the emerging online diagnosis market.

[0030] Figure 1 is an overview diagram of the embodiments described herein. As Figure 1 shown, the user device 100 (e.g., a mobile phone) can determine a Unified Parkinson's Disease Rating Scale (UPDRS) value associated with the user of the user device 100. For example, the user device 100 can execute an application that allows the user device 100 to determine the UPDRS value associated with the user operating the user device 100.

[0031] The user can correspond to a person undergoing a Parkinson's disease test, being monitored or evaluated for Parkinson's disease, etc. In this case, the user device 100 can execute an application that allows the user to be evaluated by determining the user's UPDRS value.

[0032] As Figure 1 shown, the user device 100 (e.g., the processor of the user device 100) can acquire video data 100 associated with the movement of the user's body part. As described in more detail below, the video data can allow the user device to determine the UPDRS value. The user device 100 can use the camera of the user device 100 to acquire the video data.

[0033] First, in some cases, the user device 100 may provide a prompt via an output component (e.g., a display, a speaker, etc.) indicating that the user uses a body part to perform a predetermined gesture. For example, the prompt may indicate that the user moves their hand in a predefined manner. As another example, the prompt may indicate that the user holds their hand in a stationary position. As another example, the prompt may indicate that the user moves their arm with a specific action. It should be understood that the user's body part may include any body part of the user and / or any combination of the user's body parts. In addition, it should be understood that the predetermined gesture may include any type of gesture. Alternatively, the prompt may indicate that the user performs any specific action or pose. That is, the user may move the body part (or hold a stationary position) in any undefined manner.

[0034] The user device 100 may also prompt the user to record video data of the body part using the user device 100 while the user performs a movement of the body part. Thus, the user may operate the user device 100 to obtain video data of the user's body part while the user performs a movement of the body part.

[0035] Further as Figure 1 shown, the user device 100 (e.g., the processor of the user device 100) may determine whether the measurement condition 120 is met. For example, the measurement condition may correspond to a threshold amount of time that has elapsed since the start of obtaining video data, a threshold amount of video data being obtained, the movement of the body part being completed, the number of predetermined movements or gestures performed, a set of prompts completed, a threshold amount of video data obtained for accurately determining the UPDRS value, etc.

[0036] Further as Figure 1 shown, the user device 100 (e.g., the processor of the user device 100) may use a deep neural network (DNN) model to determine. For example, the user device 100 may input the video data and / or the processed video data into the DNN model and determine the UPDRS value 130 based on the output of the model. The UPDRS value 130 may include the Movement Disorder Society (MDS) UPDRS value and may include values such as 1, 2, 3, 4, etc.

[0037] In some implementations, the user device 100 may input the original video data into the model and determine the UPDRS value 130 based on the output of the model. Alternatively, the user device 100 may perform one or more video processing techniques, input the processed video data into the model, and determine the UPDRS value 130 based on the output of the model.

[0038] The user device 100 can receive a trained model from a cloud-based platform (e.g., a server) and store the model. In this way, the user device 100 can save processor and / or memory resources by reducing the need to perform a large amount of computations. The cloud-based platform can utilize machine learning techniques to analyze data and generate a model. For example, the cloud-based platform can analyze millions, billions, trillions, etc. of data points and generate a model that correlates video data and UPDRS values. In this way, the user device 100 can receive the model and utilize the model in association with an application that allows the determination of the UPDRS value 130.

[0039] Further as Figure 1 shown, the processor of the user device 100 can provide the UPDRS value 130 to allow the evaluation of the user based on the UPDRS value 130. For example, the user device 100 can provide information identifying the UPDRS value 130 via an output component (e.g., a user interface) to allow the user, doctor, caregiver, medical staff, etc. to identify the UPDRS value.

[0040] In other cases, the user device 100 can provide the UPDRS value 130 to a group of other devices in real time via a network. In some embodiments, the user device 100 can provide the UPDRS value 130 in a standardized format to allow the updating of various databases and records based on the UPDRS value.

[0041] The user device 100 can use standardized techniques to standardize the UPDRS value 130 such that each device in the group can use the UPDRS value 130.

[0042] The user device 100 can collect the user's medical information and convert and merge the medical information into a standardized format. In addition, the user device 100 can generate the UPDRS value 130 in association with the standardized format. The user device 100 can store the standardized UPDRS value 130 in a group of network-based storage devices (e.g., a cloud-based platform) and generate a message to notify healthcare providers, doctors, medical staff, and patients whenever the UPDRS is generated or updated.

[0043] In addition, the user device 100 can provide the UPDRS value 130 to the group of devices in real time (e.g., substantially simultaneously with the generation of the UPDRS value 130) to allow the group of devices to update and / or utilize the UPDRS value 130 in real time. In this way, each user of the group of devices can immediately access the latest UPDRS value 130 of that user.

[0044] In this way, compared with non-standardized medical information associated with different healthcare providers, some implementations herein allow standardized medical information and / or UPDRS values 130 to be generated and provided in real time to multiple different devices, thereby allowing different users to share medical information and / or UPDRS values 130.

[0045] In addition, in this way, some implementations herein allow complete and accurate medical information and / or UPDRS values 130 to be provided in real time. Compared with the situation where multiple different medical staff have incomplete or inaccurate medical or diagnostic information, some implementations herein allow complete and accurate medical information and UPDRS values 130 to be disseminated and easily shared among medical staff.

[0046] Figure 2 is a schematic diagram of an example environment 200 provided by an embodiment of the present application, in which the systems and / or methods described herein can be implemented. As Figure 2 shown, the environment 200 may include a user device 210, a platform 220, and a network 230. The devices in the environment 200 can be interconnected by a wired connection, a wireless connection, or a combination of wired and wireless connections.

[0047] The user device 210 includes one or more devices capable of receiving, generating, storing, processing, and / or providing information related to the platform 220. For example, the user device 210 may include a computing device (e.g., a desktop computer, a laptop computer, a tablet computer, a handheld computer, a smart speaker, a server, etc.), a mobile phone (e.g., a smartphone, a wireless phone, etc.), a wearable device (e.g., smart glasses or a smart watch), or a similar device. In some implementations, the user device 210 can receive information from the platform 220 and / or send information to the platform 220.

[0048] As described elsewhere herein, the platform 220 includes one or more devices capable of determining Unified Parkinson's Disease Rating Scale (UPDRS) values associated with a user of the user device. In some implementations, the platform 220 may include a cloud server or a group of cloud servers. In some implementations, the platform 220 can be designed to be modular such that certain software components can be swapped in or out according to specific needs. Thus, the platform 220 can be easily and / or quickly reconfigured for different uses.

[0049] In some implementations, as shown in the figure, the platform 220 can be disposed in a cloud computing environment 222. It is noted that although the implementations described herein describe the platform 220 as being disposed in the cloud computing environment 222, in some implementations, the platform 220 may not be cloud-based (i.e., can be implemented outside of a cloud computing environment) or may be partially cloud-based.

[0050] The cloud computing environment 222 includes the environment that hosts the platform 220. The cloud computing environment 222 can provide services such as computing, software, data access, storage, etc., and these services do not require the end user (e.g., the user device 210) to know the physical location and configuration of the systems and / or devices of the hosting platform 220. As shown in the figure, the cloud computing environment 222 can include a set of computing resources 224 (collectively referred to as "multiple computing resources 224" and individually referred to as "computing resource 224").

[0051] The computing resources 224 include one or more personal computers, workstation computers, server devices, or other types of computing and / or communication devices. In some implementations, the computing resources 224 can host the platform 220. Cloud resources can include computing instances executed in the computing resources 224, storage devices provided in the computing resources 224, data transmission devices provided by the computing resources 224, etc. In some implementations, the computing resources 224 can communicate with other computing resources 224 through a wired connection, a wireless connection, or a combination of wired and wireless connections.

[0052] As Figure 2 shown, the computing resources 224 include a set of cloud resources, such as one or more applications ("APPs") 224-1, one or more virtual machines ("VMs") 224-2, virtualized storage ("VSs") 224-3, one or more hypervisors ("HYPs") 224-4, etc.

[0053] The application 224-1 includes one or more software applications that can be provided to or accessed by the user device 210. The application 224-1 can enable the software application program not to be installed and executed on the user device 210. For example, the application 224-1 can include software associated with the platform 220 and / or any other software that can be provided through the cloud computing environment 222. In some implementations, one application 224-1 can send / receive information to / from one or more other applications 224-1 through the virtual machine 224-2.

[0054] The virtual machine 224-2 includes a software implementation of a machine (e.g., a computer) that executes programs similar to a physical machine. The virtual machine 224-2 can be a system virtual machine or a process virtual machine, depending on the degree of use and correspondence of the virtual machine 224-2 with any actual machine. The system virtual machine can provide a complete system platform that supports the execution of a complete operating system ("OS"). The process virtual machine can execute a single program or support a single process. In some implementations, the virtual machine 224-2 can execute on behalf of a user (e.g., the user device 210) and can manage the infrastructure of the cloud computing environment 222, such as data management, synchronization, or long-term data transmission.

[0055] The virtualized storage 224-3 includes one or more storage systems and / or one or more devices that use virtualization technology within the storage system or device of the computing resource 224. In some implementations, in the context of a storage system, the types of virtualization may include block virtualization and file virtualization. Block virtualization may refer to the abstraction (or separation) of logical storage from physical storage so that the storage system can be accessed regardless of the physical storage or heterogeneous architecture. This separation can enable the administrator of the storage system to flexibly manage storage for end users. File virtualization can eliminate the dependency between the data accessed at the file level and the location of the physical storage file. This can optimize storage utilization, server consolidation, and / or the performance of non-disruptive file migration.

[0056] The hypervisor 224-4 can provide hardware virtualization technology that allows multiple operating systems (such as "guest operating systems") to run concurrently on a host, such as the computing resource 224. The hypervisor 224-4 can provide a virtual operating system platform for the guest operating systems and can manage the execution of the guest operating systems. Multiple instances of various operating systems can share the virtualized hardware resources.

[0057] The network 230 includes one or more wired and / or wireless networks. For example, the network 230 can include cellular networks (such as fifth-generation (5G) networks, long-term evolution (LTE) networks, third-generation (3G) networks, code division multiple access (CDMA) networks, etc.), public land mobile networks (PLMNs), local area networks (LANs), wide area networks (WANs), metropolitan area networks (MANs), telephone networks (such as the public switched telephone network (PSTN)), private networks, ad hoc networks, intranets, the Internet, fiber-based networks, etc., and / or combinations of these or other types of networks.

[0058] Figure 2 The number and arrangement of the devices and networks shown are only some examples of this application. In actual use, compared with Figure 2 the devices and / or networks shown, there may be other devices and / or networks, or fewer devices and / or networks, or different devices and / or networks, or devices and / or networks with different arrangements. In addition, Figure 2 two or more of the devices shown can be implemented within a single device, or Figure 2 a single device shown can be implemented as multiple distributed devices. Additionally, in some embodiments, a set of devices (such as one or more devices) in the environment 200 can perform one or more functions performed by another set of devices in the environment 200.

[0059] Figure 3 is a schematic diagram of example components of the device 300 in an embodiment of this application. The device 300 can correspond to the user equipment 210 and / or the platform 220. As Figure 3As shown, the device 300 may include a bus 310, a processor 320, a memory 330, a storage component 340, an input component 350, an output component 360, and a communication interface 370.

[0060] The bus 310 includes a component that permits communication between the components of the device 300. The processor 320 is implemented in hardware, firmware, or a combination of hardware and software. The processor 320 may be a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or other type of processing component. In some implementations, the processor 320 includes one or more processors that can be programmed to perform certain functions. The memory 330 includes random access memory (RAM), read only memory (ROM), and / or other types of dynamic or static storage devices (e.g., flash memory, magnetic memory, and / or optical memory) that store information and / or instructions that can be used by the processor 320.

[0061] The storage component 340 stores information and / or software related to the operation and use of the device 300. For example, the storage component 340 may include a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optical disk, and / or a solid state disk), a compact disc (CD), a digital versatile disc (DVD), a floppy disk, a cassette tape, a magnetic tape, and / or other non-transitory computer readable media, as well as corresponding drives.

[0062] The input component 350 includes components that permit the device 300 to receive information via user input (e.g., a touch screen display, a keyboard, a keypad, a mouse, a button, a switch, and / or a microphone). Additionally, in some embodiments, the input component 350 may also include sensors for sensing information (e.g., a global positioning system (GPS) component, an accelerometer, a gyroscope, and / or an actuator). The output component 360 includes components that provide output information from the device 300 (e.g., a display, a speaker, and / or one or more light emitting diodes (LEDs)).

[0063] The communication interface 370 includes components similar to a transceiver (e.g., a transceiver and / or separate receivers and transmitters) that enable the device 300 to communicate with other devices, such as via a wired connection, a wireless connection, or a combination of wired and wireless connections. The communication interface 370 may permit the device 300 to receive information from another device and / or provide information to another device. For example, the communication interface 370 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi interface, a cellular network interface, etc.

[0064] Device 300 may perform one or more of the processes described herein. Device 300 may perform these processes in response to a processor 320 executing software instructions stored in a non-volatile computer-readable medium, such as memory 330 and / or storage component 340. A computer-readable medium is defined herein as a non-volatile memory device. A memory device includes memory space within a single physical storage device or memory space distributed across multiple physical storage devices.

[0065] The software instructions may be read into memory 330 and / or storage component 340 from another computer-readable medium or from another device via communication interface 370. When executed, the software instructions stored in memory 330 and / or storage component 340 may cause processor 320 to perform one or more of the methods described herein. Additionally, in some embodiments, hardware circuitry may be used to replace or used in conjunction with the software instructions for one or more of the processes described herein. Accordingly, the implementations described herein are not limited to any particular combination of hardware circuitry and software.

[0066] Figure 3 The number and arrangement of the components shown are provided only as an example. In a practical application, device 300 may include more components, fewer components, different components, or different arrangements of components than those shown. Additionally, in some embodiments, a set of components (e.g., one or more components) of device 300 may perform one or more functions performed by another set of components of device 300. Figure 3 is a flowchart of an example process 400 for determining a Unified Parkinson's Disease Rating Scale (UPDRS) value associated with a user of a user device. In some implementations,

[0067] Figure 4 one or more of the process blocks of Figure 4 may be performed by user device 210. In some implementations, Figure 4 one or more of the process blocks of

[0068] As Figure 4 shown, process 400 may include obtaining, by a processor of the user device, video data associated with the movement of a user body part (block 410).

[0069] For example, the user device 210 may obtain video data associated with the movement of a user's body part to determine the UPDRS score. In some implementations, the user device 210 may use the camera of the user device 210 to obtain the video data. That is, a single user device 210 may obtain the video data. Additionally or alternatively, the user device 210 may obtain the video data from another user device 210. That is, multiple user devices 210 may be used to obtain the video data. For example, a set of fixed or movable user devices 210 may obtain the video data of the user.

[0070] In some implementations, the user device 210 may obtain video data at a specific angle. For example, the user device 210 may obtain video data related to the user's body part at a specific angle. Additionally or alternatively, the user device 210 may obtain video data having multiple angles with respect to the user's body part. For example, the user device 210 may be moved to obtain video data of multiple angles of the user's body part. As an alternative, multiple user devices 210 arranged at different shooting angles with respect to the user's body part may obtain respective video data arranged at different angles with respect to the user's body part.

[0071] In some implementations, the user device 210 may obtain video data of the body part that the user is interested in. For example, the user device 210 may obtain video data of the user's hand, user's finger, user's arm, user's leg, user's torso, user's head, user's leg, user's foot, the user's entire body, etc.

[0072] In some implementations, the user device 210 may provide a prompt instructing the user to perform a predetermined action via an output component. For example, the prompt may instruct the user to use the user's hand and perform a predefined action. As another example, the prompt may instruct the user to walk in a predetermined manner. As another example, the prompt may instruct the user to perform a repetitive action such as tapping the finger.

[0073] In some implementations, the user device 210 may use the video data to perform image processing techniques. For example, referring to Figure 5 , the user device 210 may obtain video data associated with the movement of the user's body part (step 510). Additionally, the user device 210 may perform image segmentation to segment the video data of the user's body part into segments (step 520). The user device 210 may perform image segmentation techniques, edge detection techniques, computer vision techniques, etc.

[0074] Thus, when determining the UPDRS score as described elsewhere herein, the user device 210 can analyze a subset of the video data that specifically corresponds to the body part of interest to the user. Therefore, the accuracy of the UPDRS score is improved.

[0075] Thus, the user device 210 can acquire the video data and use the video data to determine the UPDRS score, as described below.

[0076] As Figure 4 Further shown, the process 400 can include determining by a processor of the user device whether a measurement condition is met (block 420).

[0077] For example, the user device 210 can determine whether a measurement condition is met to acquire sufficient video data for determining the UPDRS score. The measurement condition can correspond to a threshold amount of time, a threshold amount of video data, a predetermined motion being performed, video data having a threshold image quality, a waveform of the video data meeting a threshold, an identified, segmented, and labeled body part of interest, and / or the like.

[0078] As Figure 4 Further shown, if the measurement condition is not met (block 420 - No), then the process 400 can include acquiring additional video data (return to block 410).

[0079] As Figure 4 Further shown, if the measurement condition is met (block 420 - Yes), then the process 400 can include determining by a processor of the user device the user's UPDRS value based on the model and video data associated with the motion of the body part (block 430).

[0080] In some implementations, the user device 210 can determine the UPDRS score based on the video data and the model. For example, the user device 210 can input the video data to the model and acquire the UPDRS score based on the output of the model.

[0081] Alternatively, the user device 210 can convert the video to data in another format and determine the UPDRS score based on the converted data. For example, the user device 210 can convert the video data to two - dimensional data. As an example, the data can correspond to frequency data (e.g., frequency versus time) of the motion of the user's body part. In some embodiments, the user device 210 can determine the UPDRS score based on analyzing the motion frequency of the user's body part, the motion amplitude of the user's body part, and the like.

[0082] As another alternative, the user device 210 can be based on performing Figure 5An image segmentation technique is used to identify a subset of the video corresponding to the body part of interest to the user. In addition, the user device 210 may determine the UPDRS score based on the subset of the video data.

[0083] In any case, the user device 210 may determine the UPDRS score based on the video data and provide the score as described below.

[0084] As Figure 4 As further shown, the process 400 may include providing, by a processor of the user device, a UPDRS value to allow the user to be evaluated based on the UPDRS value (block 440).

[0085] For example, the user device 210 may provide the UPDRS score for output to allow the user to identify the score.

[0086] In some implementations, the user device 210 may provide the UPDRS score for display.

[0087] As another alternative, the user device 210 may provide the data corresponding to the video data for output. For example, the user device 210 may provide a waveform of the movement of the user's body part. In this case, the user device 210 may display the change in the amplitude of the movement of the user's body part over time, the change in the frequency of the movement of the user's body part over time, etc. The user device 210 may provide any of the foregoing data together with the UPDRS score as output. Alternatively, the user device 210 may provide any of the foregoing data as output without outputting the UPDRS score. Therefore, it should be understood that the user device 210 may provide any of the foregoing data and / or its arrangement as output.

[0088] Although Figure 4 illustrates an example process of the method 400, in some implementations, the method 400 may include more operations, or fewer operations, or different operations, or operations with different settings, compared to the process shown in Figure 4 . In addition, or, two or more operations of the method 400 may be performed in parallel.

[0089] The examples and descriptions provided in the foregoing disclosure are not intended to be exhaustive or to limit the implementation to the exact form disclosed. Modifications and variations can be made in light of the foregoing disclosure, and can also be obtained from the practice of the implementation.

[0090] As used herein, the term component is broadly interpreted as hardware, firmware, or a combination of hardware and software.

[0091] It is obvious that the systems and / or methods described herein can be implemented in different forms of hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods does not limit the implementation. Therefore, the operations and behaviors of the systems and / or methods are described herein without reference to specific software code. It should be understood that the software and hardware can be designed to implement the systems and / or methods based on the descriptions herein.

[0092] Even if specific combinations of features are recited in the claims and / or the specification, these combinations are not intended to limit the disclosure that may be implemented. In fact, many of these features can be combined in ways not specifically recited in the claims and / or not disclosed in the specification. Although each dependent claim listed below may directly depend on only one claim, the disclosure that may be implemented includes the combination of each dependent claim with every other claim in the claims.

[0093] Unless explicitly stated, any element, act, or instruction used herein shall not be construed as critical or essential. Additionally, as used herein, "a" is intended to include one or more items and may be used interchangeably with "one or more". Additionally, as used herein, the term "set" is intended to include one or more items (e.g., related items, unrelated items, combinations of related and unrelated items, etc.) and may be used interchangeably with "one or more". If only one item is intended, the term "a" or similar language is used. Additionally, as used herein, the term "comprising" or similar terms are intended to be open-ended terms. Additionally, unless otherwise explicitly stated, the phrase "based on" means "at least partially based on".

Claims

1. A method for determining a Unified Parkinson's Disease Rating Scale (UPDRS) value associated with a user of a user device, performed by the user device, characterized in that Comprising: Obtaining, by a processor of a user device, video data associated with the movement of a user's body part and converting the video data into two-dimensional data; the two-dimensional data includes frequency-time data corresponding to the movement of the user's body part and / or amplitude-time data corresponding to the movement of the user's body part; Determining, by the processor of the user device, whether a measurement condition is met by executing a client application; And indicating, on a user interface of the client application, whether the measurement condition is met; the measurement condition includes: a set of gestures indicated by a prompt has been completed, a threshold amount of video data has been obtained, the video data reaches a threshold image quality, the waveform of the video data meets a threshold, and an interested body part is identified, segmented, and labeled from the video data; If the measurement condition is met, inputting, by the processor of the user device by executing the client application, the converted two-dimensional data into a model located in the cloud and determining a UPDRS value based on the output of the model located in the cloud; wherein, the client application follows a Model-View-ViewModel (MVVM) architecture, and the model located in the cloud is only used for inference calculation during the life cycle of the client application; there is a communication mechanism between the data model in the MVVM architecture and the model located in the cloud and both provide data inputs to the view model in the MVVM architecture; the view model is configured to provide services for the user interface through a data binding mechanism; and After determining the UPDRS value, storing, by the processor of the user device, the UPDRS value in a set of network-based storage devices to allow the set of devices to update and / or utilize the UPDRS value in real time.

2. The method according to claim 1, further comprising: Generating, by a sensor of the user device, sensor data based on an operation of the user of the user device on the user device; And Determining the UPDRS value based on the sensor data and the video data.

3. The method according to claim 1, further comprising: Determining, for the user, a stage of Parkinson's disease based on the video data; And Providing information identifying the stage of Parkinson's disease.

4. The method according to claim 1, wherein The UPDRS value is a Movement Disorder Society (MDS) UPDRS value.

5. The method according to claim 1, wherein, The UPDRS value is at least one of 0, 1, 2, 3, or 4.

6. The method according to claim 1, wherein, The model located in the cloud is a Deep Neural Network (DNN) model.

7. The method according to claim 1, further comprising: Providing, by an output component of the user device, a prompt for a predefined gesture of the user's body part; And Obtaining the video data based on the provided prompt.

8. A user device for determining a Unified Parkinson's Disease Rating Scale (UPDRS) value associated with a user of the user device, characterized in that, Comprising: At least one memory configured to store program code; At least one processor configured to read the program code and operate according to the instructions of the program code, the program code including: Acquisition code, configured to cause the at least one processor to acquire video data associated with the movement of a user's body part and convert the video data into two-dimensional data; the two-dimensional data includes frequency-time data corresponding to the movement of the user's body part and / or amplitude-time data corresponding to the movement of the user's body part; Determination code, configured to cause the at least one processor, by executing a client application, to determine whether a measurement condition is satisfied; and indicate on the user interface of the client application whether the measurement condition is satisfied, the measurement condition including: a set of gestures indicated by the prompts have been completed, a threshold amount of video data has been acquired, the video data reaches a threshold image quality, the waveform of the video data meets a threshold, and an interested body part has been identified, segmented, and labeled from the video data; if the measurement condition is satisfied, by executing the client application, input the converted two-dimensional data into a model located in the cloud and determine the UPDRS value according to the output of the model located in the cloud; wherein, the client application follows the Model-View-ViewModel MVVM architecture, and the model located in the cloud is only used for inference calculation during the life cycle of the client application; there is a communication mechanism between the data model in the MVVM architecture and the model located in the cloud and both provide data inputs to the view model in the MVVM architecture; the view model is configured to provide services for the user interface through a data binding mechanism; and Provision code, configured to cause the at least one processor, after determining the UPDRS value, store the UPDRS value in a set of network-based storage devices to allow the set of devices to update and / or utilize the UPDRS value in real time.

9. The user equipment according to claim 8, further comprising: Generation code, configured to cause the at least one processor to generate sensor data based on the operation of the user of the user equipment on the user equipment, wherein, the determination code is further configured to cause the at least one processor to determine the UPDRS value based on the sensor data and the video data.

10. The user equipment according to claim 8, wherein: the determination code is further configured to cause the at least one processor to determine the stage of Parkinson's disease for the user based on the video data, and the provision code is further configured to cause the at least one processor to provide information identifying the stage of Parkinson's disease.

11. The user equipment according to claim 8, wherein, The UPDRS value is the Movement Disorder Society MDS-UPDRS value.

12. The user equipment according to claim 8, wherein, The UPDRS value is at least one of 0, 1, 2, 3, or 4.

13. The user equipment according to claim 8, wherein, The model located in the cloud is a deep neural network DNN model.

14. The user equipment according to claim 8, further comprising: Provision code, configured to cause the at least one processor to provide prompts for predefined gestures of the user's body part through an output component of the user equipment, wherein, the acquisition code is further used to cause the at least one processor to acquire the video data based on the provided prompts.

15. A non-transitory computer-readable medium storing instructions, characterized in that, The instructions include: one or more instructions that, when executed by a processor of a user device for determining a Unified Parkinson's Disease Rating Scale (UPDRS) value associated with a user of the user device, implement the method according to any one of claims 1 to 7.

Images