Arm movement automatic evaluation system and method for dyskinesia of cerebral palsy patient

Abstract

The invention belongs to the technical field of computer vision, and discloses an arm movement automatic evaluation system and method for dyskinesia of a cerebral palsy patient, and the system is characterized in that a data collection module carries out the movement data collection of the movement task of the arm of the patient through a smart phone camera; the posture estimation and key point detection module uses a posture estimation algorithm to automatically detect and track arm movement of a patient and recognize key points of a human body, and after coordinate data of the key points are obtained, data preprocessing is carried out; the muscular dyskinesia feature extraction module extracts muscular dyskinesia features from the motion data of the arm of the patient and constructs a corresponding feature set; the automatic scoring module uses a multi-layer feedforward neural network model to automatically calculate the score of the muscular dystonia based on the arm movement characteristics of the patient; the report generation module is used for generating an evaluation report. The method is suitable for arm movement evaluation of myodynia of cerebral palsy patients, and an objective, convenient and efficient evaluation tool is provided for clinicians.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular relates to an automatic evaluation system and method for arm movement of dystonia in patients with cerebral palsy. Background Art

[0002] Dystonia is a common neurological movement disorder, usually manifested as involuntary muscle contractions, causing abnormal postures and repetitive twisting movements. Dystonia is a relatively common symptom in patients with cerebral palsy, which seriously affects the patient's quality of life and ability to carry out daily activities. Traditional methods of assessing dystonia mainly rely on clinical observations and scoring systems, such as the Fahn-Marsden Dystonia Rating Scale (FMDRS) and the Burke-Fahn-Marsden Dystonia Rating Scale (BFMDRS). These scales score by observing the patient's muscle tone, range of motion, and motor coordination. However, this method has certain limitations, which are mainly reflected in the following aspects:

[0003] 1. High subjectivity: Traditional dystonia assessment relies on the subjective judgment of doctors. Different doctors may have large differences in the assessment results of the same patient, which leads to poor repeatability of the assessment results. In addition, the precision of the scoring system is limited and it cannot capture subtle changes in movement, resulting in some early or mild dystonia symptoms being ignored.

[0004] 2. Time-consuming and inefficient: Traditional assessment methods require doctors to judge the patient's motor function through long-term observation and recording, which is a time-consuming task for both doctors and patients. In some cases, patients need to repeat the motor task multiple times so that doctors can obtain enough information for evaluation, which not only increases the burden on patients but also reduces clinical efficiency.

[0005] 3. Low assessment accuracy: Traditional assessment methods are mainly based on visual observation and cannot accurately quantify the patient's movement characteristics, especially in subtle movements such as arm movements. It is difficult for doctors to judge specific parameters such as movement trajectory and joint angle changes by naked eyes. This lack of accuracy limits the accuracy of dystonia assessment and affects the formulation of diagnosis and treatment plans.

[0006] In order to solve the above problems, automated motion assessment technology based on computer vision has developed rapidly in recent years. This type of technology uses a camera to record the patient's motion video, and uses a posture estimation algorithm to extract the key points of the human body's motion from it, and further analyzes the motion characteristics. The advantages of this method are that it is automated, objective, and can efficiently quantify motion characteristics, providing more detailed motion data than traditional assessment methods.

[0007] Pose estimation algorithms in computer vision technology, such as Mediapipe and OpenPose, can detect multiple key points of the human body (such as shoulders, elbows, wrists, etc.) from a video with a single camera. These key points can reflect the patient's posture changes and range of motion during exercise. Compared with traditional motion capture systems, this vision-based markerless technology has the advantages of low cost, strong portability, and a wide range of applications, making it very suitable for use in clinical environments.

[0008] Although the application of computer vision technology in movement disorder assessment has shown great potential, there are still some challenges in specific neurological assessment scenarios. For example, for the assessment of dystonia in patients with cerebral palsy, the patient's movement performance may deviate from the normal movement data during algorithm training due to pathological factors, resulting in a decrease in the accuracy of the posture estimation algorithm. In addition, equipment interference and light changes in the medical environment may also affect the stability of the algorithm. Therefore, the application of existing computer vision algorithms in dystonia assessment requires further optimization and verification. Summary of the invention

[0009] In view of the problems existing in the prior art, the present invention provides an automated assessment system and method for arm movement of dystonia in patients with cerebral palsy, aiming to provide a low-cost, high-precision, easy-to-operate tool to help doctors objectively and effectively evaluate the motor function of patients with cerebral palsy, especially dystonia in arm movement.

[0010] The present invention is implemented as follows: an automated arm movement assessment system for dystonia in patients with cerebral palsy, comprising:

[0011] Data collection module: The motion data of the patient's arm is collected through the smartphone camera for five motion tasks; the motion tasks include elbow extension and flexion task, rest-tapping task, arm-to-target touch task, resting arm movement monitoring, and repeated grasping and relaxation task;

[0012] Posture estimation and key point detection module: Use the posture estimation algorithm Mediapipe to automatically detect and track the patient's arm movement, identify the key points of the human body through a convolutional neural network, and pre-process the raw data after obtaining the key point coordinate data;

[0013] Dystonia feature extraction module: extracts features related to dystonia from the patient's arm motion data and constructs a corresponding feature set;

[0014] Automatic scoring module: Uses a pre-trained multi-layer feedforward neural network (FFNN) model to automatically calculate the dystonia score based on the patient's arm movement characteristics;

[0015] Report generation module: used to generate evaluation reports, which include detailed analysis data of each motion task, total score and comparison with standard values; it also generates dynamic curve graphs to show the movement trajectory and angle changes of the patient's arm in different tasks.

[0016] Furthermore, the evaluation system is deployed through the cloud and supports simultaneous use by multiple people; users upload the patient's video data through a smartphone or computer, and the system can complete data processing and analysis in the cloud; the evaluation system supports later maintenance and updates, and can continuously optimize the performance of the neural network model based on newly accumulated patient data. The parameter adjustment and update of the model can be carried out through regular data training to cope with different types of patients with dystonia.

[0017] Furthermore, the camera in the data acquisition module should be placed in front of the patient's range of motion, with a distance of 2-3 meters to ensure that the video can fully capture the patient's arm movement.

[0018] Furthermore, the Mediapipe model is used in the posture estimation and key point detection module to automatically detect key points in the patient's arm movement based on the RGB image captured by the camera, including the shoulder, elbow, wrist, fingers and other parts related to the arm movement. These key points are represented by x, y, and z three-dimensional coordinates. The x and y coordinates represent the position of the arm in the plane, and the z coordinate is used to estimate the depth of the arm. Through the posture estimation algorithm, the system can track the patient's arm movement trajectory in real time.

[0019] Furthermore, after obtaining the key point coordinate data, the system preprocesses the original data, including removing noise and filtering abnormal data points. The system also smoothes the coordinate changes between frames to ensure the continuity and accuracy of the data.

[0020] Further, the dystonia feature extraction module performs feature extraction, specifically including:

[0021] Motion trajectory analysis: Analyze the motion trajectory of the arm, including the path of the arm raised, the angle change of the elbow when bending, etc.

[0022] Joint angle analysis: Track and analyze the joint angles of the shoulder, elbow, and wrist;

[0023] Time series feature extraction: convert motion data into time series data and perform prediction and analysis through the long short-term memory (LSTM) neural network model;

[0024] Acceleration and velocity analysis: Calculate the acceleration and velocity changes of the arm during movement.

[0025] Further, the multi-layer feed-forward neural network (FFNN) model in the automatic scoring module includes an input layer, a hidden layer, and an output layer, and is capable of processing 32 feature inputs and outputting a dystonia score for each motor task;

[0026] The number of neurons in the input layer matches the size of the feature set, the hidden layer contains 20 neurons, and the output layer corresponds to the scoring results of each motor task; the model learns the correlation between different motor features and dystonia scores by training a large-scale patient data set, so that it can accurately predict the patient's dystonia manifestations.

[0027] Furthermore, the automatic scoring module adopts a hierarchical scoring method, referring to the definition of Burke-Fahn-Marsden Movement Scale (BFMMS), to quantify the severity of the patient's dystonia; the scoring range is 0-4 points; 0 indicates no obvious dystonia, and 4 indicates severe dystonia manifestations; the multi-layer feedforward neural network (FFNN) model automatically generates a score for each movement task based on the weight of each feature, and sums up the scores of all tasks to form the final dystonia assessment result.

[0028] Furthermore, the contents of the evaluation report: the report lists the results of the patient's movement characteristics analysis in detail and provides specific data on the dystonia score; the evaluation results of each movement task are presented in a graphical form to help clinicians intuitively understand the patient's movement status;

[0029] Data comparison and tracking: The system supports comparison of assessment results in different time periods to help doctors track the progress of patients in the rehabilitation process; by comparing assessment reports on different dates, doctors can adjust the patient's rehabilitation training plan in a timely manner.

[0030] Another object of the present invention is to provide an automated assessment system for arm movement of dystonia in patients with cerebral palsy and an automated assessment method for arm movement of dystonia in patients with cerebral palsy, comprising the following steps:

[0031] Step 1: Use a smartphone camera to collect motion data of five motion tasks of the patient's arm; the motion tasks include elbow extension and flexion task, rest-tapping task, arm-to-target touch task, resting arm motion monitoring, and repeated grasping and relaxation task;

[0032] Step 2: Use the posture estimation algorithm Mediapipe to automatically detect and track the patient's arm movement, identify the key points of the human body through a convolutional neural network, and pre-process the raw data after obtaining the key point coordinate data;

[0033] Step 3, extracting features related to dystonia from the patient's arm motion data and constructing a corresponding feature set;

[0034] Step 4: Use a pre-trained multi-layer feedforward neural network (FFNN) model to automatically calculate the dystonia score based on the patient's arm movement characteristics;

[0035] Step five: Generate an evaluation report, which includes detailed analysis data for each motion task, the total score, and comparison with the standard value; it also generates a dynamic curve chart to show the movement trajectory and angle changes of the patient's arm in different tasks.

[0036] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0037] First, the present invention proposes an automated dystonia assessment system that combines computer vision and machine learning technologies, specifically for assessing the arm motor function of patients with cerebral palsy; the system automatically identifies and analyzes the motion characteristics of the patient's arm, such as joint angles, motion trajectories, speed, and acceleration, by recording the patient's motion data through a smartphone camera. By inputting these features into a pre-trained neural network model, the system can automatically generate a dystonia score to help doctors accurately assess the patient's motor function. The present invention has the following significant advantages:

[0038] 1. Objectivity and accuracy: The system conducts evaluation in a data-driven manner, reducing the bias caused by subjective judgment, capturing subtle changes in motion, and providing accurate quantitative data.

[0039] 2. Efficient automation: The system can quickly process large amounts of motion data, reducing the workload of doctors and significantly improving the efficiency of clinical evaluation.

[0040] 3. Convenience and low cost: This system does not require special hardware equipment. The assessment can be completed by simply recording the patient's video through the smartphone camera. It is suitable for a wide range of clinical and rehabilitation environments.

[0041] In summary, the present invention can effectively solve the limitations of the prior art and is particularly suitable for the arm movement assessment of dystonia in patients with cerebral palsy, providing clinicians with an objective, convenient and efficient assessment tool.

[0042] Second, the technical solution of the present invention brings a number of breakthrough advances in the assessment of dystonia in patients with cerebral palsy by realizing an automated arm movement assessment system, and solves a number of problems in the prior art.

[0043] 1. Problems with existing technologies

[0044] At present, the assessment of dystonia for patients with cerebral palsy mostly relies on manual observation and evaluation, which leads to strong subjectivity, large errors, and low efficiency. Traditional assessment requires on-site operation by professional doctors and is limited by time, energy, and region. There are few existing automated assessment technologies, which cannot achieve real-time monitoring and feedback, and it is difficult to meet the patients' needs for assessment at any time in different scenarios. At the same time, the lack of flexible muscle tension feature analysis algorithms and accurate scoring systems leads to unsatisfactory accuracy and consistency of the results, limiting the application of the system in actual medical diagnosis and rehabilitation management.

[0045] 2. Significant technological progress brought about by the technical solution of the present invention

[0046] The arm movement automated assessment system proposed in this invention has a series of technological advances, realizing full process automation from data collection, posture estimation and key point detection, feature extraction, automatic scoring to report generation. Specifically:

[0047] Multi-task data collection: Complete data collection of 5 key motion tasks through the smartphone camera, covering elbow joint movement, tapping, grasping and other actions, forming a complete arm motion task set. The data collection process does not require special equipment, which greatly simplifies patient operation and improves convenience.

[0048] Intelligent posture detection and data processing: Using Mediapipe posture estimation and convolutional neural network models, the system can automatically detect and track the key points of the patient's arm movement. Based on the automated key point recognition algorithm, it not only reduces the complexity of data collection, but also enhances accuracy, allowing the entire detection process to be carried out in real time and efficiently, ensuring accurate capture of movement information.

[0049] Feature extraction and automatic scoring model: Features closely related to dystonia are extracted from arm movement data, and automatic scoring is performed based on a pre-trained multi-layer feedforward neural network model, making the evaluation results more objective and consistent and avoiding human errors. In addition, the scoring model will be optimized based on the accumulated data to continuously improve the system's adaptability to diverse patients.

[0050] Report generation and data visualization: The system can automatically generate detailed evaluation reports, including specific task analysis, total scores and comparisons with standard values. The patient's movement trajectory and angle changes will also be presented in the form of dynamic curve graphs, making it easier for doctors and patients to intuitively understand the changes in arm movement characteristics.

[0051] 3. Advantages and feasibility in industrial applications

[0052] Assessment service anytime, anywhere: Based on the cloud-based deployment structure design, the system supports remote uploading and analysis of patient video data, which greatly expands the use scenarios of assessment. Users do not need to go to professional institutions, but only need a smartphone or computer to complete data collection and analysis, which effectively improves the patient's self-assessment ability.

[0053] Flexible and scalable update capability: The system continuously optimizes the scoring algorithm based on regular data training and model updates, and can improve the assessment accuracy in real time according to actual application conditions. It has good industrialization potential and market adaptability, and can support the rehabilitation management of more types of dystonia patients.

[0054] In summary, the automated assessment system of the present invention greatly improves the accuracy, real-time and convenience of dystonia assessment in cerebral palsy patients, reduces the assessment cost and time for patients, and promotes the digitalization process of the medical and rehabilitation industries. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a structural diagram of an automated arm movement assessment system for dystonia in patients with cerebral palsy provided by an embodiment of the present invention;

[0056] Figure 2 Schematic diagram of an automated arm movement assessment system for dystonia in patients with cerebral palsy provided by an embodiment of the present invention;

[0057] Figure 3 It is a schematic diagram of a task option page of an automated arm movement assessment system for dystonia in patients with cerebral palsy provided by an embodiment of the present invention;

[0058] Figure 4 It is a schematic diagram of a score calculation page of an automated arm movement assessment system for dystonia in patients with cerebral palsy provided by an embodiment of the present invention;

[0059] Figure 5 It is a flow chart of an automated assessment method for arm movement of dystonia in patients with cerebral palsy provided by an embodiment of the present invention.

[0060] Figure 6 This is a diagram of a gesture recognition example provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0062] like Figure 1 As shown, the embodiment of the present invention provides an automated assessment system for arm movement of dystonia in patients with cerebral palsy, comprising:

[0063] Data collection module: The motion data of the patient's arm is collected through the smartphone camera for five motion tasks; the motion tasks include elbow extension and flexion task, rest-tapping task, arm-to-target touch task, resting arm movement monitoring, and repeated grasping and relaxation task;

[0064] Posture estimation and key point detection module: Use the posture estimation algorithm Mediapipe to automatically detect and track the patient's arm movement, identify the key points of the human body through a convolutional neural network, and pre-process the raw data after obtaining the key point coordinate data;

[0065] Dystonia feature extraction module: extracts features related to dystonia from the patient's arm motion data and constructs a corresponding feature set;

[0066] Automatic scoring module: Uses a pre-trained multi-layer feedforward neural network (FFNN) model to automatically calculate the dystonia score based on the patient's arm movement characteristics;

[0067] Report generation module: used to generate evaluation reports, which include detailed analysis data of each motion task, total score and comparison with standard values; it also generates dynamic curve graphs to show the movement trajectory and angle changes of the patient's arm in different tasks.

[0068] The system uses a smartphone camera to collect data on five designated movement tasks performed by the arm of a cerebral palsy patient, including elbow extension and flexion, tapping at rest, arm touching a target, arm movement monitoring at rest, and repeated grasping and relaxing tasks. The data acquisition module captures the patient's movement data from the camera, and the system uses these tasks to fully cover the patient's arm function and the manifestations of dystonia.

[0069] After data collection is completed, the posture estimation and key point detection module starts working. This module uses the Mediapipe posture estimation algorithm to automatically detect and track the movement of the patient's arm through a convolutional neural network, identify the key points of the human body, and obtain the joint coordinates of the patient's arm. Next, these raw coordinate data are preprocessed to eliminate noise and outliers, providing high-quality basic data for subsequent feature extraction.

[0070] The dystonia feature extraction module in the system is responsible for extracting features related to dystonia from the motion data, such as flexibility, speed, angle change, and coordination of arm movements. By extracting these features, the system establishes a feature set containing the patient's motion features for automated evaluation and scoring.

[0071] Finally, the automatic scoring module uses a pre-trained multi-layer feedforward neural network model to analyze the feature set and generate a scoring result for the patient's dystonia. The report generation module further displays the results as a complete evaluation report, which includes a detailed analysis of each movement task, the patient's overall score and a comparison with the standard value. It also generates a dynamic curve chart to show the arm movement trajectory and angle changes, helping doctors fully understand the patient's motor function status.

[0072] The data acquisition module is used to record the patient's arm movements through the smartphone camera. The patient needs to perform preset tasks, such as extending and flexing the elbow, or repetitive arm lifting movements. These tasks can be performed in a hospital, rehabilitation center, or the patient's daily life environment. The camera should be placed in front of the patient's range of motion and the distance should be kept between 2-3 meters to ensure that the video can fully capture the patient's arm movements. The following are the specific task setting steps:

[0073] (1) Elbow extension and flexion tasks:

[0074] Objective: To detect dystonia in patients during passive and active arm movements, especially stiffness and involuntary movements of the joints.

[0075] Task description: The patient sits in a chair with his back straight, his arms in a neutral shoulder position, his elbows bent 90 degrees, and his forearms in a supine position. The experimenter uses a dedicated device or manually guides the patient's arm to perform passive extension and flexion. This task evaluates the patient's muscle tension response and joint stiffness at different speeds by pushing with external forces at different speeds. The experimenter will perform the test at slow speeds (25 times / minute), medium speed (100 times / minute), and fast speed (175 times / minute), and collect 6 cycles of motion data at each speed.

[0076] (2) Rest-tapping task:

[0077] Objective: To detect dystonia induced by movements of the other hand at rest, particularly to assess the intensity and extent of overflow movements.

[0078] Task description: The patient remains at rest in a chair with his hands on his thighs and his back upright. At the beginning of the experiment, the patient is asked to repeatedly tap (once per second) with the thumb and index finger of the contralateral (unmeasured) hand, which will induce overflow movement of the arm on the measured side. The system records abnormal movements of the patient's resting arm through a camera, such as arm shaking, involuntary arm movement or muscle tension.

[0079] (3) Arm-to-target touch task:

[0080] Objective: To evaluate the effects of dystonia on arm control and movement trajectory during active movements, especially the curvature, velocity, and endpoint error of the arm during the task.

[0081] Task description: The patient sits in a chair with his back straight, shoulders relaxed, and arms hanging naturally. After the start instruction, the patient is asked to quickly reach out and touch a 4-cm diameter target ball located 90% of the arm length in front of the shoulder height. The patient needs to touch the target with his fingers and hold it for a period of time to assess the motor control ability of the arm due to dystonia. The experimenter records the patient's movement trajectory, peak speed, path curvature, end point error, and holding distance.

[0082] (4) Resting arm movement monitoring:

[0083] Purpose: To detect spontaneous dystonia of the arm at rest, particularly involuntary muscle contractions or hypermobility.

[0084] Task description: The patient is asked to sit in a chair and relax, with the arms hanging naturally at both sides of the body. The camera continuously records the movement of the arm in a static state, and the system automatically detects and analyzes whether the arm has involuntary shaking, twisting or other movement abnormalities in the static state. This task is used to monitor the muscle performance of patients with dystonia without external stimulation, and is an important task for evaluating non-task-related movement abnormalities.

[0085] (5) Repeated grasping and relaxing task:

[0086] Objective: To evaluate patients for dystonia during repetitive movements, specifically changes in coordination and muscle tone in the fingers and wrist.

[0087] Task description: The patient is asked to repeatedly grasp and relax the fingers, and the system records the speed, frequency, range of finger movements, and joint flexibility. This task helps to detect the impact of dystonia in repetitive movements, especially the control of wrist and finger muscles. The time and amplitude of each grasping movement are recorded and analyzed by the camera to evaluate the patient's movement abnormalities caused by dystonia.

[0088] The posture estimation and key point detection modules use computer vision algorithms such as open source Mediapipe to automatically detect and track the patient's arm movements from the video. These algorithms use convolutional neural networks (CNNs) to identify 33 key points of the human body, including the shoulder, elbow, wrist, fingers and other parts related to arm movements.

[0089] (1) Posture estimation algorithm: The Mediapipe model automatically detects key points in the patient's arm movement based on the RGB image captured by the camera. These key points are represented by x, y, and z three-dimensional coordinates. The x and y coordinates represent the position of the arm in the plane, and the z coordinate is used to estimate the depth of the arm. Through the posture estimation algorithm, the system can track the patient's arm movement trajectory in real time.

[0090] (2) Data preprocessing: After obtaining the key point coordinate data, the system preprocesses the raw data. This step includes removing noise and filtering abnormal data points. In order to avoid interference with the detection results caused by camera jitter or light changes, the system will smooth the coordinate changes between frames to ensure data continuity and accuracy.

[0091] The dystonia feature extraction module extracts features related to dystonia from the patient's arm motion data. These features can reflect the patient's involuntary muscle contraction and abnormal movement control.

[0092] (1) Movement trajectory analysis: The system first analyzes the movement trajectory of the arm, including the path of the arm when raised and the angle change of the elbow when bent. For patients with dystonia, the trajectory of arm movement may be abnormally bent or twisted, and these abnormalities can be reflected by analyzing the curvature and irregularity of the movement trajectory.

[0093] (2) Joint angle analysis: The system tracks and analyzes the joint angles of the shoulder, elbow, and wrist. By calculating the angle changes of the joints at different time points, the system can detect whether the arm has stiffness, twisting, or other manifestations of dystonia. Changes in joint angles are an important indicator in the assessment of dystonia, especially when the patient performs repetitive movements, abnormal changes in angles can often clearly reflect the problem.

[0094] (3) Time series feature extraction: The system converts motion data into time series data and uses a long short-term memory (LSTM) neural network model for prediction and analysis. The LSTM model is specifically designed to process time series data and can capture dynamic changes in arm movements and identify abnormal movement patterns caused by dystonia. The model detects subtle abnormalities associated with dystonia by learning the continuity and regularity of the patient's movements.

[0095] (4) Acceleration and velocity analysis: The system also calculates the acceleration and velocity changes of the arm during movement. For patients with dystonia, arm movements are often accompanied by sudden pauses or irregular velocity changes, which can be detected by analyzing the peak value and frequency of movement velocity. Especially when patients perform repetitive movement tasks, abnormal changes in acceleration and velocity are important bases for evaluating dystonia.

[0096] The automatic scoring module uses a multi-layer feedforward neural network (FFNN) model, which includes an input layer, a hidden layer, and an output layer. It can process 32 feature inputs and output a dystonia score for each motor task.

[0097] (1) Model structure: The number of neurons in the input layer matches the size of the feature set, the hidden layer contains 20 neurons, and the output layer corresponds to the score results of each motor task. The model is trained on a large-scale patient dataset to learn the correlation between different motor features and dystonia scores, so that it can accurately predict the patient's dystonia manifestations.

[0098] (2) Scoring criteria: The system uses a hierarchical scoring method, referring to the definition of the Burke-Fahn-Marsden Movement Scale (BFMMS), to quantify the severity of the patient's dystonia. The score ranges from 0 to 4 points. 0 indicates no obvious dystonia, and 4 indicates severe dystonia. The model automatically generates a score for each movement task based on the weight of each feature, and sums up the scores of all tasks to form the final dystonia assessment result.

[0099] The report generation module includes detailed analysis data, total score and comparison with standard values ​​for each motion task. In addition, the system can also generate dynamic curve graphs to show the movement trajectory and angle changes of the patient's arm in different tasks, providing clinicians with detailed evaluation results.

[0100] (1) Report content: The report lists the results of the patient's movement characteristics analysis in detail and provides specific data on the dystonia score. The evaluation results of each movement task are presented in the form of a chart to help clinicians intuitively understand the patient's movement status.

[0101] (2) Data comparison and tracking: The system supports comparison of evaluation results at different time periods, helping doctors track the progress of patients’ rehabilitation. By comparing evaluation reports from different dates, doctors can adjust patients’ rehabilitation training plans in a timely manner to improve treatment outcomes.

[0102] The evaluation system of the present invention can be deployed through the cloud. Users only need to upload the patient's video data through a smartphone or computer, and the system can complete data processing and analysis in the cloud. The system supports multiple people to use at the same time and can be widely used in hospitals, rehabilitation centers and other environments.

[0103] (1) Cloud deployment: The system’s computing and storage resources are hosted in the cloud, which can process data from multiple patients simultaneously. Users only need to upload videos, and the cloud system will automatically analyze and return an evaluation report. The operation is simple and suitable for rapid application in clinical environments.

[0104] (2) System maintenance and update: The system supports post-maintenance and update, and can continuously optimize the performance of the neural network model based on newly accumulated patient data to ensure the accuracy and reliability of the evaluation results. The model's parameter adjustment and update can be performed through regular data training to cope with different types of dystonia patients.

[0105] like Figure 5 As shown, an embodiment of the present invention provides an automated assessment system for arm movement of dystonia in patients with cerebral palsy and an automated assessment method for arm movement of dystonia in patients with cerebral palsy, comprising the following steps:

[0106] Step 1: Use a smartphone camera to collect motion data of five motion tasks of the patient's arm; the motion tasks include elbow extension and flexion task, rest-tapping task, arm-to-target touch task, resting arm motion monitoring, and repeated grasping and relaxation task;

[0107] Step 2: Use the posture estimation algorithm Mediapipe to automatically detect and track the patient's arm movement, identify the key points of the human body through a convolutional neural network, and pre-process the raw data after obtaining the key point coordinate data;

[0108] Step 3, extracting features related to dystonia from the patient's arm motion data and constructing a corresponding feature set;

[0109] Step 4: Use a pre-trained multi-layer feedforward neural network (FFNN) model to automatically calculate the dystonia score based on the patient's arm movement characteristics;

[0110] Step five: Generate an evaluation report, which includes detailed analysis data for each motion task, the total score, and comparison with the standard value; it also generates a dynamic curve chart to show the movement trajectory and angle changes of the patient's arm in different tasks.

[0111] The following are two specific application examples of the system:

[0112] Example 1: Assessment of dystonia in children with cerebral palsy

[0113] In a rehabilitation treatment, the doctor used the automatic arm movement assessment system for a child with cerebral palsy. First, the data acquisition module recorded the patient's arm movement data for five movement tasks through the smartphone camera, including elbow extension, flexion, and movement of the arm in a static state. Then, the system's posture estimation and key point detection module used the Mediapipe algorithm to extract the movement coordinates of the child's elbow, wrist and other key points. Through the dystonia feature extraction module, the system extracted the incoordination and abnormal angle changes of the patient's arm during movement, and generated a feature set related to dystonia. Finally, the automatic scoring module calculated the child's dystonia score based on these feature sets and generated an assessment report, providing detailed analysis of each task and the total score. The dynamic curve chart shows the movement trajectory of the patient's arm in each task, helping doctors determine a suitable rehabilitation training plan.

[0114] Example 2: Remote Assessment of Dystonia in Adult Patients

[0115] like Figure 6 As shown in the figure, an adult patient with cerebral palsy living in a remote area undergoes a remote rehabilitation assessment via a smartphone. The patient performs five arm movement tasks specified by the system at home using the smartphone camera and uploads the data to the cloud in real time. The system's posture estimation and key point detection modules process this data remotely, using convolutional neural networks to track the movement of the patient's arm, and extract relevant features such as the speed change of arm movements and abnormal muscle tone features through the dystonia feature extraction module. The evaluation report and dynamic motion curve graph generated by the system are viewed by the remote rehabilitation doctor, who can use the report to adjust the rehabilitation plan for the patient without the patient having to go to the rehabilitation center in person, greatly facilitating the application of telemedicine and rehabilitation assessment.

[0116] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0117] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. An automated arm movement assessment system for dystonia in patients with cerebral palsy, characterized in that: include: Data collection module: The motion data of the patient's arm is collected through the smartphone camera for five motion tasks; the motion tasks include elbow extension and flexion task, rest-tapping task, arm-to-target touch task, resting arm movement monitoring, and repeated grasping and relaxation task; Posture estimation and key point detection module: Use the posture estimation algorithm Mediapipe to automatically detect and track the patient's arm movement, identify the key points of the human body through a convolutional neural network, and pre-process the raw data after obtaining the key point coordinate data; Dystonia feature extraction module: extracts features related to dystonia from the patient's arm motion data and constructs a corresponding feature set; Automatic scoring module: Uses a pre-trained multi-layer feedforward neural network model to automatically calculate the dystonia score based on the patient's arm movement characteristics; Report generation module: used to generate evaluation reports, which include detailed analysis data of each motion task, total score and comparison with standard values; it also generates dynamic curve graphs to show the movement trajectory and angle changes of the patient's arm in different tasks.

2. The automated arm movement assessment system for dystonia in patients with cerebral palsy as claimed in claim 1, characterized in that: The evaluation system is deployed through the cloud and supports simultaneous use by multiple people; users upload the patient's video data through smartphones or computers, and the system completes data processing and analysis in the cloud; the evaluation system supports later maintenance and updates, and continuously optimizes the performance of the neural network model based on newly accumulated patient data. The model's parameter adjustment and update are carried out through regular data training to cope with different types of dystonia patients.

3. The automated arm movement assessment system for dystonia in patients with cerebral palsy as claimed in claim 1, characterized in that: The camera in the data acquisition module should be placed in front of the patient's range of motion, with a distance of 2-3 meters to ensure that the video can fully capture the patient's arm movement.

4. The automated arm movement assessment system for dystonia in patients with cerebral palsy as claimed in claim 1, characterized in that: The Mediapipe model is used in the posture estimation and key point detection module to automatically detect key points in the patient's arm movement based on the RGB image captured by the camera, including the shoulder, elbow, wrist, fingers and other parts related to the arm movement. These key points are represented by x, y, and z three-dimensional coordinates. The x and y coordinates represent the position of the arm in the plane, and the z coordinate is used to estimate the depth of the arm. Through the posture estimation algorithm, the system tracks the patient's arm movement trajectory in real time.

5. The automated arm movement assessment system for dystonia in patients with cerebral palsy as claimed in claim 4, characterized in that: After obtaining the key point coordinate data, the system preprocesses the original data, including removing noise and filtering abnormal data points. The system also smoothes the coordinate changes between frames to ensure the continuity and accuracy of the data.

6. The automated arm movement assessment system for dystonia in patients with cerebral palsy as claimed in claim 1, characterized in that: The dystonia feature extraction module performs feature extraction, specifically including: Movement trajectory analysis: Analyze the movement trajectory of the arm, including the path of the arm when raised and the angle change of the elbow when bent; Joint angle analysis: Track and analyze the joint angles of the shoulder, elbow, and wrist; Time series feature extraction: convert motion data into time series data, and predict and analyze it through long short-term memory neural network model; Acceleration and velocity analysis: Calculate the acceleration and velocity changes of the arm during movement.

7. The automated arm movement assessment system for dystonia in patients with cerebral palsy as claimed in claim 1, characterized in that: The multi-layer feedforward neural network model in the automatic scoring module includes an input layer, a hidden layer, and an output layer, processes 32 feature inputs, and outputs a dystonia score for each motor task; The number of neurons in the input layer matches the size of the feature set, the hidden layer contains 20 neurons, and the output layer corresponds to the scoring results of each motor task; the model learns the correlation between different motor features and dystonia scores by training a large-scale patient data set, and accurately predicts the patient's dystonia manifestations.

8. The automated arm movement assessment system for dystonia in patients with cerebral palsy as claimed in claim 1, characterized in that: The automatic scoring module uses a hierarchical scoring method, referring to the definition of the Burke-Fahn-Marsden Movement Scale, to quantify the severity of the patient's dystonia; the scoring range is 0-4 points; 0 indicates no obvious dystonia, and 4 indicates severe dystonia manifestations; The multi-layer feedforward neural network model automatically generates a score for each motor task based on the weight of each feature, and sums up the scores of all tasks to form the final dystonia assessment result.

9. The automated arm movement assessment system for dystonia in patients with cerebral palsy as claimed in claim 1, characterized in that: Contents of the evaluation report: The report lists the results of the patient's movement characteristics analysis in detail and provides specific data on the dystonia score; the evaluation results of each movement task are presented in a graphical form to help clinicians intuitively understand the patient's movement status; Data comparison and tracking: The system supports comparison of assessment results in different time periods to help doctors track the progress of patients in the rehabilitation process; by comparing assessment reports on different dates, doctors can adjust patients' rehabilitation training plans in a timely manner.

10. An automated assessment method for arm movement of dystonia in patients with cerebral palsy according to the system of any one of claims 1 to 9, characterized in that: The following steps are involved: Step 1: Use a smartphone camera to collect motion data of five motion tasks of the patient's arm; the motion tasks include elbow extension and flexion task, rest-tapping task, arm-to-target touch task, resting arm motion monitoring, and repeated grasping and relaxation task; Step 2: Use the posture estimation algorithm Mediapipe to automatically detect and track the patient's arm movement, identify the key points of the human body through a convolutional neural network, and pre-process the raw data after obtaining the key point coordinate data; Step 3, extracting features related to dystonia from the patient's arm motion data and constructing a corresponding feature set; Step 4: Use a pre-trained multi-layer feedforward neural network model to automatically calculate the dystonia score based on the patient's arm movement characteristics; Step five: Generate an evaluation report, which includes detailed analysis data for each motion task, the total score, and comparison with the standard value; it also generates a dynamic curve chart to show the movement trajectory and angle changes of the patient's arm in different tasks.

Images