Parkinson degree identification system and method based on UPDRS scale

Abstract

The invention discloses a Parkinson degree identification system based on a UPDRS scale, and the system comprises a camera module which is used for shooting a video and converting the video into a format for analysis; the visual analysis module is used for acquiring key points of each frame of image in the video and analyzing the key points to generate probability scores serving as function evaluation indexes; and a fusion analysis module. According to the system, the visual analysis module analyzes portraits in videos, and compared with manual evaluation according to experience, the system can quantify identification standards, shorten the diagnosis time of doctors on illness conditions, reduce subjective errors and improve the accuracy of results; visual analysis is carried out on the video, a machine learning and deep learning method is applied, judgment of a doctor on the severity degree of the Parkinson's disease based on a UPDRS scale is quantified, and the severity degree of the Parkinson's disease is judged by analyzing the sitting posture, the hand tremor under the standing posture, the muscle stiffness state during standing up, the walking pace and the arm swinging action of the portrait in the RGB video. And automatic evaluation of the Parkinson's disease is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of determining the degree of Parkinson's disease, and in particular to a system and method for identifying the degree of Parkinson's disease based on the UPDRS scale. Background Art

[0002] Parkinson's disease is a common degenerative disease of the nervous system in middle-aged and elderly people. It is mainly characterized by the pathological changes of progressive degeneration of dopaminergic neurons in the substantia nigra and the formation of Lewy bodies, biochemical changes of dopamine transmitter reduction and imbalance of dopamine and acetylcholine transmitters in the striatum, motor symptoms of tremor, muscle rigidity, bradykinesia, postural balance disorder, and non-motor symptoms such as sleep disorders, olfactory disorders, autonomic dysfunction, cognitive and mental disorders. Epidemiological surveys show that the prevalence of Parkinson's disease in people over 60 years old is more than 1%. Parkinson's disease has an insidious onset. As the disease progresses, the motor and non-motor symptoms of Parkinson's disease will gradually worsen, which will impair the patient's daily activities on the one hand, and also bring huge social and medical burdens on the other hand. Therefore, early diagnosis and timely intervention are particularly critical to improve the efficacy and quality of life. Since the treatment of Parkinson's disease is not static, the needs of patients for treatment effects at different stages of the disease are different. Therefore, doctors need to conduct individualized treatment based on experience to improve the quality of life of patients.

[0003] At present, Parkinson's disease lacks specific imaging or blood and body fluid biological markers, and its diagnosis still relies on clinical symptoms and signs. The judgment of the severity of Parkinson's disease mainly relies on the Unified Parkinson's Disease Rating Scale (UPDRS) for evaluation. Medical staff guide patients to complete specific movements through the UPDRS scale, and the assessor scores each item according to the clinical manifestations to identify Parkinson's or assess its severity. Each assessment takes about 30 minutes or even longer, and is more dependent on the assessor's experience. This identification method has a certain degree of subjectivity, and it is difficult to avoid assessment errors. In addition, a long assessment time is more likely to cause patients to be nervous or disturbed by the outside world, which further affects the accuracy of the results. Summary of the invention

[0004] The purpose of the present invention is to solve the above-mentioned background technical problems and to propose a system and method for identifying the degree of Parkinson's disease based on the UPDRS scale.

[0005] The purpose of the present invention can be achieved through the following technical solutions:

[0006] A Parkinson's degree identification system based on the UPDRS scale includes: a camera module, which is used to shoot videos and convert them into a format that can be analyzed; a visual analysis module, which is used to obtain and analyze the key points of each frame of the video to generate a probability score as a function evaluation indicator; and a fusion analysis module, which is used to fuse multiple probability scores and standing time generated by the visual analysis module to generate a feature score.

[0007] As a further solution of the present invention: the probability score includes a hand tremor score, a start-up state score and a movement state score.

[0008] As a further solution of the present invention: the visual analysis module is provided with a Mediapipe toolkit, a random forest model and a softmax function.

[0009] A method for using a Parkinson's degree identification system based on the UPDRS scale, comprising:

[0010] S1, capturing a video of the patient in a static state, recorded as a static video, performing hand tremor analysis on the hand of the person in each frame of the static video to generate a hand tremor score;

[0011] S2, capturing a video of the patient's standing up action, recording it as a start-up video, obtaining the state of the person in the start-up video and the time taken to stand up in the start-up video, and performing a limb stiffness analysis on the limbs of the person in the start-up video to generate a start-up state score;

[0012] S3, capturing a video of the patient in motion, recording it as a dynamic video, fusing features in multiple frames of the dynamic video into a single-channel image, performing motion state analysis, and generating a motion state score;

[0013] S4. The hand tremor score, starting state score, time taken to stand up and movement state score are input into the random forest model for judgment and classification, and the probability score of each state is used as the characteristic score of the Parkinson's state.

[0014] As a further solution of the present invention: in the step S1, in the analysis of the still video, it is necessary to track the hands of the portrait through the Mediapipe toolkit to obtain a number of hand key points, set a reference key point for the several hand key points, and based on the reference key point, obtain the average of the distances between the remaining hand key points and the reference key point as a feature value.

[0015] As a further solution of the present invention: in step S1, the acquired characteristic values ​​are input as overall classification features into the random forest model for analysis.

[0016] As a further solution of the present invention: in the step S2, the analysis of the startup video includes analysis of the standing up process and analysis of distinguishing between the standing up and sitting states.

[0017] As a further solution of the present invention: in the step S2, in the analysis of the startup video, the limbs of the portrait are tracked by the Meidiapiape toolkit to obtain a number of limb key points;

[0018] The standing-up process is analyzed by taking the (x, y) coordinates of several limb key points of each image in the startup video as features, selecting 15 images per second, and concatenating the features of each image with a limit of ten seconds to form an array. The array is used as a single-channel image, RestNet-18 is used as the network body, and softmax is used as the classification function for analysis to obtain the predicted probability of each state, and the predicted probability is used as the startup state score;

[0019] To distinguish and analyze the standing and sitting states, the features of each image are input into the random forest model for binary classification to determine the current state of the portrait, and the time required for the state change is used as the standard to measure the state change.

[0020] As a further solution of the present invention: in the step S3, in the analysis of the dynamic video, the limbs of the portrait are tracked by the Meidiapiape toolkit to obtain a number of limb key points;

[0021] For motion state analysis, the (x, y) coordinates of several limb key points of each image in the dynamic video are used as features. 15 images are selected per second, and the features of each image are spliced ​​to form an array with a limit of 20 seconds. The array is used as a single-channel image, with RestNet-18 as the network body and softmax as the classification function for analysis to obtain the predicted probability of each state, and the predicted probability is used as the motion state score.

[0022] Beneficial effects of the present invention:

[0023] (1) In the present invention, the visual analysis module analyzes the human image in the video. Compared with manual evaluation based on experience, it can quantify the recognition standard, shorten the doctor's diagnosis time, reduce subjective errors, and improve the accuracy of the results.

[0024] (2) In the present invention, the video is visually analyzed, and machine learning and deep learning methods are used to quantify the doctor's judgment on the severity of Parkinson's disease based on the UPDRS scale. By analyzing the sitting posture, hand tremor when standing, muscle stiffness when standing up, walking pace, and arm swinging movements of the portrait in the RGB video, an automated assessment of Parkinson's disease is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The present invention will be further described below in conjunction with the accompanying drawings.

[0026] Figure 1 It is a structural schematic diagram of a Parkinson's degree identification system based on the UPDRS scale of the present invention;

[0027] Figure 2 It is a flow chart of the method for identifying the degree of Parkinson's disease based on the UPDRS scale of the present invention;

[0028] Figure 3 yes Figure 2 The hand tremor analysis flow chart of step S1;

[0029] Figure 4 yes Figure 2 The standing state analysis flow chart of step S2;

[0030] Figure 5 yes Figure 2 The motion state analysis flow chart of step S3;

[0031] Figure 6 This is a schematic diagram of the principle of the random forest model;

[0032] Figure 7 This is a schematic diagram of the ResNet-18 network structure. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] See also Figure 1 As shown, the present invention is a Parkinson's degree identification system based on the UPDRS scale, including: a camera module, the camera module is used to shoot video and convert it into a format that can be analyzed. In this embodiment, a video in RGB format is used.

[0035] Visual analysis module,The visual analysis module is used to obtain and analyze the key points of each frame of the video, and generate a probability score as a functional evaluation indicator. The probability score includes a hand tremor score, a start-up state score, and a motion state score.

[0036] The visual analysis module also includes the Mediapipe toolkit, random forest model and softmax function, which can be used as analysis components.

[0037] Among them, the Mediapipe toolkit is used to extract and track limb key points in real time to help analyze the status of portraits in videos. The random forest model, as a classification algorithm, is used to process the extracted feature data. The softmax function is used in the output layer of multi-classification tasks to convert the model's prediction results into probability distribution, which facilitates the understanding and interpretation of the possibility of different states.

[0038] A fusion analysis module is also provided with a random forest model. The fusion analysis module is used to fuse multiple probability scores and standing times generated by the visual analysis module to generate a feature score.

[0039] See also Figure 2-Figure 7 As shown, a method for identifying the degree of Parkinson's disease based on the UPDRS scale includes:

[0040] S1. A video of the patient in a static state is captured by a camera module, which is recorded as a static video. A visual analysis module performs hand tremor analysis on the hands of the person in each frame of the static video to generate a hand tremor score.

[0041] It is understandable that patients with Parkinson's disease often show resting tremor. Parkinson's disease is caused by the degeneration and loss of dopaminergic neurons in the substantia nigra, a significant decrease in dopamine content in the striatum, and a relative hyperfunction of the acetylcholine (Ach) system, resulting in clinical symptoms such as tremor. Hand tremors are usually more obvious when the patient is relaxed or still (such as when sitting or standing), and usually alleviated during movement (such as when grasping objects). Some hand tremors manifest as typical "pill-rolling movements", in which the thumb and index finger rub back and forth involuntarily. The frequency of the tremor is usually 4-6Hz (4 to 6 times per second), and it usually appears first on one hand or finger. As the disease progresses, it is easy to affect both hands or other parts.

[0042] In this embodiment, in the analysis of still videos, it is necessary to track the hands of the portrait through the Mediapipe toolkit to obtain a number of hand key points, set a reference key point for the several hand key points, and obtain the mean value of the distance between the remaining hand key points and the reference key point based on the reference key point as the feature value, and input the obtained feature value as the overall classification feature into the random forest model for analysis.

[0043] Specifically, according to the typical symptoms of hand tremors in patients, two actions are designed to analyze the hand tremors of the portraits in the video: one is to be still in a sitting position, and the other is to be still with both hands across the chest while standing (for postural tremors, the hands can be in a natural still state. In other embodiments, sitting, standing with hands drooping, or lying flat with both hands on both sides are all possible, and are not limited to being still in a sitting position or standing with both hands across the chest). These two action-like tremors are more obvious in a relaxed state, and the hand states of the portraits are easier to capture in these postures. In the analysis of the hand states of the portraits, the left and right hands need to be processed separately, because tremors often start from one hand and easily spread to other parts as the disease progresses. For the analysis of the still video, the hand key points are obtained through the Mediapipe toolkit, and a total of 21 key points are obtained, recorded as Point0~Point 20 , take the wrist point 0 as the reference point, and obtain the mean of the distance between other key points and the point 0 as the eigenvalue. The formula for obtaining the eigenvalue is:

[0044]

[0045] Among them, Len(Point k -Point0) means to find the distance between point k and point 0. The distance formula is:

[0046]

[0047] The obtained feature value Feature = (feature1, feature2...) is input into the random forest model as the overall classification feature for analysis.

[0048] The random forest model is an integrated learning method and an extension of the decision tree. It builds multiple decision trees on multiple subsets of the data set, and then combines the prediction results of these decision trees to produce the final prediction, thereby improving the accuracy of the prediction and reducing the risk of overfitting. The random forest model can handle a large number of features and can automatically evaluate the importance of each feature. Even if there are missing or unbalanced data sets, the random forest model can usually give good results, and the random forest can be trained in parallel, with a faster training speed. In this embodiment, the hand features obtained are analyzed by the random forest model, and the hand tremor score of each hand can be obtained. The hand tremor score includes the hand tremor score in the sitting state and the hand tremor score in the state of standing on the chest.

[0049] In this embodiment, by measuring the distance between each key point of the hand and the key point of the wrist as a single feature, the distance change of multiple frames of images is used as a basis for judgment. This method achieves precise analysis of the hand tremor of the portrait in the video while ensuring accuracy.

[0050] S2. The camera module captures a video of the patient's standing up action, which is recorded as the start-up video. The visual analysis module obtains the state of the person in the start-up video and the time taken to stand up in the start-up video. The time taken to stand up is the time required for the person to go from a sitting state to a standing state. The limb stiffness of the limbs of the person in the start-up video is analyzed to generate a start-up state score.

[0051] Among them, the analysis of the startup video includes the analysis of the standing up process and the analysis of the distinction between standing up and sitting states.

[0052] Muscle stiffness in Parkinson's disease, also known as myotonia, is one of its main core symptoms. It manifests as increased skeletal muscle tension, and movement of the limbs, neck, and trunk becomes difficult. The main cause of myotonia also stems from the neuronal degeneration of the substantia nigra and striatum dopamine pathway and the lack of dopamine transmitters. The muscles of the face, neck, trunk, and limbs can all be affected. The affected limbs move slowly, and there is uniform resistance when the joints are passively moved, presenting a "lead pipe-like rigidity." If combined with tremor, "gear-like rigidity" with intermittent pauses on the uniform resistance occurs when the joints are passively extended and flexed. Severe myotonia can cause pain in the limbs, which is called painful cramps.

[0053] To address this symptom, a start-up state analysis experiment was designed, in which the patient performed a standing-up movement and filmed a starting video. By analyzing the time required for the person in the starting video to stand up and the smoothness of the movement from sitting to standing, the function of the starting state was evaluated and the corresponding score was calculated.

[0054] In this embodiment, in the analysis of the startup video, it is necessary to track the limbs of the portrait through the Meidiapiape toolkit to obtain several limb key points, which include shoulders, elbows, hips, knees, and feet. In this application, each portrait has 23 limb key points.

[0055] Specifically, for the analysis of the standing up process, the (x, y) coordinates of the 23 limb key points of each image in the startup video are used as features. 15 images are selected per second, with a limit of ten seconds, for a total of 150 images. The features of each image are concatenated into an array of shape (150, 46). The array is used as a single-channel image, with RestNet-18 as the network body and softmax as the classification function for analysis. The predicted probability of each state is obtained, and the predicted probability is used as the startup state score.

[0056] Among them, the residual network is a deep convolutional neural network structure proposed by Microsoft Data Institute, which aims to solve the problems of gradient vanishing and gradient exploding in deep neural network training. In traditional deep convolutional neural networks, network layers usually learn the mapping relationship between input and output, but as the network layer deepens, the problem of gradient vanishing occurs. The residual network introduces residual blocks to learn the identity mapping and learn the difference between input and output instead of directly learning the mapping. This structure makes it easier for the network to learn the identity mapping and avoid the problem of overfitting the training data. ResNet-18 is an architecture of deep convolutional neural networks and is part of the residual network series. Due to its relatively shallow structure, it is suitable for resource-constrained scenarios, but still provides powerful feature learning capabilities.

[0057] To distinguish and analyze the standing and sitting states, the 46 features obtained from each image are input into the random forest model for binary classification to determine the current state of the portrait. The time required for the state change is used as the criterion for measuring the state change, and it is also used as the criterion for subsequent overall analysis.

[0058] S3. The camera module captures the video of the patient in motion, which is recorded as dynamic video. The visual analysis module fuses the features in multiple frames of the dynamic video into a single-channel image to perform motion state analysis and generate a motion state score.

[0059] The analysis of movement status is closely related to muscle rigidity. The bradykinesia of Parkinson's disease patients is manifested as a decrease in voluntary movement, mainly slow movement speed and reduced amplitude, and daily activities (such as writing, tying shoelaces, and dressing) become difficult. Parkinson's disease patients often have special postures, which are manifested as postural balance disorders: the whole body is bent forward, the head and neck are tilted forward, the trunk is bent, the elbows are flexed, the forearms are adducted, and the hips and knees are slightly bent. When walking, there is a lack of joint movements such as the swing of the upper limbs forward and backward, and the postural reflexes are reduced until they are lost, which makes it easy to fall. The early manifestation of gait disorder is the dragging of the lower limbs, which gradually develops into difficulty in starting. You want to take a step but can't. The feet seem to be stuck to the ground. Once you take a step, you can walk. Once you stop, you will have difficulty starting again, which is called "frozen gait". Or after taking a step, you rush forward with very small steps (small steps), walking faster and faster, and you can't turn or stop in time, which is called "panic gait". For the analysis of movement status, it is similar to the analysis of the standing process.

[0060] Specifically, in the analysis of dynamic videos, the limbs of the portrait are tracked through the Meidiapiape toolkit to obtain several limb key points, which include shoulders, elbows, hips, knees, and feet. In this application, each portrait has 23 limb key points.

[0061] For motion state analysis, the (x, y) coordinates of the 23 limb key points of each image in the dynamic video are used as features. 15 images are selected per second, with a limit of 20 seconds, so there are a total of 300 images. The features of each image are spliced ​​into an array of shape (300, 46). The array is used as a single-channel image, with RestNet-18 as the network body and softmax as the classification function for analysis to obtain the predicted probability of each state, and the predicted probability is used as the motion state score.

[0062] In step S2 and step S3, the limb stiffness and movement state are analyzed, and the key point positions of each frame are used as features. The features of multiple frames are fused into a single-channel "image" for analysis. Compared with manual evaluation based on experience, the recognition standard can be quantified, shortening the doctor's diagnosis time and reducing subjective errors. Compared with direct analysis of the video, this method eliminates irrelevant pixels, which not only improves the accuracy of the analysis, but also greatly shortens the processing time.

[0063] S4. The fusion analysis module inputs the hand tremor score, starting state score, time taken to stand up and movement state score into the random forest model for judgment and classification, and uses the probability score of each state as the characteristic score of the Parkinson's state.

[0064] The acquired features include the hand tremor score in the sitting state feature_set, the hand tremor score in the hand standing in front of the chest state feature_stand, the starting state score feature_rise, the time taken to stand up feature_time and the exercise state score feature_exercise. For features = [feature_set, feature_stand, feature_rise, feature_time, feature_exercise], it is used as the final feature score for judging the Parkinson's state. The features are input into the random forest model for judgment and classification. The probability score of each state is used as the feature score of the Parkinson's state to realize the feature analysis of the Parkinson's state.

[0065] In this embodiment, the fusion analysis of multiple indicators can avoid the one-sidedness and poor anti-interference ability of single indicator judgment, improve the accuracy of the recognition result, and can quickly and accurately quantify and analyze the degree of Parkinson's disease.

[0066] In the present invention, the Parkinson's degree recognition system based on the UPDRS scale uses machine learning and deep learning methods in its visual analysis module and fusion analysis module to quantify the doctor's judgment on the severity of Parkinson's disease based on the UPDRS scale. By analyzing the sitting posture, hand tremor in standing posture, muscle stiffness when standing up, walking pace and arm swinging movements of the portrait in the RGB video, the automatic evaluation of Parkinson's disease is achieved.

[0067] In this application, the visual analysis module is used to analyze the hands, limbs and movement status, and these features are integrated to achieve automatic diagnosis and quantitative evaluation of Parkinson's disease. The camera module can use a low-cost, easy-to-install camera acquisition solution, as well as a method for recognizing people in the video, which can assist doctors in diagnosis and improve accuracy.

[0068] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made according to the scope of application of the present invention should still fall within the scope of the claims of the present invention.

Claims

1. A Parkinson's degree identification system based on the UPDRS scale, characterized in that: include: A camera module, which is used to shoot videos and convert them into a format that can be analyzed; A visual analysis module, which is used to obtain and analyze the key points of each frame of the video and generate a probability score as a function evaluation indicator; A fusion analysis module is used to fuse the multiple probability scores and standing times generated by the visual analysis module to generate a feature score.

2. A Parkinson's degree identification system based on the UPDRS scale according to claim 1, characterized in that: The probability scores include a hand tremor score, a start-up state score, and a movement state score.

3. A Parkinson's degree identification system based on the UPDRS scale according to claim 1, characterized in that: The visual analysis module is provided with a Medi ap i pe toolkit, a random forest model and a softmax function.

4. A method for using the Parkinson's degree identification system based on the UPDRS scale according to any one of claims 1 to 3, characterized in that: include: S1. capturing a video of the patient in a static state, recorded as a static video, performing hand tremor analysis on the hand of the person in each frame of the static video, and generating a hand tremor score; S2, capturing a video of the patient's standing up action, recording it as a start-up video, obtaining the state of the person in the start-up video and the time taken to stand up in the start-up video, and performing a limb stiffness analysis on the limbs of the person in the start-up video to generate a start-up state score; S3, capturing a video of the patient in motion, recording it as a dynamic video, fusing features in multiple frames of the dynamic video into a single-channel image, performing motion state analysis, and generating a motion state score; S4. The hand tremor score, starting state score, time taken to stand up and movement state score are input into the random forest model for judgment and classification, and the probability score of each state is used as the characteristic score of the Parkinson's state.

5. The method according to claim 4, characterized in that: In the step S1, in the analysis of the still video, it is necessary to track the hands of the portrait through the Mediapipe toolkit to obtain a number of hand key points, set a reference key point for the several hand key points, and based on the reference key point, obtain the average of the distances between the remaining hand key points and the reference key point as a feature value.

6. The method according to claim 5, characterized in that: In step S1, the acquired characteristic values ​​are input as overall classification features into the random forest model for analysis.

7. The method according to claim 4, characterized in that: In the step S2, the analysis of the startup video includes analysis of the standing up process and analysis of distinguishing between standing up and sitting states.

8. The method according to claim 7, characterized in that: In the step S2, in the analysis of the startup video, the limbs of the portrait are tracked by using the Meidiapiape toolkit to obtain a number of limb key points; The standing-up process is analyzed by taking the (x, y) coordinates of several limb key points of each image in the startup video as features, selecting 15 images per second, and concatenating the features of each image with a limit of ten seconds to form an array. The array is used as a single-channel image, RestNet-18 is used as the network body, and softmax is used as the classification function for analysis to obtain the predicted probability of each state, and the predicted probability is used as the startup state score; To distinguish and analyze the standing and sitting states, the features of each image are input into the random forest model for binary classification to determine the current state of the portrait, and the time required for the state change is used as the standard to measure the state change.

9. The method according to claim 4, characterized in that: In the step S3, in the analysis of the dynamic video, the limbs of the portrait are tracked by using the Meidiapiape toolkit to obtain a number of limb key points; For motion state analysis, the (x, y) coordinates of several limb key points of each image in the dynamic video are used as features. 15 images are selected per second, and the features of each image are spliced ​​to form an array with a limit of 20 seconds. The array is used as a single-channel image, with RestNet-18 as the network body and softmax as the classification function for analysis to obtain the predicted probability of each state, and the predicted probability is used as the motion state score.

Images