Methods, devices and electronic equipment for assessing motor function in Parkinson's disease

Abstract

This invention provides a method, device, and electronic device for assessing motor function in Parkinson's disease, comprising: acquiring image data of a user performing an action and scores for other items in Part III of the Unified Parkinson's Disease Rating Scale (UPRS), excluding limb rigidity and postural stability assessment items; preprocessing the image data to obtain preprocessed image data; extracting the positional information of facial features and human feature points from the preprocessed image data; inputting the positional information of facial features and human feature points into a machine learning model to output the user's limb rigidity score and postural stability score; determining the user's Part III score of the UPS based on the limb rigidity score, postural stability score, and scores of other items; and determining the user's motor function assessment result for Parkinson's disease based on the Part III score. This method directly scores facial features and human feature points in the image using a machine learning model, improving assessment efficiency.

Description

Technical Field

[0001] This invention relates to the field of medical diagnostic technology, and in particular to a method, device, and electronic device for assessing motor function in Parkinson's disease. Background Technology

[0002] Currently, in the assessment of limb rigidity and postural stability in Part III of the MDS-UPDRS (Unified-Parkinson's Disease Rating Scale), trained assessors need to physically touch the patient's body. For assessing limb rigidity, the assessor rotates the patient's neck, feels muscle tone in the limb joints, and assigns a score. For postural stability, the assessor performs a pull-back test, scoring the patient based on how many steps they need to step back to regain balance. During this process, patients may fall due to balance issues, so trained assessors need to be prepared to catch them at any time.

[0003] Furthermore, because the assessment process in the third part of MDS-UPDRS is mainly performed by assessors, the current methods for assessing motor function in Parkinson's disease are inefficient. Summary of the Invention

[0004] The purpose of this invention is to provide a method, device, and electronic device for assessing limb stiffness and postural stability, which alleviates the problem that the assessment process of the third part of MDS-UPDRS is mainly performed by the assessor, thereby improving the assessment efficiency of motor function assessment for Parkinson's disease.

[0005] In a first aspect, embodiments of the present invention provide a method for assessing motor function in Parkinson's disease, comprising: acquiring image data of a user performing a preset action, and scores for other items in Part III of the Unified Parkinson's Disease Rating Scale (UPRS), excluding limb rigidity and postural stability assessment items; the preset action being the action corresponding to Part III of the UPS; preprocessing the image data to obtain preprocessed image data; extracting the user's facial features and the location information of the user's body feature points from the preprocessed image data; inputting the location information of the facial features and the body feature points into a pre-trained machine learning model, and outputting the user's limb rigidity score and postural stability score; the machine learning model being pre-trained based on images carrying preset limb rigidity scores and preset postural stability scores; and determining the user's Part III score of the UPS based on the limb rigidity score, the postural stability score, and the scores of the other items, thereby determining the user's motor function assessment result for Parkinson's disease based on the Part III score.

[0006] In a preferred embodiment of the present invention, the aforementioned machine learning model includes: an upper limb stiffness scoring model, a lower limb stiffness scoring model, a neck stiffness scoring model, and a posture stability scoring model; the step of inputting the positional information of the aforementioned facial features and the aforementioned human feature points into the pre-trained machine learning model and outputting the user's limb stiffness score and posture stability score includes: inputting the positional information of the aforementioned human feature points into the aforementioned upper limb stiffness scoring model and outputting an upper limb stiffness score; inputting the positional information of the aforementioned human feature points into the aforementioned lower limb stiffness scoring model and outputting a lower limb stiffness score; inputting the positional information of the aforementioned facial features and the aforementioned human feature points into the aforementioned neck stiffness scoring model and outputting a neck stiffness score; inputting the positional information of the aforementioned human feature points into the aforementioned posture stability scoring model and outputting a posture stability score; determining the user's limb stiffness score based on the aforementioned upper limb stiffness score, the aforementioned lower limb stiffness score, and the aforementioned neck stiffness score, and outputting the user's limb stiffness score.

[0007] In a preferred embodiment of the present invention, after the step of extracting the user's facial features and the location information of the user's human body feature points from the preprocessed image data, the method includes: determining the user's kinematic features and signal characteristics based on the location information of the human body feature points and the time data corresponding to the location information; and inputting the facial features and the location information of the human body feature points into a pre-trained machine learning model to output the user's limb stiffness score and posture stability score, which includes: inputting the facial features, the kinematic features, and the signal characteristics into a pre-trained machine learning model to output the user's limb stiffness score and posture stability score.

[0008] In a preferred embodiment of the present invention, the step of determining the kinematic and signal characteristics of the user based on the location information of the human feature points and the time data corresponding to the location information includes: determining the change curve of the vertical coordinate of the location information of the feature points based on the location information of the human feature points and the time data corresponding to the location information; and determining the kinematic and signal characteristics of the user based on the change curve.

[0009] In a preferred embodiment of the present invention, the step of determining the kinematic characteristics of the user based on the aforementioned variation curve includes: smoothing the aforementioned variation curve using a smooth spline method to obtain a smooth curve signal; extracting the frequency and amplitude of the smooth curve signal; calculating the maximum value, minimum value, mean, and standard deviation of the aforementioned frequency and amplitude, respectively; and determining the kinematic characteristics of the user based on the maximum value, minimum value, mean, and standard deviation of the aforementioned frequency and amplitude.

[0010] In a preferred embodiment of the present invention, the step of determining the signal characteristics of the user based on the aforementioned variation curve includes: calculating the maximum value, minimum value, peak value, quantile, root mean square, absolute mean, standard deviation, coefficient of variation, kurtosis coefficient, and skewness coefficient of the aforementioned variation curve, and performing a Fourier transform on the aforementioned variation curve to obtain first preprocessed data; performing a wavelet transform on the aforementioned variation curve to obtain second preprocessed data; calculating the sum of amplitudes corresponding to different frequencies in the aforementioned first preprocessed data to obtain an amplitude sum; and filtering the coefficient values ​​of the aforementioned second preprocessed data; calculating the absolute values ​​of the aforementioned coefficient values, and summing the absolute values ​​of the aforementioned coefficient values ​​to obtain an absolute value sum; and determining the maximum value, minimum value, peak value, quantile, root mean square, absolute mean, standard deviation, coefficient of variation, kurtosis coefficient, skewness coefficient, amplitude sum, and absolute value sum of the aforementioned variation curve as the signal characteristics of the user.

[0011] In a preferred embodiment of the present invention, the above-mentioned machine learning model is constructed based on Light GBM.

[0012] In a preferred embodiment of the present invention, before inputting the positional information of the facial features and the human body feature points into a pre-trained machine learning model, the method includes: inputting the facial features, the kinematic features, and the signaling features into a preset initial machine learning model and outputting information gain; sorting the information gain to obtain sorted information gain; grouping the sorted information gain according to preset parameters to obtain multiple feature groups; training the initial machine learning model with the target facial features, target kinematic features, and target signaling features corresponding to the multiple feature groups respectively to obtain multiple trained initial machine learning models; evaluating the performance of the multiple trained initial machine learning models using leave-one-out cross-validation to obtain multiple performance values; and determining the initial machine learning model whose performance value reaches a preset threshold as the pre-trained machine learning model.

[0013] Secondly, embodiments of the present invention also provide a motor function assessment device for Parkinson's disease, comprising: a data acquisition module, used to acquire image data of a user completing a preset action, and scores of other items in Part III of the Unified Parkinson's Disease Rating Scale (UPRS), excluding limb rigidity and postural stability assessment items; the preset action is the action corresponding to Part III of the UPS; an image preprocessing module, used to preprocess the image data to obtain preprocessed image data; a feature extraction module, used to extract the user's facial features and the location information of the user's human body feature points from the preprocessed image data; a model evaluation module, used to input the location information of the facial features and the human body feature points into a pre-trained machine learning model, and output the user's limb rigidity score and postural stability score; the machine learning model is pre-trained based on images carrying preset limb rigidity scores and preset postural stability scores; and an evaluation result output module, used to determine the user's Part III score of the UPS based on the limb rigidity score, the postural stability score, and the scores of the other items, so as to determine the user's Parkinson's disease motor function assessment result based on the Part III score.

[0014] Thirdly, embodiments of the present invention also provide an electronic device, which includes a processor and a memory. The memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the above-mentioned method for assessing motor function in Parkinson's disease.

[0015] The embodiments of the present invention have the following beneficial technical effects:

[0016] This invention provides a method, apparatus, and electronic device for assessing motor function in Parkinson's disease, comprising: acquiring image data of a user performing a preset action, and scores for other items in Part III of the Unified Parkinson's Disease Rating Scale (UPRS), excluding limb rigidity and postural stability assessment items; the preset action being the action corresponding to Part III of the UPS; preprocessing the image data to obtain preprocessed image data; extracting the user's facial features and the location information of the user's body feature points from the preprocessed image data; inputting the facial features and the location information of the body feature points into a pre-trained machine learning model, and outputting the user's limb rigidity score and postural stability score; the machine learning model being pre-trained based on images carrying preset limb rigidity scores and preset postural stability scores; and determining the user's Part III score of the UPS based on the limb rigidity score, the postural stability score, and the scores of the other items, thereby determining the user's Parkinson's disease motor function assessment result based on the Part III score. This method alleviates the problem that the assessment process in the third part of MDS-UPDRS is mainly performed by assessors, and improves the assessment efficiency of motor function assessment for Parkinson's disease by directly scoring users' limb stiffness and postural stability through machine learning models.

[0017] Other features and advantages disclosed in this embodiment will be set forth in the following description, or some features and advantages may be inferred from the description or determined without doubt, or may be learned by practicing the techniques described above.

[0018] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0019] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating a method for assessing motor function in Parkinson's disease, provided in an embodiment of the present invention;

[0021] Figure 2 A flowchart illustrating another method for assessing motor function in Parkinson's disease provided in an embodiment of the present invention;

[0022] Figure 3A schematic diagram illustrating the kinematic basis of a motion signal provided in an embodiment of the present invention;

[0023] Figure 4 A schematic diagram illustrating the positional basis of a motion signal provided in an embodiment of the present invention;

[0024] Figure 5 A schematic diagram illustrating the angle basis of a motion signal provided in an embodiment of the present invention;

[0025] Figure 6 A schematic diagram of the vertical curve of the toe provided in an embodiment of the present invention;

[0026] Figure 7 A schematic diagram of a smooth signal of the toe in the vertical direction provided in an embodiment of the present invention;

[0027] Figure 8 This is a schematic diagram of an inter-point angle signal provided in an embodiment of the present invention;

[0028] Figure 9 A schematic diagram of an angle signal spectral peak provided in an embodiment of the present invention;

[0029] Figure 10 This is a schematic diagram of the structure of a motor function assessment device for Parkinson's disease provided in an embodiment of the present invention;

[0030] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0031] Icons: 11-Data acquisition module; 12-Image preprocessing module; 13-Feature extraction module; 14-Model evaluation module; 15-Evaluation result output module; 21-Memory; 22-Processor; 23-Bus; 24-Communication interface. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0033] In Part III of the MDS-UPDRS (Unified-Parkinson's Disease Rating Scale), the assessment of limb rigidity and postural stability requires trained assessors to physically touch the patient. For assessing limb rigidity, the assessor rotates the patient's neck, feels muscle tone in the limb joints, and assigns a score. For postural stability, the assessor performs a pull-back test, scoring the patient based on the number of steps they need to take to regain balance. During this process, patients may fall due to balance issues, so trained assessors must be prepared to catch them. Furthermore, because the assessment process in Part III of the MDS-UPDRS is primarily performed by assessors, current methods for assessing motor function in Parkinson's disease are inefficient.

[0034] Based on this, embodiments of the present invention provide a method, device, and electronic device for assessing motor function in Parkinson's disease. This method alleviates the problem that the assessment process in the third part of MDS-UPDRS is mainly performed by assessors, and directly scores the user's limb rigidity and postural stability using a machine learning model, thereby improving the assessment efficiency of motor function in Parkinson's disease. To facilitate understanding of the embodiments of the present invention, a detailed description of the method for assessing motor function in Parkinson's disease disclosed in the present invention will be provided first.

[0035] Example 1

[0036] This invention provides a method for assessing motor function in Parkinson's disease. Figure 1 This is a flowchart illustrating a method for assessing motor function in Parkinson's disease, provided in an embodiment of the present invention.

[0037] Depend on Figure 1 As can be seen, the above methods include:

[0038] Step S101: Obtain image data of the user completing the preset action, as well as the scores of other items in Part III of the Unified Parkinson's Disease Rating Scale, excluding the limb rigidity and postural stability assessment items; the preset action is the action corresponding to Part III of the Unified Parkinson's Disease Rating Scale.

[0039] Here, the image data is in RGB video format. The image data of the user's face was captured at 1080p@20fps; the image data of the other data besides the user's face was captured at 540p@20fps.

[0040] Step S102: Preprocess the above image data to obtain preprocessed image data.

[0041] Here, step S102 generally involves adjusting the brightness and contrast of the image data and performing noise reduction to improve the clarity of the image data.

[0042] Step S103: Extract the facial features of the user and the location information of the user's human body feature points from the preprocessed image data.

[0043] Here, the correspondence between facial emotions and facial muscles in the preprocessed image data is defined through a facial motion coding system, thereby extracting the facial features of the user in the preprocessed image data.

[0044] Furthermore, the location information of the user's human body feature points in the processed image data is extracted using the HRNet algorithm. Here, traditional techniques typically use OpenPose to extract human body points when estimating pose in images. However, OpenPose may result in inaccurate estimates of some joint-related movements when the patient is asked to straighten their arms. To mitigate this impact, this approach uses HRNet instead of OpenPose to improve the accuracy of the joint estimation.

[0045] Step S104: Input the location information of the above facial features and the above human body feature points into the pre-trained machine learning model, and output the user's limb stiffness score and posture stability score; the above machine learning model is pre-trained based on images carrying preset limb stiffness scores and preset posture stability scores.

[0046] Here, the machine learning model described above is built based on Light GBM.

[0047] Step S105: Based on the above limb stiffness score, the above postural stability score, and the scores of the above other items, determine the third part score of the above user's Unified Parkinson's Disease Rating Scale, so as to determine the user's Parkinson's disease motor function assessment result based on the above third part score.

[0048] This invention provides a method for assessing motor function in Parkinson's disease, comprising: acquiring image data of a user performing a preset action, and scores for other items in Part III of the Unified Parkinson's Disease Rating Scale (UPRS), excluding limb rigidity and postural stability assessment items; the preset action being the action corresponding to Part III of the UPS; preprocessing the image data to obtain preprocessed image data; extracting the user's facial features and the location information of the user's body feature points from the preprocessed image data; inputting the facial features and the location information of the body feature points into a pre-trained machine learning model, and outputting the user's limb rigidity score and postural stability score; the machine learning model being pre-trained based on images carrying preset limb rigidity scores and preset postural stability scores; and determining the user's Part III score of the UPS based on the limb rigidity score, the postural stability score, and the scores of the other items, thereby determining the user's motor function assessment result for Parkinson's disease based on the Part III score. This method alleviates the problem that the assessment process in the third part of MDS-UPDRS is mainly performed by assessors, and improves the assessment efficiency of motor function assessment for Parkinson's disease by directly scoring users' limb stiffness and postural stability through machine learning models.

[0049] Example 2

[0050] Another method for assessing motor function in Parkinson's disease is provided in this embodiment of the invention. Figure 2 This is a flowchart illustrating another method for assessing motor function in Parkinson's disease provided in an embodiment of the present invention.

[0051] Depend on Figure 2 As seen, the method includes:

[0052] Step S201: Obtain image data of the user completing the preset action, as well as the scores of other items in Part III of the Unified Parkinson's Disease Rating Scale, excluding the limb rigidity and postural stability assessment items; the preset action is the action corresponding to Part III of the Unified Parkinson's Disease Rating Scale.

[0053] Step S202: Preprocess the above image data to obtain preprocessed image data.

[0054] Step S203: Extract the facial features of the user and the location information of the user's human body feature points from the preprocessed image data.

[0055] Step S204: Input the location information of the above human feature points into the upper limb stiffness scoring model of the above machine learning model, and output the upper limb stiffness score.

[0056] Step S205: Input the location information of the above human feature points into the lower limb stiffness scoring model of the above machine learning model, and output the lower limb stiffness score.

[0057] Step S206: Input the above facial features and the location information of the above human body feature points into the neck stiffness scoring model of the above machine learning model, and output the neck stiffness score.

[0058] Step S207: Input the position information of the above human feature points into the posture stability scoring model of the above machine learning model, and output the posture stability score.

[0059] Step S208: Determine the user's limb stiffness score based on the above upper limb stiffness score, the above lower limb stiffness score, and the above neck stiffness score, and output the user's limb stiffness score.

[0060] Step S209: Based on the above limb stiffness score, the above postural stability score, and the scores of the above other items, determine the third part score of the above user's Unified Parkinson's Disease Rating Scale, so as to determine the user's Parkinson's disease motor function assessment result based on the above third part score.

[0061] In one embodiment, after step S203, the method includes the following step A1:

[0062] Step A1: Based on the location information of the human body feature points and the time data corresponding to the location information, determine the kinematic and signal characteristics of the user.

[0063] Furthermore, the step of inputting the aforementioned facial features and the location information of the aforementioned human body feature points into a pre-trained machine learning model and outputting the user's limb stiffness score and posture stability score includes: inputting the aforementioned facial features, the aforementioned kinematic features and the aforementioned signaling features into a pre-trained machine learning model and outputting the user's limb stiffness score and posture stability score.

[0064] Here, the aforementioned facial features are output by the preset OpenFace, including detailed changes in each motion unit, such as the size of the mouth opening and closing.

[0065] Furthermore, every motion signal has three fundamental bases: kinematic, positional, and angular. For ease of understanding, Figure 3 A schematic diagram illustrating the kinematic basis of a motion signal provided in an embodiment of the present invention; Figure 4 A schematic diagram illustrating the positional basis of a motion signal provided in an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the angle basis of a motion signal according to an embodiment of the present invention. (The above is a partial translation of the original text.) Figures 3 to 5 As observed, kinematic characteristics reflect the performance of the movement, including speed, amplitude, hesitation, and deceleration. Positional and angular characteristics reflect the position of the joint or the angle of its range of motion during the test, respectively.

[0066] Furthermore, step A1 above includes the following steps B1-B2:

[0067] Step B1: Based on the location information of the human body feature points and the time data corresponding to the location information, determine the change curve of the vertical coordinates of the location information of the feature points.

[0068] Step B2: Based on the above change curves, determine the kinematic and signal characteristics of the user.

[0069] Here, the steps for determining the kinematic characteristics of the user based on the aforementioned change curves include the following steps C1-C4:

[0070] Step C1: Smooth the above-mentioned curve using the smooth spline method to obtain a smooth curve signal.

[0071] Step C2: Extract the frequency and amplitude of the smooth curve signal above.

[0072] For ease of understanding, Figure 6 A schematic diagram of the vertical curve of the toe provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of a smooth signal of the toe in the vertical direction provided by an embodiment of the present invention. Wherein, Figure 6 In this context, "Signal of toe tip on y-axis" represents the curve of the toe tip in the vertical direction, and "Distance from the ground" represents the above. Figure 6 The ordinate represents the vertical distance of change, and Time(s) represents the above. Figure 6 The horizontal axis represents time, in seconds. Figure 7 In this context, "Smoothed Signal of toe tip on y-axis" refers to the smooth signal of the toe tip in the vertical direction, and "Distance from the ground" refers to the above. Figure 7 The ordinate represents the vertical distance change, and Time(s) represents the above. Figure 6 The horizontal axis represents time, in seconds.

[0073] Step C3: Calculate the maximum, minimum, mean, and standard deviation of the above frequencies and amplitudes respectively.

[0074] Step C4: Determine the kinematic characteristics of the user based on the above frequency and the maximum, minimum, mean and standard deviation of the above amplitude.

[0075] Furthermore, the steps for determining the signal characteristics of the aforementioned user based on the above-mentioned change curves include steps D1-D4:

[0076] Step D1: Calculate the maximum value, minimum value, peak value, quantile, root mean square, absolute mean, standard deviation, coefficient of variation, kurtosis coefficient, and skewness coefficient of the above variation curve, and perform Fourier transform on the above variation curve to obtain the first preprocessed data; and perform wavelet transform on the above variation curve to obtain the second preprocessed data.

[0077] Step D2: Calculate the sum of amplitudes corresponding to different frequencies in the first preprocessed data to obtain the amplitude sum; and filter the coefficient values ​​of the second preprocessed data.

[0078] Step D3: Calculate the absolute value of the above coefficients, and sum the absolute values ​​of the above coefficients to obtain the sum of absolute values.

[0079] Step D4: The maximum value, minimum value, peak value, quantile, root mean square, absolute mean, standard deviation, coefficient of variation, kurtosis coefficient, skewness coefficient, amplitude, and absolute value of the above-mentioned variation curves are determined as the signal characteristics of the above-mentioned user.

[0080] For ease of understanding, Figure 8 This is a schematic diagram of an inter-point angle signal provided in an embodiment of the present invention; Figure 9 This is a schematic diagram of an angle signal spectrum peak provided in an embodiment of the present invention. Wherein, "Signal of angle between joints" represents a schematic diagram of the angle signal between points, and "Angle" represents the aforementioned... Figure 8 The vertical axis represents the angle, and Time(s) represents the above. Figure 8 The horizontal axis represents time, in seconds. The Spectrummap of the angle signal is a schematic diagram of the spectral peaks of the angle signal, and Amplitude represents the above... Figure 9 The vertical axis represents the amplitude, and Frequency (Hz) represents the above. Figure 9 The horizontal axis represents frequency, with the unit being Hertz.

[0081] Furthermore, the aforementioned machine learning model is constructed based on Light GBM. Before inputting the location information of the facial features and human feature points into the pre-trained machine learning model, the method includes the following steps E1-E6:

[0082] Step E1: Input the above facial features, kinematic features and signaling features into the preset initial machine learning model and output information gain.

[0083] Step E2: Sort the above information gains to obtain the sorted information gains;

[0084] Step E3: Group the sorted information gain according to preset parameters to obtain multiple feature groups.

[0085] Step E4: Train the above initial machine learning models using the target facial features, target kinematic features, and target signaling features corresponding to the above multiple feature groups to obtain multiple trained initial machine learning models.

[0086] Step E5: Use leave-one-out cross-validation to evaluate the performance of the multiple trained initial machine learning models and obtain multiple performance values.

[0087] Step E6: The initial machine learning model whose performance value reaches the preset threshold is identified as the pre-trained machine learning model.

[0088] This invention provides a method for assessing motor function in Parkinson's disease, comprising: acquiring image data of a user performing a preset action, and scores for other items in Part III of the Unified Parkinson's Disease Rating Scale (UPRS), excluding limb rigidity and postural stability assessment items; the preset action being the action corresponding to Part III of the UPS; preprocessing the image data to obtain preprocessed image data; extracting the user's facial features and the location information of the user's body feature points from the preprocessed image data; inputting the location information of the body feature points into the upper limb rigidity scoring model of the machine learning model to output an upper limb rigidity score; and inputting the location information of the body feature points into the lower limb rigidity scoring model of the machine learning model. The method involves several steps: First, it calculates the user's upper limb stiffness score. Then, it inputs the location information of the facial features and body feature points into the machine learning model's neck stiffness scoring model, outputting a neck stiffness score. Next, it inputs the location information of the body feature points into the machine learning model's posture stability scoring model, outputting a posture stability score. Finally, it determines the user's limb stiffness score based on the upper limb stiffness score, lower limb stiffness score, and neck stiffness score, and outputs the user's limb stiffness score. Based on the limb stiffness score, posture stability score, and scores from other items, it determines the user's Part III score on the Unified Parkinson's Disease Rating Scale (UPDRS), and uses this Part III score to determine the user's Parkinson's disease motor function assessment result. This method alleviates the problem that the assessment process of Part III of the MDS-UPDRS is mainly performed by the assessor, and by directly scoring the user's upper limb stiffness score, lower limb stiffness score, neck stiffness score, and posture stability using a machine learning model, it further improves the assessment efficiency of Parkinson's disease motor function.

[0089] Example 3

[0090] Figure 10 This is a schematic diagram of a motor function assessment device for Parkinson's disease provided in an embodiment of the present invention.

[0091] Depend on Figure 10 As observed, the Parkinson's disease motor function assessment device includes:

[0092] The data acquisition module 11 is used to acquire image data of the user completing the preset action, as well as the scores of other items in Part III of the Unified Parkinson's Disease Rating Scale, excluding the limb rigidity and postural stability assessment items; the preset action is the action corresponding to Part III of the Unified Parkinson's Disease Rating Scale.

[0093] The image preprocessing module 12 is used to preprocess the above image data to obtain preprocessed image data.

[0094] The feature extraction module 13 is used to extract the facial features of the user and the location information of the user's human body feature points from the preprocessed image data.

[0095] The model evaluation module 14 is used to input the location information of the above-mentioned facial features and the above-mentioned human body feature points into the pre-trained machine learning model and output the user's limb stiffness score and posture stability score; the above-mentioned machine learning model is pre-trained based on images carrying preset limb stiffness scores and preset posture stability scores.

[0096] The assessment result output module 15 is used to determine the third part score of the user's Unified Parkinson's Disease Rating Scale based on the above-mentioned limb stiffness score, the above-mentioned postural stability score, and the scores of the above-mentioned other items, so as to determine the user's Parkinson's disease motor function assessment result based on the above-mentioned third part score.

[0097] The data acquisition module 11, image preprocessing module 12, feature extraction module 13, model evaluation module 14, and evaluation result output module 15 are connected in sequence.

[0098] In one embodiment, the machine learning model includes: an upper limb stiffness scoring model, a lower limb stiffness scoring model, a neck stiffness scoring model, and a posture stability scoring model; the evaluation result output module 15 is further configured to input the location information of the human feature points into the upper limb stiffness scoring model and output an upper limb stiffness score; input the location information of the human feature points into the lower limb stiffness scoring model and output a lower limb stiffness score; input the location information of the facial features and the human feature points into the neck stiffness scoring model and output a neck stiffness score; input the location information of the human feature points into the posture stability scoring model and output a posture stability score; determine the user's limb stiffness score based on the upper limb stiffness score, the lower limb stiffness score, and the neck stiffness score, and output the user's limb stiffness score.

[0099] In one embodiment, the feature extraction module 13 is further configured to determine the kinematic features and signal characteristics of the user based on the location information of the human feature points and the time data corresponding to the location information; the evaluation result output module 15 is further configured to input the facial features, the kinematic features and the signal characteristics into a pre-trained machine learning model, and output the user's limb stiffness score and posture stability score.

[0100] In one embodiment, the feature extraction module 13 is further configured to determine the change curve of the vertical coordinate of the position information of the human feature points based on the position information of the human feature points and the time data corresponding to the position information; and to determine the kinematic features and signal characteristics of the user based on the change curve.

[0101] In one embodiment, the feature extraction module 13 is further configured to smooth the aforementioned change curve using a smooth spline method to obtain a smooth curve signal; extract the frequency and amplitude of the smooth curve signal; calculate the maximum, minimum, mean, and standard deviation of the frequency and amplitude respectively; and determine the kinematic characteristics of the user based on the maximum, minimum, mean, and standard deviation of the frequency and amplitude.

[0102] In one embodiment, the feature extraction module 13 is further configured to calculate the maximum value, minimum value, peak value, quantile, root mean square, absolute mean, standard deviation, coefficient of variation, kurtosis coefficient, and skewness coefficient of the aforementioned change curve, and perform Fourier transform on the aforementioned change curve to obtain first preprocessed data; and perform wavelet transform on the aforementioned change curve to obtain second preprocessed data; calculate the sum of amplitudes corresponding to different frequencies in the aforementioned first preprocessed data to obtain amplitude sum; and filter the coefficient values ​​of the aforementioned second preprocessed data; calculate the absolute values ​​of the aforementioned coefficient values, and perform summation operation on the absolute values ​​of the aforementioned coefficient values ​​to obtain absolute value sum; and determine the maximum value, minimum value, peak value, quantile, root mean square, absolute mean, standard deviation, coefficient of variation, kurtosis coefficient, skewness coefficient, amplitude sum, and absolute value sum of the aforementioned change curve as the signal characteristics of the aforementioned user.

[0103] In one embodiment, the model evaluation module 14 is further configured to input the aforementioned facial features, kinematic features, and signaling features into a preset initial machine learning model and output information gain; sort the aforementioned information gains to obtain sorted information gains; group the sorted information gains according to preset parameters to obtain multiple feature groups; train the aforementioned initial machine learning model using the target facial features, target kinematic features, and target signaling features corresponding to the aforementioned multiple feature groups respectively to obtain multiple trained initial machine learning models; evaluate the performance of the aforementioned multiple trained initial machine learning models using leave-one-out cross-validation to obtain multiple performance values; and determine the initial machine learning model whose performance value reaches a preset threshold as the aforementioned pre-trained machine learning model.

[0104] The Parkinson's disease motor function assessment device provided in this embodiment of the invention has the same technical features as the Parkinson's disease motor function assessment method provided in the above embodiments, and therefore can solve the same technical problems and achieve the same technical effects. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0105] Example 4

[0106] This embodiment provides an electronic device, including a processor and a memory. The memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the steps of a method for assessing motor function in Parkinson's disease.

[0107] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of a method for assessing motor function in Parkinson's disease.

[0108] See Figure 11 The diagram shows the structure of an electronic device, which includes a memory 21 and a processor 22. The memory 21 stores a computer program that can run on the processor 22. When the processor executes the computer program, it implements the steps provided by the above-mentioned Parkinson's disease motor function assessment method.

[0109] like Figure 11 As shown, the device also includes a bus 23 and a communication interface 24. The processor 22, the communication interface 24 and the memory 21 are connected via the bus 23. The processor 22 is used to execute executable models, such as computer programs, stored in the memory 21.

[0110] The memory 21 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 24 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0111] Bus 23 can be an ISA bus, PCI bus, or EISA bus, etc. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 2 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0112] The memory 21 stores the program, and the processor 22 executes the program after receiving the execution instruction. The method performed by the dual Parkinson's disease motor function assessment device disclosed in any of the foregoing embodiments of the present invention can be applied to the processor 22, or implemented by the processor 22. The processor 22 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 22. The processor 22 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this invention can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software models in the decoding processor. The software model can reside in a random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 21, and processor 22 reads information from memory 21 and, in conjunction with its hardware, completes the steps of the above method.

[0113] Furthermore, this embodiment of the invention also provides a machine-readable storage medium storing machine-executable instructions. When these machine-executable instructions are invoked and executed by the processor 22, they cause the processor 22 to implement the aforementioned Parkinson's disease motor function assessment method.

[0114] The electronic devices and computer-readable storage media provided in the embodiments of the present invention have the same technical features, so they can also solve the same technical problems and achieve the same technical effects.

[0115] Furthermore, in the description of the embodiments of the present invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention based on the specific circumstances.

Claims

1. A method for assessing motor function in Parkinson's disease, characterized in that, include: Acquire image data of users performing preset actions, as well as scores for other items in Part III of the Unified Parkinson's Disease Rating Scale, excluding the limb rigidity and postural stability assessment items; The preset action is the action corresponding to the third part of the Unified Parkinson's Disease Rating Scale; The image data is preprocessed to obtain preprocessed image data; Extract the user's facial features and the location information of the user's human body feature points from the preprocessed image data; Based on the location information of the human body feature points and the time data corresponding to the location information, the kinematic and signal characteristics of the user are determined; The facial features and the location information of the human body feature points are input into a pre-trained machine learning model, which outputs the user's limb stiffness score and posture stability score. The facial features contain detailed changes in each action unit, and the machine learning model is pre-trained based on images carrying preset limb stiffness scores and preset posture stability scores. Based on the limb stiffness score, the postural stability score, and the scores of the other items, the user's third part score of the Unified Parkinson's Disease Rating Scale is determined, and the user's Parkinson's disease motor function assessment result is determined based on the third part score. The steps of inputting the facial features and the location information of the human body feature points into a pre-trained machine learning model to output the user's limb stiffness score and posture stability score include: The facial features, kinematic features, and signaling features are input into a pre-trained machine learning model, which outputs the user's limb stiffness score and posture stability score. The machine learning models include: an upper limb stiffness scoring model, a lower limb stiffness scoring model, a neck stiffness scoring model, and a posture stability scoring model. The steps of inputting the facial features and the location information of the human body feature points into a pre-trained machine learning model, and outputting the user's limb stiffness score and posture stability score, include: Input the location information of the human feature points into the upper limb stiffness scoring model and output the upper limb stiffness score; The location information of the human feature points is input into the lower limb stiffness scoring model, and the lower limb stiffness score is output. The location information of the facial features and the human body feature points is input into the neck stiffness scoring model, and the neck stiffness score is output. The position information of the human feature points is input into the posture stability scoring model, and the posture stability score is output. The user's limb stiffness score is determined based on the upper limb stiffness score, the lower limb stiffness score, and the neck stiffness score, and then the user's limb stiffness score is output.

2. The method for assessing motor function in Parkinson's disease according to claim 1, characterized in that, The steps of determining the user's kinematic and signal characteristics based on the location information of the human feature points and the corresponding time data include: Based on the location information of the human feature points and the time data corresponding to the location information, determine the change curve of the vertical coordinate of the location information of the feature points; Based on the change curve, the user's kinematic and signal characteristics are determined.

3. The method for assessing motor function in Parkinson's disease according to claim 2, characterized in that, The step of determining the user's kinematic characteristics based on the change curve includes: The changing curve is smoothed using the smooth spline method to obtain a smoothed curve signal; Extract the frequency and amplitude of the smooth curve signal; Calculate the maximum, minimum, mean, and standard deviation of the frequency and the amplitude, respectively; The user's kinematic characteristics are determined based on the maximum, minimum, mean, and standard deviation of the frequency and amplitude.

4. The method for assessing motor function in Parkinson's disease according to claim 2, characterized in that, The step of determining the user's signal characteristics based on the change curve includes: Calculate the maximum value, minimum value, peak value, quantile, root mean square, absolute mean, standard deviation, coefficient of variation, kurtosis coefficient, and skewness coefficient of the variation curve, and perform Fourier transform on the variation curve to obtain the first preprocessed data; and perform wavelet transform on the variation curve to obtain the second preprocessed data. Calculate the sum of amplitudes corresponding to different frequencies in the first preprocessed data to obtain the amplitude sum; and filter the coefficient values ​​of the second preprocessed data. Calculate the absolute value of the coefficient value, and sum the absolute values ​​of the coefficient values ​​to obtain the sum of absolute values; The maximum value, minimum value, peak value, quantile, root mean square, absolute mean, standard deviation, coefficient of variation, kurtosis coefficient, skewness coefficient, amplitude, and absolute value of the variation curve are determined as the signal characteristics of the user.

5. The method for assessing motor function in Parkinson's disease according to claim 4, characterized in that, The machine learning model was built based on LightGBM.

6. The method for assessing motor function in Parkinson's disease according to claim 5, characterized in that, Before the step of inputting the facial features and the location information of the human body feature points into the pre-trained machine learning model, the method includes: The facial features, kinematic features, and signaling features are input into a preset initial machine learning model, and information gain is output. The information gains are sorted to obtain the sorted information gains; The sorted information gain is grouped according to preset parameters to obtain multiple feature groups; The initial machine learning model is trained by the target facial features, target kinematic features and target signaling features corresponding to the multiple feature groups, respectively, to obtain multiple trained initial machine learning models; Leave-one-out cross-validation was used to evaluate the performance of the multiple trained initial machine learning models, resulting in multiple performance values; The initial machine learning model whose performance value reaches a preset threshold is identified as the pre-trained machine learning model.

7. A device for assessing motor function in Parkinson's disease, characterized in that, include: The data acquisition module is used to acquire image data of the user completing preset actions, as well as scores of other items in Part III of the Unified Parkinson's Disease Rating Scale, excluding the limb rigidity and postural stability assessment items. The preset action is the action corresponding to the third part of the Unified Parkinson's Disease Rating Scale; An image preprocessing module is used to preprocess the image data to obtain preprocessed image data; The feature extraction module is used to extract the user's facial features and the location information of the user's human body feature points from the preprocessed image data; The feature extraction module is also used to determine the user's kinematic and signal characteristics based on the location information of the human feature points and the time data corresponding to the location information; The model evaluation module is used to input the facial features and the positional information of the human body feature points into a pre-trained machine learning model and output the user's limb stiffness score and posture stability score; the facial features include the detailed changes of each action unit, and the machine learning model is pre-trained based on images carrying preset limb stiffness scores and preset posture stability scores. The assessment result output module is used to determine the third part score of the user's Unified Parkinson's Disease Rating Scale based on the limb stiffness score, the postural stability score, and the scores of the other items, so as to determine the user's Parkinson's disease motor function assessment result based on the third part score. The step of inputting the facial features and the location information of the human body feature points into a pre-trained machine learning model and outputting the user's limb stiffness score and posture stability score includes: inputting the facial features, the kinematic features and the signaling features into a pre-trained machine learning model and outputting the user's limb stiffness score and posture stability score. The machine learning model includes: an upper limb stiffness scoring model, a lower limb stiffness scoring model, a neck stiffness scoring model, and a posture stability scoring model. The step of inputting the positional information of the facial features and the human body feature points into the pre-trained machine learning model and outputting the user's limb stiffness score and posture stability score includes: inputting the positional information of the human body feature points into the upper limb stiffness scoring model and outputting an upper limb stiffness score; inputting the positional information of the human body feature points into the lower limb stiffness scoring model and outputting a lower limb stiffness score; inputting the positional information of the facial features and the human body feature points into the neck stiffness scoring model and outputting a neck stiffness score; inputting the positional information of the human body feature points into the posture stability scoring model and outputting a posture stability score; determining the user's limb stiffness score based on the upper limb stiffness score, lower limb stiffness score, and neck stiffness score, and outputting the user's limb stiffness score.

8. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing computer-executable instructions that can be executed by the processor, the processor executing the computer-executable instructions to implement the Parkinson's disease motor function assessment method according to any one of claims 1 to 6.

Images