An automatic rehabilitation assessment method and assessment terminal based on LSTM

Abstract

The present invention provides an automatic rehabilitation assessment method and assessment terminal based on LSTM, which autonomously selects a corresponding scale according to the patient's condition and the affected part, and uses an inertial sensor to obtain the patient's motion data; multiple patients with the same condition are used to repeat multiple times to obtain different data samples, extract the value changes at each time point, and sequentially process the data samples through an SC-LSTM layer, a self-attention mechanism layer, a fully connected layer, and a soft-max layer, and then use a cross-entropy loss function for verification; when the rehabilitation assessment algorithm is verified to be effective, the patient is automatically assessed for rehabilitation using the rehabilitation assessment algorithm; the collected interference data is filtered out; the score of a single assessment action is calculated; the total score of the rehabilitation assessment is checked for score rationality, and the score after passing the score rationality check is output, and if the score rationality check is not passed, an error is displayed. The present invention reduces the impact of subjective factors of rehabilitation therapists on the rehabilitation assessment results.

Description

Background Art

[0002] "Stroke", also known as "cerebrovascular accident", is a medical problem and one of the three most common causes of death. Among them, motor dysfunction is one of the common sequelae of stroke, which seriously affects the normal life of patients. Systematic and reasonable rehabilitation treatment for stroke patients can help patients regain the ability to live independently.

[0003] In rehabilitation treatment, because the lesion part, severity and degree of movement disorder of stroke patients are different, quantifying the patient's abnormal movement is the basis for diagnosis and further formulation of treatment plans. Nowadays, the assessment of motor dysfunction in stroke patients generally adopts a scale-based assessment method: the patient makes corresponding movements according to the content of the scale, and the rehabilitation therapist gives a score based on the patient's completion.

[0004] Although this type of assessment method is widely used in clinical practice, there are still some problems: first, the scale contains many assessment items, the assessment time is long, and professional rehabilitation therapists need to participate in real time, which is not conducive to implementation in community and family environments; second, the existing stroke rehabilitation assessment is mainly performed manually by rehabilitation therapists, which is easily affected by the subjective factors of rehabilitation therapists; finally, different rehabilitation therapists may give different assessment results for the same patient's condition, which is not conducive to the formulation of rehabilitation treatment plans. Summary of the invention

[0005] In order to overcome the deficiencies in the above-mentioned prior art, the present invention provides an automatic rehabilitation assessment method based on LSTM, which can analyze the patient's motion data and provide assessment scores and assessment results for the patient's reference, thereby helping the patient to recover as soon as possible.

[0006] The LSTM-based automatic rehabilitation assessment method includes:

[0007] Step 1: Configure the corresponding scale according to the patient's condition and the affected part, and use N inertial sensors to select the patient's motion data during the time window T for rehabilitation assessment;

[0008] The motion data is calculated as follows:

[0009] x t It is the motion data measured by N inertial sensors at time point t; according to the scores of the motion data, a data set is created for algorithm training to achieve automatic rehabilitation assessment.

[0010] Step 2: Use multiple patients with the same condition to repeat multiple times to obtain different data samples and make the data samples into a data set And divide the data set D into a training set S and a test set Te;

[0011] Step 3: Extract the value changes at each time point:

[0012] Where Δt is the sampling time interval of the inertial sensor, x' wt is the slope of the straight line passing through the data measured at time point t and the data measured at the previous time point t-1;

[0013] Step 4: x wt With x' wt Input to the SC-LSTM layer;

[0014] Step 5: Input the data passing through the SC-LSTM layer into the self-attention mechanism layer;

[0015] Step 6: Input the highly abstracted data through the self-attention mechanism layer into the fully connected layer;

[0016] Step 7: Input the data passing through the fully connected layer into the soft-max layer to obtain the probability values ​​of different categories corresponding to the time series data;

[0017] Step 8: Use the Adam algorithm and the cross entropy loss function to optimize until the evaluation algorithm converges, and then use the test set Te to verify the evaluation algorithm;

[0018] Step 9: When the rehabilitation assessment algorithm is verified to be effective, the patient is automatically assessed for rehabilitation using the rehabilitation assessment algorithm, and the rehabilitation scale used in step 1 is used. The inertial sensor is placed at the position where the inertial sensor is placed on the patient in step 1, so that the output data dimension is the same as the x in step 1. t The dimensions are the same;

[0019] Step 10, filtering out the interference data collected in step 9;

[0020] Step 11: Analyze the original data after filtering out the interference data in step 10 using the trained rehabilitation assessment algorithm, and calculate the score of a single assessment action;

[0021] Step 12: Calculate the total rehabilitation assessment score obtained in step 11, perform a score rationality check on the total rehabilitation assessment score, and output the score that passes the score rationality check. If the score rationality check fails, an error is displayed.

[0022] The present invention also provides an assessment terminal, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the LSTM-based automatic rehabilitation assessment method when executing the program.

[0023] It can be seen from the above technical solutions that the present invention has the following advantages:

[0024] The LSTM-based automatic rehabilitation assessment method provided by the present invention reduces the influence of subjective factors of rehabilitation therapists on the rehabilitation assessment results. Moreover, the rehabilitation assessment method of the present invention independently selects the corresponding scale according to the patient's condition and the affected part, uses an inertial sensor to obtain the patient's motion data, improves the accuracy of rehabilitation data collection, and assesses the patient's rehabilitation part in a targeted manner. During the assessment process, the numerical changes at each time point are extracted, and the data samples are processed in sequence through the SC-LSTM layer, the self-attention mechanism layer, the fully connected layer, and the soft-max layer, and then the cross entropy loss function is used for verification, and the collected interference data is filtered out to ensure the accuracy of the data, and finally the score of a single assessment action is calculated; the total score of the rehabilitation assessment is checked for score rationality, and the patient's rehabilitation status is given, so that the patient and his family can understand the rehabilitation process, provide data support for the subsequent rehabilitation process, and improve the efficiency of the patient's rehabilitation.

[0025] The automatic rehabilitation assessment method based on LSTM of the present invention can allow patients to make corresponding actions according to the content of the scale, process the action data, and obtain a rehabilitation score. The automatic rehabilitation assessment method of the present invention does not require the real-time participation of professional rehabilitation therapists and can be implemented in community and family environments. The assessment of data does not require manual assessment by rehabilitation therapists, which reduces the influence of subjective factors of rehabilitation therapists. It avoids the problem that different rehabilitation therapists may give different assessment results for the same patient's condition, which is not conducive to the formulation of rehabilitation treatment plans. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the description will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 This is a flow chart of the LSTM-based automatic rehabilitation assessment method.

[0028] Figure 2 This is the training parameter process of the automatic rehabilitation assessment method in the embodiment of the present invention.

[0029] Figure 3 This is a flow chart of filtering out interference data in the present invention.

[0030] Figure 4 The flowchart of the score rationality check in the example of the present invention is shown in FIG.

[0031] Figure 54 is a scoring flow chart of the automatic rehabilitation assessment system in an embodiment of the present invention. DETAILED DESCRIPTION

[0032] like Figure 1 The flowchart of the preferred embodiment of the automatic rehabilitation assessment method based on LSTM of the present invention is shown. The automatic rehabilitation assessment method based on LSTM is applied to one or more assessment terminals, which are devices that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and whose hardware includes but is not limited to microprocessors, application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), digital signal processors (DSP), embedded devices, etc.

[0033] The assessment terminal can be any electronic product that can interact with the user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), an interactive network television (IPTV), a smart wearable device, etc.

[0034] The assessment terminal may also include network equipment and / or user equipment, wherein the network equipment includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud consisting of a large number of hosts or network servers based on cloud computing.

[0035] The network where the assessment terminal is located includes but is not limited to the Internet, wide area network, metropolitan area network, local area network, virtual private network (VPN), etc.

[0036] The following will be combined Figures 1 to 5 The automatic rehabilitation assessment method based on LSTM of the present invention is described in detail. The method can allow patients to make corresponding actions according to the content of the scale, process the action data, and obtain a rehabilitation score. The automatic rehabilitation assessment method of the present invention does not require the real-time participation of professional rehabilitation therapists and can be implemented in community and family environments. The assessment of the data does not require manual assessment by rehabilitation therapists, which reduces the influence of subjective factors of rehabilitation therapists. It avoids the problem that different rehabilitation therapists may give different assessment results for the same patient's condition, which is not conducive to the formulation of rehabilitation treatment plans.

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] See also Figures 1 to 5 FIG. 1 is a flowchart of an automatic rehabilitation assessment method based on LSTM in a specific embodiment, and the method includes:

[0039] Step 1: select the corresponding scale according to the patient's condition and the affected part, and use N inertial sensors to select the patient's motion data during the time window T for rehabilitation assessment;

[0040] The motion data is calculated as follows:

[0041] x t is the motion data measured by N inertial sensors at time t. t The dimension is selected by the output of the inertial sensor used, and can be adaptive to the output of the selected inertial sensor. T is the longest time for the patient to complete the rehabilitation assessment action. For the motion data whose completion time does not meet T, the cubic spline interpolation algorithm is used to make the data meet the conditions.

[0042] Among them, the inertial sensor can be placed on the patient, and the specific location can be determined by the location where the patient needs to be rehabilitated, or it can be determined by the scale used, and the scale can be selected independently according to needs. The scale can be set with different rehabilitation rules and requirements. For example, the rehabilitation of both arms and legs can set the scale for the diseased arm and leg.

[0043] Step 2: Use multiple patients with the same condition to repeat multiple times to obtain different data samples and make the data samples into a data set And divide the data set D into a training set S and a test set Te;

[0044] Step 3: Extract the value changes at each time point:

[0045] Where Δt is the sampling time interval of the inertial sensor, x' wt is the slope of the straight line passing through the data measured at time point t and the data measured at the previous time point t-1;

[0046] Step 4: x wt With x' wt Input to the SC-LSTM layer;

[0047] Specifically, the SC-LSTM layer processing methods include:

[0048] Forget Gate∫ t =σ[W fx x ft +W fh h t-1 +lanh(W' fx x' ft )+b f ],

[0049] where x ft is the input value at the current time point, h t-1 is the output value at the previous time point, W fx For the Forgotten Gate ft The weight, W fh For the Forgotten Gate t-1 The weight, W' fx For the Forgotten Gate x' ft The weight of b i is the bias of the forget gate,

[0050] Input Gate i t =σ[W ix x it +W ih h t-1 +tanh(W' ix x' it )+b i ], where x it is the input value at this time point, h t-1 is the output value at the previous time point, W ix is the input gate x it The weight, W Lh h is the input gate t-1 The weight, W' ix is the input gate x' it The weight of b l is the bias of the input gate,

[0051] Candidate cell state g t =tanh[W gx x gt +W gh h t 1 +tanh(W' gx x' gt )+h g ],

[0052] where x gt is the input value at the current time point, h t-1is the output value at the previous time point, W gx x is the candidate cell state gt The weight, W gh h is the candidate cell state t-1 The weight, W gx x' in the candidate cell state gt The weight of b g is the bias of the candidate cell state;

[0053] New cell state C t =f t ⊙C t-1 +i t ⊙g t , where C t-1 is the cell state at the previous time point, ⊙ represents the multiplication of the corresponding elements of the matrix;

[0054] Output gate o t =σ[W ox x ot +W oh h t-1 +tanh(W' ox x' oc )+b o ], where x ot is the input value at the current time point, h t-1 is the output value at the previous time point, W ox is the output gate x ot The weight, W oh h is the output gate t-1 The weight, W' ox is x' in the output gate ot The weight of b o is the bias of the output gate,

[0055] The final output is h t =o t ⊙tanh C t , where ⊙ represents the multiplication of corresponding elements of the matrix;

[0056] Get the continuous output of the SC-LSTM layer

[0057] Step 5: Input the data passing through the SC-LSTM layer into the self-attention mechanism layer;

[0058] For the self-attention mechanism layer in step 5,

[0059] The output of the self-attention layer

[0060] Where Q = W qH, K = W k H, V = W v H, D represent the dimensions of the SC-LSTM output, softmax represents converting the output value into a probability distribution in the range [0, 1] and summing to 1, × represents the cross product of the matrix, W q , W k and W v Represents a trainable transformation matrix, and ∑ represents the addition of corresponding elements in each row of the matrix.

[0061] Step 6: Input the highly abstracted data through the self-attention mechanism layer into the fully connected layer;

[0062] The fully connected layer processing methods include:

[0063] The output of the fully connected layer F = W F ×A+b F , where W F is the weight of the fully connected layer, b F is the bias of the fully connected layer, and × represents the cross product of the matrix.

[0064] Step 7: Input the data passing through the fully connected layer into the soft-max layer to obtain the probability values ​​of different categories corresponding to the time series data;

[0065] The soft-max layer processing method in step 7 includes: the output of the soft-max layer ,in is the weight of the soft-max layer, is the bias of the soft-max layer, represents the cross product of matrices, Indicates that the output value is converted to the range The probability distribution of sum is 1, and normalization ensures that the sum is 1, are the probability values ​​of different categories.

[0066] Step 8: Use the Adam algorithm and the cross entropy loss function to optimize until the evaluation algorithm converges, and then use the test set Te to verify the evaluation algorithm;

[0067] Cross entropy loss function J = -∑ylogy * , where y is the true category.

[0068] Step 9: When the rehabilitation assessment algorithm is verified to be effective, the patient is automatically assessed for rehabilitation using the rehabilitation assessment algorithm, and the rehabilitation scale used in step 1 is used. The inertial sensor is placed at the position where the inertial sensor is placed on the patient in step 1, so that the output data dimension is the same as the x in step 1. t The dimensions are the same;

[0069] Step 10, filtering out the interference data collected in step 9;

[0070] Specifically, the rules for filtering out interference data in step 2 and step 10 include:

[0071] Eliminate differences caused by different patient heights in Unit: m; represents the distance between the wearing position of N inertial sensors and the ground.

[0072] The original data after eliminating the height difference of patients is normalized to Where min is the minimum value in the original data after eliminating the difference in patient height, and max is the maximum value in the original data after eliminating the difference in patient height;

[0073] The original data after eliminating the height difference of patients is filtered to obtain the data after filtering out the interference data. in

[0074] Step 11: Analyze the original data after filtering out the interference data in step 10 using the trained rehabilitation assessment algorithm, and calculate the score of a single assessment action;

[0075] Step 12: Calculate the total rehabilitation assessment score obtained in step 11, perform a score rationality check on the total rehabilitation assessment score, and output the score that passes the score rationality check. If the score rationality check fails, an error is displayed.

[0076] The score rationality check in step 12 is based on the following rules:

[0077] Compare the total score of this assessment t2 =score t2 With full score y score =Score size;

[0078] If y t2 ≤y score , then find the total score y for this assessment t2 =score t2 To the scale full score straight line y score = Score distance dis t2 =score-score t2 ;

[0079] If dis t2 ≤dis t1 , then the score of this assessment is determined to be valid and output, where dis t1 The total score of the last assessment and the full score of the scalescore =The distance between scores.

[0080] The LSTM-based automatic rehabilitation assessment method provided by the present invention reduces the influence of subjective factors of rehabilitation therapists on the rehabilitation assessment results. Moreover, the rehabilitation assessment method of the present invention independently selects the corresponding scale according to the patient's condition and the affected part, uses an inertial sensor to obtain the patient's motion data, improves the accuracy of rehabilitation data collection, and assesses the patient's rehabilitation part in a targeted manner. During the assessment process, the numerical changes at each time point are extracted, and the data samples are processed in sequence through the SC-LSTM layer, the self-attention mechanism layer, the fully connected layer, and the soft-max layer, and then the cross entropy loss function is used for verification, and the collected interference data is filtered out to ensure the accuracy of the data, and finally the score of a single assessment action is calculated; the total score of the rehabilitation assessment is checked for score rationality, and the patient's rehabilitation status is given, so that the patient and his family can understand the rehabilitation process, provide data support for the subsequent rehabilitation process, and improve the efficiency of the patient's rehabilitation.

[0081] The LSTM-based automatic rehabilitation assessment method provided by the present invention is a combination of the units and algorithm steps of each example described in the embodiments disclosed herein, and can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0082] Those skilled in the art will appreciate that various aspects of the LSTM-based automatic rehabilitation assessment method provided by the present invention can be implemented as a system, method or program product. Therefore, various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software, which can be collectively referred to as "circuit", "module" or "system" here.

[0083] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An automatic rehabilitation assessment method based on LSTM, characterized in that: Methods include: Step 1: Configure the corresponding scale according to the patient's condition and the affected part, and use N inertial sensors to select the patient's motion data during the time window T for rehabilitation assessment; The motion data is calculated as follows: ; The motion data measured by N inertial sensors at time point t is used to create a data set for algorithm training based on the scores of the motion data to achieve automatic rehabilitation assessment; Step 2: Use multiple patients with the same condition to repeat multiple times to obtain different data samples and make the data samples into a data set , and divide the data set D into a training set S and a test set Te; Step 3: Extract the value changes at each time point: , in is the sampling time interval of the inertial sensor, is the slope of the straight line passing through the data measured at time point t and the data measured at the previous time point t-1; Step 4: and Input to the SC-LSTM layer; Step 5: Input the data passing through the SC-LSTM layer into the self-attention mechanism layer; The self-attention mechanism layer consists of: The output of the self-attention layer , in , , , represents the dimension of SC-LSTM output, Indicates that the output value is converted to the range The probability distribution of sum is 1, represents the cross product of matrices, , and represents a trainable transformation matrix, It means adding the corresponding elements of each row of the matrix; Step 6: Input the data that has been highly abstracted by the self-attention mechanism layer into the fully connected layer; Step 7: Input the data passing through the fully connected layer into the soft-max layer to obtain the probability values ​​of different categories corresponding to the time series data; The soft-max layer processing methods include: The output of the soft-max layer ,in is the weight of the soft-max layer, is the bias of the soft-max layer, represents the cross product of matrices, Indicates that the output value is converted to the range The probability distribution of sum is 1, and normalization ensures that the sum is 1, are the probability values ​​of different categories; Step 8: Use the Adam algorithm and the cross entropy loss function to optimize until the evaluation algorithm converges, and then use the test set Te to verify the evaluation algorithm; Step 9: When the rehabilitation assessment algorithm is verified to be effective, the patient is automatically assessed for rehabilitation using the rehabilitation assessment algorithm, and the rehabilitation scale used in step 1 is used. The inertial sensor is placed at the position where the inertial sensor is placed on the patient in step 1, so that the output data dimension is the same as that in step 1. The dimensions are the same; Step 10, filtering out the interference data collected in step 9; Step 11: Analyze the original data after filtering out the interference data in step 10 using the trained rehabilitation assessment algorithm, and calculate the score of a single assessment action; Step 12: Calculate the total score of the rehabilitation assessment obtained in step 11, perform a score rationality check on the total score of the rehabilitation assessment, and output the score that passes the score rationality check. If the score rationality check fails, an error is displayed; The rules for filtering out interference data in step 2 and step 10 include: Eliminate differences caused by different patient heights ,in , unit: m; represents the distance between the wearing position of N inertial sensors and the ground, ; The original data after eliminating the height difference of patients is normalized to , where min is the minimum value in the original data after eliminating the height difference of patients, and max is the maximum value in the original data after eliminating the height difference of patients; The original data after eliminating the height difference of patients is filtered to obtain the data after filtering out the interference data. ,in .

2. The LSTM-based automatic rehabilitation assessment method according to claim 1, characterized in that: In the step 1, T is the longest time for the patient to complete the rehabilitation assessment action. For the motion data whose completion time does not meet T, the cubic spline interpolation algorithm is used to make the data meet the conditions.

3. The LSTM-based automatic rehabilitation assessment method according to claim 1, characterized in that: The SC-LSTM layer processing methods in step 4 include: Forget Gate , in is the input value at the current time point, is the output value at the previous time point, Forgotten Gate The weight of Forgotten Gate The weight of Forgotten Gate The weight of is the bias of the forget gate, ; Input Gate ,in is the input value at this time point, is the output value at the previous time point, For the input gate The weight of For the input gate The weight of For the input gate The weight of is the bias of the input gate, ; Candidate cell states , in is the input value at the current time point, is the output value at the previous time point, For candidate cell states The weight of For candidate cell states The weight of For candidate cell states The weight of is the bias of the candidate cell state; New cell state ,in is the cell state at the previous time point, Represents the multiplication of corresponding elements of the matrix; Output Gate ,in is the input value at the current time point, is the output value at the previous time point, For the output gate The weight of For the output gate The weight of For the output gate The weight of is the bias of the output gate, ; Final Output ,in Represents the multiplication of corresponding elements of the matrix; Get the continuous output of the SC-LSTM layer .

4. The LSTM-based automatic rehabilitation assessment method according to claim 1, characterized in that: The fully connected layer processing methods in step 6 include: Output of the fully connected layer ,in is the weight of the fully connected layer, is the bias of the fully connected layer, Represents the cross product of matrices.

5. The LSTM-based automatic rehabilitation assessment method according to claim 1, characterized in that: In step 8, the cross entropy loss function , where y is the true category.

6. The LSTM-based automatic rehabilitation assessment method according to claim 1, characterized in that: The score rationality check in step 12 is based on the following rules: Compare the total score of this assessment Full score on the scale size; like , then find the total score of this assessment To the full score line Distance ; like , then the evaluation score is determined to be valid and output, where The total score of the last assessment and the full score of the scale The distance between.

7. An assessment terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the LSTM-based automatic rehabilitation assessment method as described in any one of claims 1 to 6 are implemented.

Images