Effect detection method for rehabilitation training of upper limb dyskinesia

Abstract

The invention discloses an effect detection method for rehabilitation training of upper limb dyskinesia. The method comprises the following steps: acquiring sample data and a sample recognition result corresponding to the sample data; the sample data comprises a sample electromyographic signal and a sample speed parameter corresponding to the sample electromyographic signal; performing directed graph construction based on the sample electromyographic signals and the sample speed parameters to obtain a directed graph model; inputting the matrix data corresponding to the directed graph model into a preset action recognition model for action recognition to obtain a prediction recognition result; based on the prediction recognition result and the sample recognition result, performing model training on the preset action recognition model to obtain a target action recognition model; and inputting to-be-recognized data into the target action recognition model to obtain a target action recognition result. According to the method, the model is trained in combination with the sample data, and meanwhile, the self-attention mechanism is introduced, so that the accuracy and reliability of effect detection can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of data recognition, and more specifically, relates to a method for detecting the effect of upper limb motor dysfunction rehabilitation training. Background Art

[0002] Stroke is one of the main causes of adult disability, often causing hemiplegia, especially upper limb dysfunction. This dysfunction seriously affects the patient's daily living ability and quality of life. In order to promote the recovery of upper limb function, rehabilitation training has become an indispensable means.

[0003] In the prior art, the recognition of rehabilitation training movements mostly uses electromyographic signals of the forearm, upper arm and shoulder. The acquired electromyographic signals do not directly reflect the specific movement information of the fingers, and the lack of correlation with the finger movements limits the accuracy of movement recognition. In addition, due to differences in physical characteristics and muscle distribution among different individuals, the characteristics of the electromyographic signals vary significantly between different people, resulting in the defect of low recognition reliability. Summary of the invention

[0004] In order to solve the deficiencies in the prior art, the purpose of the present invention is to solve the above-mentioned defects and further propose a method for detecting the effect of upper limb motor dysfunction rehabilitation training.

[0005] The present invention adopts the following technical solution:

[0006] The first aspect of the present invention discloses a method for detecting the effect of upper limb motor dysfunction rehabilitation training, which is characterized by comprising:

[0007] Acquire sample data and a sample recognition result corresponding to the sample data; the sample data includes a sample electromyographic signal and a sample speed parameter corresponding to the sample electromyographic signal;

[0008] Constructing a directed graph based on the sample electromyographic signal and the sample speed parameter to obtain a directed graph model;

[0009] Inputting the matrix data corresponding to the directed graph model into a preset action recognition model to perform action recognition and obtain a predicted recognition result;

[0010] Based on the predicted recognition result and the sample recognition result, the preset action recognition model is trained to obtain a target action recognition model;

[0011] The data to be recognized is input into the target action recognition model to obtain the target action recognition result.

[0012] In some possible embodiments, constructing a directed graph based on the sample electromyographic signal and the sample speed parameter to obtain a directed graph model includes:

[0013] Based on each action type corresponding to the user state, determining a node corresponding to each action type;

[0014] Based on the sample electromyographic signal and the action conversion represented by the sample speed parameter, constructing directed edges between the nodes corresponding to each action type;

[0015] assigning weights to the directed edges based on the sample electromyographic signals and the motion conversion frequencies represented by the sample speed parameters;

[0016] The directed graph model is constructed according to the nodes, the directed edges and the weights.

[0017] In some possible embodiments, the method further includes:

[0018] Performing feature extraction processing on each node of the directed graph model to obtain node feature information corresponding to each node;

[0019] Determine the domain feature information of any node corresponding to each node according to the action type corresponding to each node and the node feature information corresponding to each node;

[0020] Determine the attention score of each node to any node according to the domain feature information of each node corresponding to any node combined with the self-attention mechanism;

[0021] Determine the strength of association between each node based on the attention score of each node corresponding to any node;

[0022] According to the action type corresponding to each node and the strength of association between each node, weighted processing is performed on the node feature information of the same type to determine the weighted feature information corresponding to each action type;

[0023] The weighted feature information corresponding to each action type is aggregated to generate matrix data corresponding to the directed graph model.

[0024] In some possible embodiments, the performing model training on the preset action recognition model based on the predicted recognition result and the sample recognition result to obtain the target action recognition model includes:

[0025] Determining the action recognition loss of the preset action recognition model based on the predicted recognition result and the sample recognition result;

[0026] Based on the action recognition loss, the preset action recognition model is iterated to obtain a target action recognition model.

[0027] In some possible embodiments, inputting the to-be-recognized data into the target action recognition model to obtain the target action recognition result includes:

[0028] Acquire the data to be identified corresponding to the target user and the target user status; the data to be identified includes the electromyographic signal to be identified and the speed parameter to be identified;

[0029] The to-be-recognized electromyographic signal, the to-be-recognized speed parameter, and the target user state are input into a target action recognition model for action recognition processing to obtain the target action recognition result.

[0030] In some possible embodiments, obtaining sample data includes:

[0031] Acquire original data, where the original data includes first original data corresponding to the first user state and second original data corresponding to the second user state;

[0032] Synchronously processing the first original data and the second original data respectively to obtain first synchronized data and second synchronized data;

[0033] Performing data segmentation processing on the first synchronization data and the second synchronization data respectively to obtain first segmented data and second segmented data;

[0034] Data screening is performed on the first segmented data and the second segmented data respectively to obtain first sample data and second sample data.

[0035] In some possible embodiments, performing data screening processing on the first segmented data and the second segmented data respectively to obtain the first sample data and the second sample data includes:

[0036] Performing data denoising processing on the first segmented data and the second segmented data respectively to obtain first denoised data and second denoised data;

[0037] Data normalization processing is performed on the first denoised data and the second denoised data respectively to obtain the first sample data and the second sample data.

[0038] The second aspect of the present invention discloses a device for detecting the effect of upper limb motor dysfunction rehabilitation training, comprising:

[0039] A sample data acquisition module, used to acquire sample data and a sample recognition result corresponding to the sample data; the sample data includes a sample electromyographic signal and a sample speed parameter corresponding to the sample electromyographic signal;

[0040] A directed graph construction module, used for constructing a directed graph based on the sample electromyographic signal and the sample velocity parameter to obtain a directed graph model;

[0041] A preset recognition module, used for inputting the matrix data corresponding to the directed graph model into a preset action recognition model for action recognition to obtain a predicted recognition result;

[0042] A model training module, used to perform model training on the preset action recognition model based on the predicted recognition result and the sample recognition result to obtain a target action recognition model;

[0043] The target recognition module is used to input the data to be recognized into the target action recognition model to obtain the target action recognition result.

[0044] The third aspect of the present invention discloses an electronic device, which includes a processor and a memory, wherein the memory stores at least one instruction and at least one program, and the at least one instruction and the at least one program are loaded and executed by the processor to implement the above-mentioned method for detecting the effect of upper limb motor dysfunction rehabilitation training.

[0045] The fourth aspect of the present disclosure discloses a computer storage medium, which stores at least one instruction and at least one program, and the at least one instruction and the at least one program are loaded and executed by a processor to implement the above-mentioned method for detecting the effect of upper limb motor dysfunction rehabilitation training.

[0046] The beneficial effects of the present invention are that, compared with the prior art, the present invention has the following advantages:

[0047] (1) By acquiring the electromyographic signals corresponding to the finger extensor and flexor muscles, the correlation between the electromyographic signals and the finger activities is improved, thereby improving the accuracy of action recognition;

[0048] (2) In the preprocessing of sample data, operations such as data synchronization and data segmentation are combined with time series features, which can reduce interference items in the sample data used to train the model, improve the accuracy of model training, and avoid the influence of invalid data on the model training process;

[0049] (3) Visualizing the action sequence of upper limb activities based on the directed graph model facilitates the analysis of the relationship between the actions, which helps to make the understanding of upper limb activity movements more convenient and simple during the model training process, thereby improving the effectiveness and efficiency of model training. At the same time, it can also improve the accuracy and effectiveness of action recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1A schematic flow chart of a method for detecting the effect of upper limb motor dysfunction rehabilitation training provided in an embodiment of the present application;

[0051] Figure 2 A schematic diagram of a flow chart of a method for determining matrix data corresponding to a directed graph model provided in an embodiment of the present application;

[0052] Figure 3 A first correlation diagram corresponding to a rehabilitation training effect provided in an embodiment of the present application;

[0053] Figure 4 A second correlation diagram corresponding to a rehabilitation training effect provided in an embodiment of the present application;

[0054] Figure 5 A structural schematic diagram corresponding to an effect detection device for upper limb motor dysfunction rehabilitation training provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of this specification to clearly and completely describe the technical solutions in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0057] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0058] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0059] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0060] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0061] Figure 1 The flowchart corresponding to the effect detection method provided by the embodiment of the present invention is shown; the execution subject can be any terminal or mobile device that can implement the effect detection method of upper limb motor dysfunction rehabilitation training. Figure 1 As shown, a method for detecting the effect of upper limb motor dysfunction rehabilitation training comprises:

[0062] S101: Acquire sample data and a sample recognition result corresponding to the sample data; the sample data includes a sample electromyographic signal and a sample speed parameter corresponding to the sample electromyographic signal;

[0063] In a specific embodiment, in the upper limb activity training of hemiplegia caused by stroke, the sample data can be the sample data corresponding to the upper limb activity training. Optionally, the sample data includes sample electromyographic signals and sample velocity parameters. The sample electromyographic signals are the bioelectric activities generated by muscles during contraction, which can be detected and recorded by electrodes on the surface of the skin. Specifically, the sample electromyographic signals can include surface electromyographic signals of the finger extensor muscles, finger flexor muscles, biceps brachii, triceps brachii, deltoid muscles, thenar eminence and hypothenar eminence; the sample velocity parameters can be inertial sensor data, and the sample velocity parameters provide timing information for the movement, which can characterize the movement process of the upper limb in space. Specifically, the sample velocity parameters can include accelerometer data, i.e., the acceleration of the wrist in the x, y, and z axes, gyroscope data, i.e., the angular velocity of the forearm around the x, y, and z axes, and magnetometer data: the change in the direction of the arm relative to the geomagnetic field. The sample recognition result may be a preset action recognition result. Optionally, the sample recognition result may include action category, action start and end time, action intensity and amplitude information, and action speed. The action category may include actions such as clenching a fist and raising a hand.

[0064] In an optional embodiment, the sample data may include first sample data corresponding to the first user state and second sample data corresponding to the second user state; and obtaining the sample data may include:

[0065] Acquire original data, where the original data includes first original data corresponding to the first user state and second original data corresponding to the second user state;

[0066] Synchronously processing the first original data and the second original data respectively to obtain first synchronized data and second synchronized data;

[0067] Performing data segmentation processing on the first synchronization data and the second synchronization data respectively to obtain first segmented data and second segmented data;

[0068] Data screening is performed on the first segmented data and the second segmented data respectively to obtain first sample data and second sample data.

[0069] In a specific embodiment, the raw data may include raw electromyographic signals and raw sample speed parameters that have not been subjected to data preprocessing; the user state may characterize whether the user has received upper limb activity training, the first user state may characterize that the user has not received upper limb activity training, and the second user state may characterize that the user has received upper limb activity training. Specifically, the first raw data corresponding to the first user state may be the raw data before receiving upper limb activity training, and the second raw data corresponding to the second user state may be the raw data after receiving upper limb activity training. The first sample data corresponding to the first user state may be the sample data before receiving upper limb activity training that has been subjected to data preprocessing, and the second sample data corresponding to the second user state may be the sample data after receiving upper limb activity training that has been subjected to data preprocessing.

[0070] In a specific embodiment, the selection of original data may further include control group data and experimental group data, where the control group data may be considered as first original data corresponding to the first user state, and the experimental group data may be considered as second original data corresponding to the second user state; wherein the original data may be classified according to gender, age, course of disease, hemiplegia type, and stroke type, such as the following Table 1:

[0071] Table 1 Comparison of baseline general data of the two groups of patients

[0072]

[0073] The screening criteria for raw data may include:

[0074] Inclusion criteria: patients who meet the diagnostic criteria for stroke determined by the Fourth National Cerebrovascular Disease Academic Conference, diagnosed with stroke by imaging, and have upper limb dysfunction caused by stroke; first-time, unilateral onset; course of disease 1-6 months; 35 years ≤ age ≤ 70 years old; upper limb Brunnstrom stage II-IV; upper limb muscle tension at the modified Ashworth grading standard level 0-3; able to sit independently.

[0075] Exclusion criteria: patients with upper limb dysfunction caused by other reasons before the onset of the disease; patients who did not receive conservative treatment or had skull defects; patients with Parkinson's disease or other neurological diseases; patients with serious complications, such as serious heart disease, serious infection, severe diabetes, etc.; patients with unstable conditions, such as blood pressure indicators: low pressure ≤90mmHg or high pressure ≥150mmHg; patients who are allergic to electrical stimulation; patients with epilepsy; patients with severe visual, auditory, cognitive and communication disorders and unable to undergo training. Cases with any one of the above criteria were excluded.

[0076] Exclusion and dropout criteria: patients who drop out naturally and have no valid data; patients who have serious adverse events and are not suitable to continue participating in the trial; patients with poor compliance and fail to follow the standard treatment plan.

[0077] By screening the original data according to the above conditions, it can be ensured that the original data meets the test research needs, improve the reliability of the test data, and avoid the impact of errors or inaccuracies in the test data on the test and research results.

[0078] The first original data may include a first original electromyographic signal and a first original speed parameter, and the second original data may include a second original electromyographic signal and a second original speed parameter. The first original electromyographic signal and the first original speed parameter are synchronously processed to obtain first synchronized data, and the first synchronized data may include the first synchronized electromyographic signal and the first synchronized speed parameter; the second original electromyographic signal and the second original speed parameter are synchronously processed to obtain second synchronized data, and the second synchronized data may include the second synchronized electromyographic signal and the second synchronized speed parameter. Optionally, an interpolation method may be used to synchronize the first original electromyographic signal and the first original speed parameter, and to synchronize the second original electromyographic signal and the second original speed parameter, so that the time axis of the two data is adjusted, thereby making the timestamps of the two data consistent. Specifically, the interpolation method used may be linear interpolation or cubic interpolation.

[0079] Sliding time windows of different window sizes can be used to perform data segmentation processing on the first synchronous electromyographic signal and the first synchronous speed parameter in the first synchronous data and the second synchronous electromyographic signal and the second synchronous speed parameter in the second synchronous data, respectively, to obtain the first segmented data and the second segmented data. The first segmented data may include the first segmented electromyographic signal and the first segmented speed parameter, and the second segmented data may include the second segmented electromyographic signal and the second segmented speed parameter, which can ensure that the features of the two types of data are accurately captured in the same time period, and the features in the same time window can be matched. Optionally, the window size can be set in combination with actual application requirements. Specifically, the window size can be 50 milliseconds or 100 milliseconds.

[0080] In an optional embodiment, performing data screening processing on the first segmented data and the second segmented data respectively to obtain the first sample data and the second sample data may include:

[0081] Performing data denoising processing on the first segmented data and the second segmented data respectively to obtain first denoised data and second denoised data;

[0082] Data normalization processing is performed on the first denoised data and the second denoised data respectively to obtain the first sample data and the second sample data.

[0083] In a specific embodiment, the first segmented electromyographic signal and the first segmented speed parameter in the first segmented data and the second segmented electromyographic signal and the second segmented speed parameter in the second segmented data are subjected to data denoising respectively to obtain first denoised data and second denoised data, the first denoised data may include the first denoised electromyographic signal and the first denoised speed parameter, and the second denoised data may include the second denoised electromyographic signal and the second denoised speed parameter. Optionally, the data denoising method may be set in combination with actual application requirements. Specifically, Kalman filtering and low-pass filtering may be used to perform data denoising on the first segmented speed parameter and the second segmented speed parameter to remove high-frequency noise, and band-pass filtering may be used to perform data denoising on the first segmented electromyographic signal and the second segmented electromyographic signal to remove power frequency noise and low-frequency artifacts, thereby ensuring the accuracy of the data.

[0084] The first denoised electromyographic signal and the first denoised speed parameter in the first denoised data and the second denoised electromyographic signal and the second denoised speed parameter in the second denoised data are normalized to obtain first sample data and second sample data. Optionally, the normalization method can be set in combination with actual application requirements. Specifically, minimum-maximum normalization can be used for normalization. Preprocessing of the data can ensure the accuracy of the data, thereby improving the training efficiency and stability of the subsequent model.

[0085] In a specific embodiment, the sample recognition result corresponding to the sample data may be determined manually or by a labeling system.

[0086] S102: constructing a directed graph based on the sample electromyographic signal and the sample speed parameter to obtain a directed graph model;

[0087] In an optional embodiment, the directed graph model obtained by constructing the directed graph based on the sample electromyographic signal and the sample speed parameter may include:

[0088] Based on each action type corresponding to the user state, determining a node corresponding to each action type;

[0089] Based on the sample electromyographic signal and the action conversion represented by the sample speed parameter, constructing directed edges between the nodes corresponding to each action type;

[0090] assigning weights to the directed edges based on the sample electromyographic signals and the motion conversion frequencies represented by the sample speed parameters;

[0091] The directed graph model is constructed according to the nodes, the directed edges and the weights.

[0092] In a specific embodiment, the action type may represent different actions of the user. Specifically, the action type may include actions such as extending the arm and clenching the fist; the action conversion may be a conversion between different actions. Specifically, the action conversion may include a conversion from clenching the fist to raising the hand; the action conversion frequency may be the frequency of action conversion. Based on each action type corresponding to the user state, the node corresponding to each action type is determined. Specifically, a node representing fist clenching and a node representing arm extension can be determined. Based on the action conversion represented by the sample electromyographic signal and the sample speed parameter, a directed edge is constructed for the relationship between the nodes, and the direction of the directed edge is from the node with an earlier action time to the node with a later action time. Specifically, the action conversion represented by the sample electromyographic signal and the sample speed parameter is a conversion from fist clenching to hand raising. A directed edge can be constructed between the node representing fist clenching and the node representing hand raising, and the direction of the directed edge is from the node representing fist clenching to the node representing hand raising, which can represent that the time of fist clenching is earlier than the time of hand raising. Based on the action conversion frequency represented by the sample electromyographic signal and the sample speed parameter, a weight is assigned to the directed edge. The weight can be determined by performing feature extraction processing on the sample electromyographic signal and the sample speed parameter respectively, and the electromyographic feature information and speed feature information obtained are used to determine the weight. The nodes, directed edges and weights are combined to construct a directed graph model.

[0093] In a specific embodiment, the user state includes a first user state and a second user state, and the directed graph model includes a first directed graph model and a second directed graph model. Then, the method for constructing the first directed graph model includes: determining the node corresponding to each action type corresponding to the first user state; constructing directed edges between nodes corresponding to each activity type based on the action conversion represented by the first sample data; assigning weights to the directed edges based on the action conversion frequency represented by the first sample data; and constructing the first directed graph model corresponding to the first sample data according to the nodes, directed edges, and weights. The method for constructing the second directed graph model includes: determining the node corresponding to each action type corresponding to the second user state; constructing directed edges between nodes corresponding to each activity type based on the action conversion represented by the second sample data; and assigning weights to the directed edges based on the action conversion frequency represented by the second sample data; and constructing the second directed graph model corresponding to the second sample data according to the nodes, directed edges, and weights.

[0094] In the above embodiment, the action sequence of upper limb activities can be visualized according to the directed graph model, which is convenient for analyzing the relationship between the actions, thereby making it easier and simpler to understand the upper limb activity actions during the model training process, thereby improving the effectiveness and efficiency of the model training. At the same time, it can also improve the accuracy and effectiveness of action recognition.

[0095] S103: Inputting the matrix data corresponding to the directed graph model into a preset action recognition model to perform action recognition, and obtaining a predicted recognition result;

[0096] In an alternative embodiment, Figure 2 is a flow chart of a method for determining matrix data corresponding to a directed graph model provided in an embodiment of the present application, such as Figure 2 As shown, the method for determining the matrix data corresponding to the directed graph model includes:

[0097] S201: performing feature extraction processing on each node of the directed graph model to obtain node feature information corresponding to each node;

[0098] S202: Determine domain feature information of any node corresponding to each node according to the action type corresponding to each node and the node feature information corresponding to each node;

[0099] S203: Determine the attention score of each node corresponding to any node according to the domain feature information of each node corresponding to any node combined with the self-attention mechanism;

[0100] S204: Determine the strength of association between each node based on the attention score of each node to any node;

[0101] S205: performing weighted processing on node feature information of the same type according to the action type corresponding to each node and the strength of association between each node, to determine weighted feature information corresponding to each action type;

[0102] S206: Aggregate the weighted feature information corresponding to each action type to generate matrix data corresponding to the directed graph model.

[0103] In a specific embodiment, the domain feature information can be feature information of the action type corresponding to the current node corresponding to the action type of other nodes, or it can be feature information between the current node and the nodes of the preceding action type and the subsequent action type involved in executing the action type corresponding to the current node; the attention score can reflect the relative importance between nodes in the training process.

[0104] In a specific embodiment, according to the action type corresponding to the node, the feature vector of each node after being mapped in the directed graph is extracted. Specifically, each node a i Feature information after mapping As shown below:

[0105]

[0106] Among them, Z n is the preset weight learning matrix, is the parameter to be learned, and its initial value is randomly set.

[0107] Determine the neighborhood feature vector of any node for any action type. Specifically, any node a i The neighborhood feature vector for any action type n As shown below:

[0108]

[0109] in, is a node in the directed graph i The set of nodes with action type n in the neighborhood of S i For node a i The adjacency matrix in a directed graph is, Represents node a u Feature information after mapping, where u = 1, 2, ..., N × M; S i Represents vertex a i The adjacency matrix of , I is the identity matrix.

[0110] According to the neighborhood feature vector of any node for any action type n, the attention score of any node for any action type is calculated. Specifically, any node a i Attention score for any type n As shown below:

[0111]

[0112] Among them, σ(·) represents the activation function, is the preset attention vector of the nth type, which is the parameter to be learned, and its initial value is randomly set. T represents the transpose of the vector.

[0113] According to any node a i For any type of attention score n, the strength of the association between any two nodes in the previous and next time is calculated. The two vertices in the previous and next time are a i ,a j , the strength of the correlation between them As shown below:

[0114]

[0115] Among them, t ij Represents two nodes a i ,a j The time difference between two nodes a i ,a j The corresponding ones are and Then t ij =t v -t u , l=1,2,…,M.

[0116] According to the correlation strength between two nodes in the same action type and in the previous and next time, the weighted feature vector of each type is calculated.

[0117] Specifically, the weighted feature vector P of the nth type n As shown below:

[0118]

[0119] P n =[…,p i ,…]i∈n

[0120] Among them, p i For node a i The weighted eigenvalues ​​of .

[0121] All weighted eigenvectors are aggregated to obtain the matrix data corresponding to the final directed graph model.

[0122] In some embodiments, all weighted feature vectors may be aggregated based on the attention mechanism, and the matrix data corresponding to the directed graph model may be shown as follows:

[0123] D n =R D P n

[0124] F n =R F P n

[0125] H n =R H P n

[0126]

[0127] where x is the dimension of the key vector, is a scaling factor used to control the size of the dot product to avoid excessive dot products causing the softmax function to saturate in areas where its gradient is small. D ,R F ,R H is the weight matrix in the attention mechanism model, which is the parameter to be learned and its initial value is randomized.

[0128] In a specific embodiment, the directed graph model includes a first directed graph model and a second directed graph model, and the matrix data corresponding to the directed graph model includes first matrix data and second matrix data. Then, a method for determining the first matrix data includes: performing feature extraction processing on each node of the first directed graph model to obtain node feature information corresponding to each node; determining the domain feature information of any node corresponding to each node according to the action type corresponding to each node and the node feature information corresponding to each node; determining the attention score of each node to any node based on the domain feature information of any node corresponding to each node combined with the self-attention mechanism; determining the strength of correlation between each node based on the attention score of any node corresponding to each node; performing weighted processing on the node feature information of the same type according to the action type corresponding to each node and the strength of correlation between each node to determine the weighted feature information corresponding to each action type; aggregating the weighted feature information corresponding to each action type to generate the first matrix data corresponding to the first directed graph model.

[0129] The method for determining the second matrix data includes: performing feature extraction processing on each node of the second directed graph model to obtain node feature information corresponding to each node; determining the domain feature information of each node corresponding to any node according to the action type corresponding to each node and the node feature information corresponding to each node; determining the attention score of each node for any node based on the domain feature information of each node corresponding to any node combined with the self-attention mechanism; determining the strength of correlation between each node based on the attention score of each node corresponding to any node; performing weighted processing on the node feature information of the same type according to the action type corresponding to each node and the strength of correlation between each node to determine the weighted feature information corresponding to each action type; aggregating the weighted feature information corresponding to each action type to generate the second matrix data corresponding to the second directed graph model.

[0130] In a specific embodiment, the preset action recognition model can be set in combination with actual application requirements. Specifically, the preset action recognition model can be an LSTM (Long Short-Term Memory) model. The LSTM network can be used to capture long-term temporal dependencies in input data.

[0131] For the LSTM network, in order to reduce its training time, the electromyographic signal and speed parameters can be divided into a linear trend part and a nonlinear part based on the time series decomposition method, and the linear trend part can be modeled with the ARIMA model, so that the LSTM network can focus more on the nonlinear part, thereby obtaining a more accurate prediction effect.

[0132] The training process of the LSTM network can be described as follows:

[0133] The model is set to training mode; each batch of sample data is traversed, the input data and true value are loaded by reading the sample data, and the gradient record of the optimizer is cleared; the model is forward propagated to obtain the prediction result, and the loss function is used to calculate the loss; the model is back-propagated to calculate the gradient; the model parameters are updated to achieve model iteration.

[0134] In a specific embodiment, the model can be used to make predictions based on the softmax function, and the predicted results are It can be shown as follows:

[0135]

[0136] Among them, e and w represent the parameters to be learned, e is a Q×M matrix, and w is a Q×1 vector. Q represents the number of types of rehabilitation training. For example: rehabilitation training can include at least three categories: rehabilitation exercise, drug therapy, and auxiliary therapy, then Q=3; correspondingly, if the large category of rehabilitation exercise also includes subcategories, such as rehabilitation exercise 1 for the shoulder, rehabilitation exercise 2 for the fingers, and rehabilitation exercise 3 for the upper arm, etc., then the value of Q can be expanded to a larger value (for example, Q=6), but the classification granularity of Q should depend on the amount of sample data.

[0137] For example, rehabilitation exercises can include: active and passive joint range of motion training, muscle strength training, Bobath motor control training, finger grasping, lifting wooden sticks, pushing sanding boards, lifting dumbbells, turning wooden pegs, building blocks, etc., and early intervention in training in daily living activities such as washing face, eating, dressing, and organizing hands. The specific training time can be set according to different situations, such as 30 minutes a day, once a day, 5 days a week.

[0138] For auxiliary treatment, an upper limb rehabilitation training auxiliary robot can be used. The training content includes: single joint training around the shoulder and elbow joints; shoulder-elbow coordination training: shoulder flexion-elbow extension, shoulder extension-elbow flexion multi-joint coordinated separation movement; task-oriented training: according to the game task setting, complete the action of simulating the arm to take objects forward in three-dimensional space; the specific training time can be set according to different situations, such as 30 minutes a day, once a day, 5 days a week. In addition, the auxiliary instructions can also include the use of Fourier multi-channel therapeutic apparatus to give functional electrical stimulation treatment to the corresponding prime mover key points of the affected upper limb: deltoid anterior bundle, biceps brachii, triceps brachii, abductor pollicis brevis, extensor digitorum communis, first dorsal interosseous muscle, etc.

[0139] In a specific embodiment, a loss function is constructed to determine the parameters to be learned when the matrix data is based on the self-attention mechanism and the directed graph model. The loss function Loss can be shown as follows:

[0140]

[0141] Among them, A Q Indicates the type of treatment actually selected by the individual. If the treatment type is selected, its value is 1, otherwise it is 0. Q Representation vector The Qth value of .

[0142] S104: Based on the predicted recognition result and the sample recognition result, the preset action recognition model is trained to obtain a target action recognition model;

[0143] In a specific embodiment, the predicted recognition result may be an action recognition result predicted based on a preset action recognition model.

[0144] In an optional embodiment, the method of performing model training on the preset action recognition model based on the predicted recognition result and the sample recognition result to obtain the target action recognition model includes:

[0145] Determining the action recognition loss of the preset action recognition model based on the predicted recognition result and the sample recognition result;

[0146] Based on the action recognition loss, model training is performed on the preset action recognition model to obtain a target action recognition model.

[0147] In a specific embodiment, the action recognition loss can be calculated in combination with a preset loss function; optionally, the preset loss function can be set in combination with actual application requirements, such as an exponential loss function, a cross entropy loss function, etc. The above action recognition loss can characterize the accuracy of action recognition of the current preset action recognition model.

[0148] In a specific embodiment, the above-mentioned training of the preset action recognition model based on the action recognition loss to obtain the target action recognition model may include: based on the action recognition loss, updating the model parameters of the preset action recognition model, based on the updated preset action recognition model, repeating the above-mentioned inputting the matrix data corresponding to the directed graph model into the preset action recognition model for action recognition, obtaining the predicted recognition result based on the action recognition loss, and updating the training iteration step of the model parameters of the preset action recognition model until the preset convergence condition is met. The above-mentioned preset convergence condition can be that the action recognition loss information is less than or equal to the preset loss threshold, or the number of training iteration steps reaches the preset number, etc. Specifically, the preset loss threshold and the preset number can be set in combination with the model accuracy and training speed requirements in actual applications.

[0149] S105: Input the data to be recognized into the target action recognition model to obtain a target action recognition result.

[0150] In an optional embodiment, the step of inputting the to-be-recognized data into the target action recognition model to obtain the target action recognition result includes:

[0151] Acquire the data to be identified corresponding to the target user and the target user status; the data to be identified includes the electromyographic signal to be identified and the speed parameter to be identified;

[0152] The to-be-recognized electromyographic signal, the to-be-recognized speed parameter, and the target user state are input into a target action recognition model for action recognition processing to obtain the target action recognition result.

[0153] In a specific embodiment, after obtaining the target action recognition result, the method further includes: analyzing the target action recognition result in combination with a preset rehabilitation training standard to determine the training effect of the upper limb activity.

[0154] Specifically, for the evaluation of training effects, the indicators can be tested before and after the study. The evaluation indicators and evaluation methods are as follows:

[0155] (1) Upper limb function: The short form motor function scale-upper limb (FMA-UE) was used to evaluate the patient's upper limb function. The higher the score, the better the upper limb motor function.

[0156] (2) Muscle strength: Manual muscle test (MMT) is used to evaluate the patient's upper limb muscle strength. The higher the grade, the better the muscle strength.

[0157] (3) sEMG surface electromyography: A wireless surface electromyography system is used. The patient sits upright and exposes the skin of the affected upper limb. After cleaning the skin, the wireless surface electrode receiver is attached to the anterior deltoid, biceps, triceps, abductor pollicis brevis, extensor digitorum common, and first dorsal interosseous muscle on the affected side. During the test, the limb is asked to complete the upper arm raising, forearm extension, and whole arm forward to pick up objects. Repeat three times, each time with an interval of 20 seconds. MegaWin software is used to process the collected EMG data. After filtering, rectification, and interception, the muscle integral (iEMG) and root mean square (RMS) are calculated by integration and sliding window respectively;

[0158] (4) nEMG needle electromyography: The test was performed using an electrodiagnostic instrument on the first dorsal interosseous muscle, collecting 20 MUAPs during active recruitment reaction and light contraction, and analyzing the data of reading time, amplitude and phase. All collected signals were amplified and digitized by a 3-channel amplifier, and the sampling frequency of each channel was 20-2000Hz. The power shield and ground wire were used to minimize the interference of the power line. The medical disposable needle electrode used in the test was a concentric needle electrode with a needle length of 40mm and a needle diameter of 0.45mm.

[0159] By using the above evaluation method to evaluate the effect of rehabilitation exercise and analyze the effect of rehabilitation training from multiple angles, it is possible to take into account the comprehensiveness of the detection of the effect of rehabilitation training.

[0160] In a specific embodiment, if the difference between the target action recognition result and the preset rehabilitation training is less than a preset difference threshold, then the training effect of the upper limb activity indicates that the upper limb training effect of the user is significant. The preset difference threshold is set in combination with actual application requirements.

[0161] In the above embodiment, action recognition is performed on the data to be recognized based on the target action recognition model to obtain the target action recognition result, which can improve the accuracy and efficiency of action recognition, and further improve the accuracy of determining the training effect, so as to monitor and evaluate the user's motion status.

[0162] Furthermore, in a specific embodiment, the trained data may be as follows:

[0163] (1) Comparison of FMA-UE scores between the two groups

[0164] As shown in Table 2, compared with the baseline, the FMA-UE scores of the two groups were significantly improved (P < 0.01); the inter-group comparison after intervention showed that the FMA-UE score of the experimental group was significantly better than that of the control group (P < 0.05).

[0165] Table 2 Comparison of FMA-UE scores between the two groups before and after treatment

[0166] Group Before treatment After treatment t P Control group (n=27) 19.44±8.02 28.85±10.10 -5.529 ＜0.001 Experimental group (n=29) 19.17±7.76 34.34±10.23 -10.689 ＜0.001 t 0.129 -2.019 P 0.898 0.048

[0167] (2) Comparison of MMT scores between the two groups

[0168] As shown in Table 3, compared with the baseline, the MMT scores of the deltoid anterior, biceps brachii, and abductor pollicis brevis in the control group were significantly improved (P < 0.05), and the MMT scores of the deltoid anterior, biceps brachii, triceps brachii, and extensor digitorum communis in the experimental group were significantly improved (P < 0.05); the inter-group comparison after intervention showed that the MMT scores of the deltoid anterior and biceps brachii between the two groups were statistically different (P < 0.05); no other statistical differences were found (P > 0.05).

[0169]

[0170]

[0171] Table 3 Comparison of MMT scores between the two groups before and after treatment

[0172] (3) Comparison of surface electromyography between the two groups

[0173] As shown in Table 4, compared with the baseline, the iEMG and RMS of the deltoid anterior, abductor pollicis brevis, first dorsal interosseous muscle, and iEMG of the biceps brachii in the control group were significantly increased (P < 0.05), while the iEMG and RMS of the deltoid anterior, biceps brachii, triceps brachii, abductor pollicis brevis, extensor digitorum communis, and first dorsal interosseous muscle in the experimental group were significantly increased (P < 0.05); after the intervention, the inter-group comparison showed that the iEMG of the deltoid anterior and biceps brachii showed statistical differences between the two groups (P < 0.05), and the RMS of the deltoid anterior and triceps brachii showed statistical differences between the two groups (P < 0.05); no other statistical differences were found (P > 0.05).

[0174]

[0175]

[0176] Table 4 Comparison of iEMG and RMS between the two groups before and after treatment

[0177] (4) Comparison of needle electromyography between the two groups

[0178] As shown in Table 5, compared with the baseline, the amplitude of the first dorsal interosseous muscle in the control group and the experimental group was significantly increased before and after the intervention (P < 0.05), and no statistical differences were found in other groups (P > 0.05).

[0179]

[0180] Table 5 Comparison of nEMG parameters between the two groups before and after treatment

[0181] In addition, the present invention Figure 3 and Figure 4 A correlation diagram corresponding to the rehabilitation training effect is also provided, such as Figure 3 as well as Figure 4 As shown in the figure, after the intervention, the change value of the FMA-UE score of the affected upper limb was significantly positively correlated with the iEMG of the anterior deltoid muscle (r=0.471, p=0.01); after the intervention, the change value of the FMA-UE score of the affected upper limb was significantly positively correlated with the RMS of the anterior deltoid muscle (r=0.415, p=0.025).

[0182] Through the above data comparison and image analysis, the effect of rehabilitation training can be intuitively seen, thereby improving the comprehensiveness and intuitiveness of rehabilitation training effect evaluation.

[0183] The embodiment of the present invention also provides a device for detecting the effect of upper limb motor dysfunction rehabilitation training. Figure 5 As shown, the device comprises:

[0184] The sample data acquisition module 510 is used to acquire sample data and sample recognition results corresponding to the sample data; the sample data includes a sample electromyographic signal and a sample speed parameter corresponding to the sample electromyographic signal;

[0185] A directed graph construction module 520, configured to construct a directed graph based on the sample electromyographic signal and the sample velocity parameter to obtain a directed graph model;

[0186] A preset recognition module 530, used to input the matrix data corresponding to the directed graph model into a preset action recognition model for action recognition to obtain a predicted recognition result;

[0187] A model training module 540 is used to perform model training on the preset action recognition model based on the predicted recognition result and the sample recognition result to obtain a target action recognition model;

[0188] The target recognition module 550 is used to input the data to be recognized into the target action recognition model to obtain the target action recognition result.

[0189] In some possible embodiments, the directed graph construction module 520 includes:

[0190] A node determination module, used to determine a node corresponding to each action type based on each action type corresponding to the user state;

[0191] A directed edge determination module, configured to construct directed edges between nodes corresponding to each action type based on the action conversion represented by the sample electromyographic signal and the sample speed parameter;

[0192] A weight determination module, configured to assign weights to the directed edges based on the sample electromyographic signal and the motion conversion frequency represented by the sample velocity parameter;

[0193] A construction module is used to construct the directed graph model according to the nodes, the directed edges and the weights.

[0194] In some possible embodiments, the device further includes:

[0195] A feature extraction module is used to perform feature extraction processing on each node of the directed graph model to obtain node feature information corresponding to each node;

[0196] A feature information determination module, used to determine the domain feature information of any node corresponding to each node according to the action type corresponding to each node and the node feature information corresponding to each node;

[0197] A score calculation module, used to determine the attention score of each node corresponding to any node according to the domain feature information of each node corresponding to any node combined with the self-attention mechanism;

[0198] A correlation determination module, used to determine the correlation strength between each node based on the attention score of each node corresponding to any node;

[0199] A weighted processing module, used to perform weighted processing on node feature information of the same type according to the action type corresponding to each node and the strength of association between each node, and determine weighted feature information corresponding to each action type;

[0200] The matrix data determination module is used to aggregate the weighted feature information corresponding to each action type to generate matrix data corresponding to the directed graph model.

[0201] In some possible embodiments, the model training module 540 includes:

[0202] A loss determination module, used to determine the action recognition loss of the preset action recognition model based on the predicted recognition result and the sample recognition result;

[0203] The model iteration module is used to perform model iteration on the preset action recognition model based on the action recognition loss to obtain a target action recognition model.

[0204] In some possible embodiments, the target identification module 550 includes:

[0205] A data acquisition module, used to acquire the data to be identified corresponding to the target user and the target user status; the data to be identified includes the electromyographic signal to be identified and the speed parameter to be identified;

[0206] The action recognition module is used to input the to-be-recognized electromyographic signal, the to-be-recognized speed parameter and the target user state into a target action recognition model for action recognition processing to obtain the target action recognition result.

[0207] In some possible embodiments, the sample data acquisition module 510 includes:

[0208] An original data acquisition module, used to acquire original data, wherein the original data includes first original data corresponding to a first user state and second original data corresponding to a second user state;

[0209] A first data processing module, used for synchronously processing the first original data and the second original data respectively to obtain first synchronous data and second synchronous data;

[0210] A second data processing module is used to perform data segmentation processing on the first synchronization data and the second synchronization data respectively to obtain first segmented data and second segmented data;

[0211] The third data processing module is used to perform data screening processing on the first segmented data and the second segmented data respectively to obtain first sample data and second sample data.

[0212] In some possible embodiments, the third data processing module further includes:

[0213] a fourth data processing module, configured to perform data denoising processing on the first segmented data and the second segmented data respectively to obtain first denoised data and second denoised data;

[0214] The fifth data processing module is used to perform data normalization processing on the first denoised data and the second denoised data respectively to obtain the first sample data and the second sample data.

[0215] The device and method embodiments in the device embodiment are based on the same inventive concept and are used to implement the above-mentioned method for detecting the effect of upper limb motor dysfunction rehabilitation training.

[0216] An embodiment of the present invention also provides an electronic device, comprising: a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the effect detection method of upper limb motor dysfunction rehabilitation training as described in any one of the method embodiments.

[0217] An embodiment of the present invention also provides a computer storage medium, which can be set in a server to store at least one instruction, at least one program, a code set or an instruction set for implementing a method for detecting the effect of rehabilitation training for upper limb motor dysfunction in a method embodiment. The at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement the method for detecting the effect of rehabilitation training for upper limb motor dysfunction as described in any one of the method embodiments.

[0218] Optionally, in an embodiment of the present invention, the above-mentioned storage medium may be located in at least one of a plurality of network servers of a computer network. Optionally, in an embodiment of the present invention, the above-mentioned storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0219] It should be noted that various embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes are obvious to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used in this article is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for detecting the effect of rehabilitation training for upper limb motor dysfunction, characterized in that: The method comprises: Acquire sample data and a sample recognition result corresponding to the sample data; the sample data includes a sample electromyographic signal and a sample speed parameter corresponding to the sample electromyographic signal; Constructing a directed graph based on the sample electromyographic signal and the sample speed parameter to obtain a directed graph model; Inputting the matrix data corresponding to the directed graph model into a preset action recognition model to perform action recognition and obtain a predicted recognition result; Based on the predicted recognition result and the sample recognition result, the preset action recognition model is trained to obtain a target action recognition model; The data to be recognized is input into the target action recognition model to obtain the target action recognition result.

2. The method for detecting the effect of upper limb motor dysfunction rehabilitation training according to claim 1, characterized in that: The directed graph model is constructed based on the sample electromyographic signal and the sample speed parameter, and includes: Based on each action type corresponding to the user state, determining a node corresponding to each action type; Based on the sample electromyographic signal and the action conversion represented by the sample speed parameter, constructing directed edges between the nodes corresponding to each action type; assigning weights to the directed edges based on the sample electromyographic signals and the motion conversion frequencies represented by the sample speed parameters; The directed graph model is constructed according to the nodes, the directed edges and the weights.

3. The method for detecting the effect of upper limb motor dysfunction rehabilitation training according to claim 1, characterized in that: The method further comprises: Performing feature extraction processing on each node of the directed graph model to obtain node feature information corresponding to each node; Determine the domain feature information of any node corresponding to each node according to the action type corresponding to each node and the node feature information corresponding to each node; Determine the attention score of each node to any node according to the domain feature information of each node corresponding to any node combined with the self-attention mechanism; Determine the strength of association between each node based on the attention score of each node corresponding to any node; According to the action type corresponding to each node and the strength of association between each node, weighted processing is performed on the node feature information of the same type to determine the weighted feature information corresponding to each action type; The weighted feature information corresponding to each action type is aggregated to generate matrix data corresponding to the directed graph model.

4. The method for detecting the effect of upper limb motor dysfunction rehabilitation training according to claim 1, characterized in that: The performing model training on the preset action recognition model based on the predicted recognition result and the sample recognition result to obtain the target action recognition model comprises: Determining the action recognition loss of the preset action recognition model based on the predicted recognition result and the sample recognition result; Based on the action recognition loss, the preset action recognition model is iterated to obtain a target action recognition model.

5. The method for detecting the effect of upper limb motor dysfunction rehabilitation training according to claim 1, characterized in that: The step of inputting the to-be-recognized data into the target action recognition model to obtain the target action recognition result comprises: Acquire the data to be identified corresponding to the target user and the target user status; the data to be identified includes the electromyographic signal to be identified and the speed parameter to be identified; The to-be-recognized electromyographic signal, the to-be-recognized speed parameter, and the target user state are input into a target action recognition model for action recognition processing to obtain the target action recognition result.

6. The method for detecting the effect of upper limb motor dysfunction rehabilitation training according to claim 1, characterized in that: The sample data includes first sample data corresponding to a first user state and second sample data corresponding to a second user state; The obtaining of sample data comprises: Acquire original data, where the original data includes first original data corresponding to the first user state and second original data corresponding to the second user state; Synchronously processing the first original data and the second original data respectively to obtain first synchronized data and second synchronized data; Performing data segmentation processing on the first synchronization data and the second synchronization data respectively to obtain first segmented data and second segmented data; Data screening is performed on the first segmented data and the second segmented data respectively to obtain first sample data and second sample data.

7. The method for detecting the effect of upper limb motor dysfunction rehabilitation training according to claim 6, characterized in that: The performing data screening processing on the first segmented data and the second segmented data respectively to obtain the first sample data and the second sample data comprises: Performing data denoising processing on the first segmented data and the second segmented data respectively to obtain first denoised data and second denoised data; Data normalization processing is performed on the first denoised data and the second denoised data respectively to obtain the first sample data and the second sample data.

8. An upper limb motor dysfunction rehabilitation training effect detection device, characterized in that: The device comprises: A sample data acquisition module, used to acquire sample data and a sample recognition result corresponding to the sample data; the sample data includes a sample electromyographic signal and a sample speed parameter corresponding to the sample electromyographic signal; A directed graph construction module, used for constructing a directed graph based on the sample electromyographic signal and the sample velocity parameter to obtain a directed graph model; A preset recognition module, used for inputting the matrix data corresponding to the directed graph model into a preset action recognition model for action recognition to obtain a predicted recognition result; A model training module, used to perform model training on the preset action recognition model based on the predicted recognition result and the sample recognition result to obtain a target action recognition model; The target recognition module is used to input the data to be recognized into the target action recognition model to obtain the target action recognition result.

9. An electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction and at least one program, and the at least one instruction and the at least one program are loaded and executed by the processor to implement the method for detecting the effect of upper limb motor dysfunction rehabilitation training as described in any one of claims 1-7.

10. A computer storage medium, wherein at least one instruction and at least one program are stored in the computer storage medium, wherein the at least one instruction and the at least one program are loaded and executed by a processor to implement the method for detecting the effect of upper limb motor dysfunction rehabilitation training as described in any one of claims 1 to 7.

Images