Rehabilitation training upper limb movement measuring method based on depth track coding

Abstract

The invention discloses a rehabilitation training upper limb movement measurement method based on depth trajectory coding, which comprises the following steps: acquiring movement data of an upper limb by using a sensor, processing the data by using an inertial navigation algorithm to obtain corresponding speed, direction and displacement, and further obtaining movement information of the upper limb in a three-dimensional space; performing speed correction and accumulative error correction on the motion information by using an AZVU method; coding the corrected motion information by using a track coding mapping technology to generate a track coding graph; and constructing a TEM-VGG model based on the pre-trained convolutional layer, inputting the trajectory coding graph into the model to obtain probability values corresponding to different upper limb motion posture categories, and completing measurement of the upper limb motion. The method has certain advantages on multiple indexes in an upper limb rehabilitation posture recognition task, the confusion condition during classification can be effectively reduced, the recognition accuracy is remarkably improved, and the method has excellent performance in a complex rehabilitation action recognition task.

Description

Technical Field

[0001] The present invention relates to the technical field of rehabilitation therapy, and in particular to a method for measuring upper limb movement in rehabilitation training based on deep trajectory coding. Background Art

[0002] With the increase in the number of stroke patients, the demand for upper limb hemiplegia rehabilitation is becoming increasingly urgent. Traditional inpatient rehabilitation is costly, and home-based rehabilitation training has gradually attracted attention. At present, vision-based rehabilitation motion monitoring systems have privacy issues and cost limitations. Although inertial sensors have made progress, commercial products are expensive, low-cost solutions have large errors, and existing technologies are difficult to effectively extract rehabilitation posture features. Among the existing solutions, such as the markerless upper limb kinematic measurement system using Kinect devices, although no specific markers are required, the collected patient video images involve personal privacy, and the cost and scenarios limit its widespread application. MEMS sensors can achieve effective motion position tracking measurement in rehabilitation systems, but commercial products are relatively expensive, and low-cost sensors face challenges such as low measurement accuracy, data noise, and cumulative errors. In addition, it is still difficult to effectively extract upper limb motion features from wearable sensor data to achieve accurate recognition, and existing technologies need to be further improved in these aspects. Summary of the invention

[0003] The technical problem to be solved by the present invention is: to provide a method for measuring upper limb movement in rehabilitation training based on deep trajectory coding, by combining the AZVU (Adaptive Zero Velocity Update) method and TEM (Trajectory Encoding Mapping) to correct the accumulated errors in long-term tracking, and at the same time use anonymous technology to protect privacy data, so as to provide a more accurate, efficient and safe solution for upper limb rehabilitation movement monitoring.

[0004] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0005] A method for measuring upper limb movement in rehabilitation training based on deep trajectory coding comprises the following steps:

[0006] S1. Use MEMS sensors to collect upper limb motion data, use inertial navigation algorithms to process the data, obtain the corresponding speed, direction and displacement, and then track the training movements of the upper limbs to obtain the motion information of the upper limbs in three-dimensional space. And use SHA-256 hash encryption algorithm and data desensitization technology to protect the data.

[0007] S2. Use the AZVU method to perform speed correction and cumulative error correction on the motion information obtained in step S1.

[0008] S3. Encode the corrected motion information using trajectory coding mapping technology to generate a trajectory coding map.

[0009] S4. Build a TEM-VGG model based on the pre-trained convolutional layer, input the trajectory encoding map into the model, obtain the probability values ​​corresponding to different upper limb movement posture categories, and complete the measurement of upper limb movement.

[0010] Furthermore, in step S1, the MEMS sensor is bound to the trainee's forearm for data collection, and the motion data includes time step, acceleration, angular velocity and Euler angle.

[0011] The motion information includes coordinate values, acceleration information, angular velocity information, time information and speed information.

[0012] Furthermore, in step S2, the speed correction includes the following:

[0013] By calculating the characteristic values, the statistical characteristics of the trainee's upper limbs in motion and static states are obtained. The specific formula is:

[0014] ;

[0015] ;

[0016] ;

[0017] in, represents the acceleration vector norm at the ith time step, represents the angular velocity vector norm at the i-th time step, represents the statistical variance of the acceleration at the i-th time step, represents the component of acceleration on the x-axis at the i-th time step, represents the component of acceleration on the y-axis at the i-th time step, represents the component of acceleration on the z-axis at the ith time step, represents the component of the angular velocity on the x-axis at the i-th time step, represents the component of the angular velocity on the y-axis at the i-th time step, represents the component of the angular velocity on the z-axis at the i-th time step, represents the mean acceleration of the i-th time step, Represents the acceleration value at the kth time step.

[0018] The motion state is distinguished based on the acceleration vector norm, angular velocity vector norm and statistical variance of acceleration. When the feature meets a specific threshold condition, that is, , indicating that the upper limbs of the trainee at the i-th time step are in a stationary state, and the magnitude of the velocity vector at the i-th time step is On the contrary, it indicates that the upper limbs of the trainee at the i-th time step are in motion. Update the modulus of the current velocity vector to complete the correction of the velocity value.

[0019] in, represents the motion state of the i-th time step, represents the acceleration due to gravity, represents the speed increment, Represents the magnitude of the velocity vector at the i-1th time step.

[0020] Furthermore, in step S2, the cumulative error correction includes the following:

[0021] Calculate the angle between the y-axis and the gravity field at the initial position at the initial time step The specific formula is:

[0022] ;

[0023] in, represents the gravitational acceleration at a particular location, It represents the value of the earth's gravitational field acting on the y-axis in the zero velocity state at the initial time step, and arccos represents the inverse cosine function.

[0024] Calculate the angle during motion, when the angle first appears during motion , then it is considered that the motion forms a closed loop; further verification of the closed loop is performed, if the angle The coordinates of the measured position are equal to those of the initial position, indicating that the closed loop has been achieved and no cumulative error correction is required. If there is a deviation between the measured position and the initial position, the displacement of the measured position along the angle direction minus the coordinate difference between the measured position and the initial position is subtracted to complete the correction of the cumulative error.

[0025] Furthermore, in step S3, generating a trajectory coding map includes the following contents:

[0026] The coordinate values ​​in the three-dimensional space are normalized and scaled, and mapped to a 128*128*128 three-dimensional matrix to obtain an intermediate trajectory encoding map.

[0027] The specific expression of the position index of each three-dimensional coordinate point corresponding to the element in the three-dimensional matrix is:

[0028] ;

[0029] in, Represents the position index of the element in the three-dimensional matrix of the i-th time step, Max represents the maximum value, Represents the x-axis coordinate value of the i-th time step, Represents the y-axis coordinate value of the i-th time step, Represents the z-axis coordinate value of the i-th time step, Represents the x-axis coordinate value in three-dimensional space, Represents the y-axis coordinate value in three-dimensional space, Represents the z-axis coordinate value in three-dimensional space.

[0030] Based on the RGB color space, the motion features in space, time and direction are mapped to the three channels of the intermediate trajectory encoding map, where the corrected speed information is adapted to the blue channel, the time information is adapted to the green channel, and the direction information is adapted to the red channel to obtain the trajectory encoding map. The specific expression is:

[0031] ;

[0032] ;

[0033] ;

[0034] in, Indicates the speed information of the i-th time step is mapped to the pixel value of the blue channel, Indicates that the time information of the i-th time step is mapped to the pixel value of the green channel, Indicates that the direction information of the i-th time step is mapped to the pixel value of the red channel, ceil() represents the upward rounding function, m represents a constant, V represents the velocity list set, represents the direction vector norm of the i-th time step, A represents the set of direction data, represents the i-th time step, Indicates the total number of time steps.

[0035] The direction corresponding to the angle between the y-axis and the gravity field is the direction information.

[0036] Furthermore, in step S4, completing the measurement of upper limb movement includes the following:

[0037] The TEM-VGG (Trajectory Encoding Mapping-Visual Geometry Group Network) model includes an input layer, a convolutional layer, a ReLU activation function layer, a maximum pooling layer, a batch normalization layer, a fully connected layer, and an output layer connected in sequence. The convolutional layer includes a set number of convolutional blocks, and each convolutional block includes a set number of 3×3 convolutional kernels stacked together.

[0038] The trajectory encoding map is input into the TEM-VGG model, transmitted to the convolution layer through the input layer, and the convolution block is used to extract features of the image to obtain the first feature map. The feature map passes through the ReLU activation function layer and is nonlinearly transformed using the ReLU function to obtain a nonlinear feature map. The feature map passes through the maximum pooling layer and is downsampled using the maximum pooling operation to obtain a downsampled feature map. The feature map passes through the batch normalization layer and is normalized using the batch normalization method to obtain a normalized feature map. The feature map is classified through the fully connected layer to obtain probability values ​​corresponding to different upper limb movement posture categories, and the result is output through the output layer.

[0039] Furthermore, the present invention also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the upper limb motion measurement method for rehabilitation training based on deep trajectory coding when executing the computer program.

[0040] Furthermore, the present invention also proposes a computer-readable storage medium, which stores a computer program. When the computer program is run by a processor, it executes the upper limb motion measurement method for rehabilitation training based on deep trajectory coding.

[0041] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0042] The present invention effectively reduces measurement errors and realizes relatively controllable measurement of upper limb movement information.

[0043] The present invention utilizes the advantages of the DCNN (Deep Convolutional Neural Network) model to achieve efficient abstract feature extraction and has high recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is an overall implementation flow chart of the present invention.

[0045] Figure 2 Schematic diagram of a closed loop in an embodiment of the present invention.

[0046] Figure 3 It is a coordinate system diagram of the trajectory coding diagram in the embodiment of the present invention.

[0047] Figure 4 It is a structural diagram of the TEM-VGG model of the present invention.

[0048] Figure 5 It is a graph of original speeds of x, y and z axes when the arm is extended forward and sideways in an embodiment of the present invention.

[0049] Figure 6 It is a velocity curve diagram of the x, y, and z axes after using the AZVU method in arm extension, arm lateral extension, and forearm pronation in an embodiment of the present invention.

[0050] Figure 7 It is a graph of original speeds of x, y and z axes when the forearm is pronated, touching the lumbar spine and in the initial state in an embodiment of the present invention.

[0051] Figure 8 It is a velocity curve diagram of the x, y and z axes after the forearm is pronated, the lumbar spine is touched and the AZVU method is used in the initial state in an embodiment of the present invention.

[0052] Fig. 9 It is a comparison diagram of the displacement drift phenomenon without using the AZVU method and after using the AZVU method for error correction in the embodiment of the present invention when the arm is extended forward and the arm is extended sideways.

[0053] Fig.10 It is a comparison diagram of the displacement drift phenomenon of forearm pronation and touching the lumbar spine without and after error correction using the AZVU method in an embodiment of the present invention.

[0054] Fig.11 It is a recognition performance graph obtained based on the trajectory coding graph of the node spatial motion trajectory in an embodiment of the present invention.

[0055] Fig.12 It is a recognition performance diagram obtained based on the trajectory coding diagram of the node spatial motion trajectory and color coding in the embodiment of the present invention. DETAILED DESCRIPTION

[0056] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0057] To achieve the above objectives, the present invention proposes a method for measuring upper limb movement in rehabilitation training based on deep trajectory coding, such as Figure 1 As shown, the specific steps are as follows:

[0058] S1. Bind a MEMS sensor (such as Wit-Motion JY61) to the trainee's forearm to collect data. The data includes time step, acceleration, angular velocity and Euler angle. The MEMS sensor integrates a high-precision three-axis accelerometer and gyroscope and supports a sampling frequency of 100Hz. Use an inertial navigation algorithm to process the data to obtain the corresponding speed, direction and displacement, and then track the training movements of the upper limbs to obtain the motion information of the upper limbs in three-dimensional space. The motion information includes coordinate values, acceleration information, angular velocity information, time information and speed information.

[0059] S2, using the AZVU method to perform speed correction and cumulative error correction on the motion information obtained in step S1; the specific content is:

[0060] Speed ​​correction includes the following:

[0061] By calculating the characteristic values, the statistical characteristics of the trainee's upper limbs in motion and static states are obtained. The specific formula is:

[0062] ;

[0063] ;

[0064] ;

[0065] in, represents the acceleration vector norm of the i-th time step (i.e. the acceleration value after gravity compensation), represents the angular velocity vector norm at the i-th time step, represents the statistical variance of the acceleration at the i-th time step, represents the component of acceleration on the x-axis at the i-th time step, represents the component of acceleration on the y-axis at the i-th time step, represents the component of acceleration on the z-axis at the ith time step, represents the component of the angular velocity on the x-axis at the i-th time step, represents the component of the angular velocity on the y-axis at the i-th time step, represents the component of the angular velocity on the z-axis at the i-th time step, represents the mean acceleration of the i-th time step, Represents the acceleration value at the kth time step.

[0066] The motion state is distinguished based on the acceleration vector norm, angular velocity vector norm and statistical variance of acceleration. When the feature meets a specific threshold condition, that is, , indicating that the upper limbs of the trainee at the i-th time step are in a stationary state, and the magnitude of the velocity vector at the i-th time step is On the contrary, it indicates that the upper limbs of the trainee at the i-th time step are in motion. Update the modulus of the current velocity vector to complete the correction of the velocity value.

[0067] in, represents the motion state of the i-th time step, represents the acceleration due to gravity, represents the speed increment, Represents the magnitude of the velocity vector at the i-1th time step.

[0068] Cumulative error correction includes the following:

[0069] In the long-term motion tracking process, drift errors may accumulate over time, resulting in motion tracking failure. Given that the upper limb rehabilitation training movements are repetitive and limited, a closed loop is defined to identify a complete movement process, such as Figure 2 As shown in the figure, during the upper limb rehabilitation exercise, the limb movement trajectory starts from the starting position, goes through a series of movements and then returns to the starting position to form a complete movement process. Figure 2 (a) is the starting position diagram of the closed loop. Figure 2 (b) is the closed-loop motion process diagram, Figure 2 (c) is the diagram returning to the starting position of the closed loop, where blue is the x-axis, red is the y-axis, and green is the z-axis.

[0070] To avoid complex calculations, only the initial position between the y-axis and the gravity field at the initial time step is calculated (i.e. Figure 2 The yellow axis in the The specific formula is:

[0071] ;

[0072] in, represents the gravitational acceleration at a particular location, It represents the value of the earth's gravitational field acting on the y-axis in the zero velocity state at the initial time step, and arccos represents the inverse cosine function.

[0073] Calculate the angle during motion, when the angle first appears during motion , then it is considered that the motion forms a closed loop; further verification of the closed loop is performed, if the angle If the coordinates of the measured position are equal to those of the initial position, it indicates that the closed loop has been achieved and no cumulative error correction is required. If there is a deviation between the measured position and the initial position, it indicates that cumulative errors have occurred during the movement. The displacement of the measured position along the angle direction minus the coordinate difference between the measured position and the initial position is used to complete the correction of the cumulative error.

[0074] S3, using trajectory coding mapping technology to encode the corrected motion information and generate a trajectory coding map, such as Figure 3 The specific contents are as follows:

[0075] The coordinate values ​​in the three-dimensional space are normalized and scaled, and mapped into a 128*128*128 three-dimensional matrix to obtain an intermediate trajectory encoding map.

[0076] Each three-dimensional coordinate point corresponds to the position index of the element in the three-dimensional matrix. The specific expression is:

[0077] ;

[0078] in, Represents the position index of the element in the three-dimensional matrix of the i-th time step, Max represents the maximum value, Represents the x-axis coordinate value of the i-th time step, Represents the y-axis coordinate value of the i-th time step, Represents the z-axis coordinate value of the i-th time step, Represents the x-axis coordinate value in three-dimensional space, Represents the y-axis coordinate value in three-dimensional space, Represents the z-axis coordinate value in three-dimensional space.

[0079] Based on the RGB color space, the motion features in space, time and direction are mapped to the three channels of the intermediate trajectory encoding map, where the corrected speed information is adapted to the blue channel, the time information is adapted to the green channel, and the direction information is adapted to the red channel to obtain the trajectory encoding map. The specific expression is:

[0080] ;

[0081] ;

[0082] ;

[0083] in, Indicates the speed information of the i-th time step is mapped to the pixel value of the blue channel; Indicates the time information of the i-th time step is mapped to the pixel value of the green channel; Indicates that the direction information of the i-th time step is mapped to the pixel value of the red channel; ceil() represents the upward rounding function; m represents a constant, which is used in the calculation and m is introduced to prevent the denominator from being zero; V represents the speed list set; represents the direction vector norm of the i-th time step; A represents the set of direction data; represents the i-th time step; Indicates the total number of time steps.

[0084] The direction corresponding to the angle between the y-axis and the gravity field is the direction information.

[0085] from Figure 3It can be seen that each pixel in the image coordinate system has a unique coordinate. The coordinate system of the trajectory coding map provides a reference framework for mapping the displacement and motion information in the three-dimensional space to the pixel value in the two-dimensional space image coordinate system. It is the key basis for the construction of the trajectory coding map. Its existence ensures the orderly mapping and representation of motion information on the two-dimensional image, so that the trajectory coding map can accurately contain the spatial, temporal and dynamic characteristics of the motion. Figure 3 The coordinates of the blue square in the middle are (-3, -2), indicating that the displacement and motion information in the three-dimensional space coordinate system corresponds to the pixel coordinates in the two-dimensional space image coordinate system.

[0086] S4. Inspired by the classic deep convolutional network VGG network, the TEM-VGG model is constructed based on the pre-trained convolutional layer. The trajectory encoding map is input into the model to obtain the probability values ​​corresponding to different upper limb movement posture categories, and the measurement of upper limb movement is completed; the specific contents are:

[0087] like Figure 4 As shown in the figure, the TEM-VGG (Trajectory Encoding Mapping-Visual Geometry Group Network) model includes an input layer, a convolutional layer, a ReLU activation function layer, a maximum pooling layer, a batch normalization layer, a fully connected layer, and an output layer connected in sequence.

[0088] Among them, the convolution layer includes multiple convolution blocks, each convolution block includes two or three small 3×3 convolution kernels stacked together, which can increase the depth of the feature extractor, thereby automatically extracting distinguishable features from the posture data.

[0089] During the training phase, it is difficult to retrain the deep convolutional network to automatically extract abstract features from limited upper limb rehabilitation posture samples. Therefore, the TEM-VGG model loads the weight parameters learned from the large-scale image dataset ImageNet to initialize the weights and avoid overfitting caused by training the feature extractor from scratch to learn features from rehabilitation posture data.

[0090] The trajectory encoding map is input into the TEM-VGG model, transmitted to the convolution layer through the input layer, and the convolution block is used to extract features of the image to obtain the first feature map containing rich motion features. The feature map passes through the ReLU activation function layer and is processed by nonlinear transformation using the ReLU function to obtain a nonlinear feature map with stronger expressive power, which enhances the model's ability to learn complex patterns. The feature map passes through the maximum pooling layer and is downsampled using the maximum pooling operation to obtain a downsampled feature map with reduced dimension and retained main features, which reduces the data dimension and reduces the amount of model calculation. The feature map passes through the batch normalization layer and is normalized using the batch normalization method to obtain a normalized feature map with stable distribution, which accelerates the stable convergence of the network. The feature map is classified through the fully connected layer to obtain probability values ​​corresponding to different upper limb motion posture categories, and then the result is output through the output layer.

[0091] S5. Provide security protection for data. The specific contents are as follows:

[0092] (a) Hash encryption: Use the SHA-256 hash encryption algorithm to generate a fixed-length hash value for sensitive information in the patient's privacy data (such as ID, age, height, weight, etc.) to ensure that the data is irreversible.

[0093] (b) Data desensitization: Data desensitization technology is used on patient names, home addresses, contact information, etc., such as replacing some of them with "*" to protect patient privacy.

[0094] Example:

[0095] In this embodiment, the upper limb posture at the initial position (i.e., static state) at an initial time step and the upper limb training movements in four movement processes are collected. These five movements include the other 23 relevant upper limb training movements in the Fugl-Meyer motor function scoring scale for removing the fine movements of the upper limb fingers, as shown in Table 1.

[0096] Table 1 Definition of training exercises

[0097]

[0098] The trainees completed the above five movements as required, repeated each movement 20 times, and rested for 10 seconds after each training. The data collected from the upper limb rehabilitation training was transmitted via TTL (Transistor-Transistor Logic) and stored in a computer.

[0099] acc g The value is set to 9.80 (Nanjing, China).

[0100] Figure 5It is the original x, y, and z axis velocity curve diagram for the arm forward and arm sideways movements. Figure 6 The velocity curves of x, y, and z axes after applying the AZVU method in the arm extension, arm side extension, and forearm pronation movements. Figure 7 The original x, y, and z axis velocity curves of the forearm pronating, touching the lumbar spine, and the initial state are shown. Figure 8 The velocity curves of x, y, and z axes after forearm pronation, touching the lumbar spine, and applying the AZVU method in the initial state are compared. Figure 5 , Figure 6 as well as Figure 7 , Figure 8 It can be seen that the velocity curve after applying the AZVU method is obviously smoother and more stable in the non-motion state, and the curve connecting the dynamic and non-dynamic states is smoother, indicating that the AZVU method can effectively correct the velocity curves of arm extension, arm lateral extension, forearm pronation, and lumbar touch in the non-motion state, avoiding the random walk of velocity caused by measurement noise.

[0101] Fig. 9 Comparison of displacement drift phenomenon in arm forward extension and arm side extension movements without and with AZVU method for error correction; Fig.10 The figure shows the displacement drift phenomenon comparison between before and after using the AZVU method for error correction in the action of forearm pronation and touching the lumbar spine. The blue curve shows the displacement drift phenomenon without using the AZVU method for error correction, and the green curve shows the displacement drift phenomenon with the AZVU method for error correction. It can be seen that when the AZVU method is not used for error correction, the spatial displacement will accumulate drift errors over time due to the random walk of the speed. Long-term motion tracking will cause the deviation trend to become larger and larger, indicating that the AZVU method can significantly reduce the displacement drift phenomenon.

[0102] Fig.11 The recognition performance result diagram of the trajectory coding graph that retains the node spatial motion trajectory obtained in the motion measurement stage shows that the trajectory coding graph constructed only by relying on the spatial motion trajectory information has limited ability in distinguishing similar actions. Fig.12 In order to retain the spatial motion trajectory of the nodes obtained in the motion measurement stage and complete the recognition performance result diagram of the trajectory coding map with color coding, it can be seen that the trajectory coding map incorporating multivariate information can effectively improve the recognition ability of different upper limb rehabilitation movements, and the macro-precision index is improved by nearly 7%, indicating that the integration of multivariate information (speed, time and spatial feature information of limb nodes) can effectively improve the recognition effect.

[0103] The model proposed in the present invention (TEM-VGG-all-B+G+R-channels) is compared with FCNN (Fully Connected Neural Network), DCNN (Deep Convolutional Neural Network), AlexNet (classic deep convolutional neural network model), TEM-VGG-raw (Trajectory Encoding Map-Visual Geometry Group Network-raw version), and TEM-VGG-only-B+G-channels (Trajectory Encoding Map-Visual Geometry Group Network- only Blue and Green channels version) in terms of multiple indicators, as shown in Table 2.

[0104] Table 2 Experimental results

[0105]

[0106] As can be seen from Table 2, after 128 epochs of learning, the FCNN model can only obtain a comprehensive performance of 83.56% F1 score, while the TEM-VGG model proposed in the present invention can obtain a comprehensive performance of more than 91% F1 score after 30 epochs of learning. The TEM-VGG model has far fewer training parameters than AlexNet, and has obvious advantages in computing resources and time cost. Moreover, after 30 rounds of training, the TEM-VGG-all-B+G+R-channels model has better performance indicators than other models, and has high accuracy and reliability in recognizing upper limb rehabilitation postures. At the same time, when generating trajectory coding graphs, the configuration with complete feature information (B+G+R channels) has the best performance, and can effectively use multivariate information to improve recognition capabilities. Therefore, in the comparison of multiple models, the TEM-VGG model has obvious advantages in recognizing upper limb rehabilitation posture tasks. It combines the trajectory coding graph generation strategy and the pre-training fine-tuning strategy to effectively improve the recognition accuracy, reduce training parameters, and avoid overfitting.

[0107] The embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. It should be noted that when the processor executes the computer program, the specific steps of the method provided in the embodiment of the present invention are corresponding to the specific steps of the method provided in the embodiment of the present invention, and the processor has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the method provided in the embodiment of the present invention.

[0108] The embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. It should be noted that when the computer program is executed by the processor, it corresponds to the specific steps of the method provided in the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the method provided in the embodiment of the present invention.

[0109] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for measuring upper limb movement in rehabilitation training based on deep trajectory coding, characterized in that: include: S1. Use MEMS sensors to collect upper limb motion data, use inertial navigation algorithms to process the data, obtain corresponding speed, direction and displacement, and then track the training movements of the upper limbs, obtain the motion information of the upper limbs in three-dimensional space, and protect the data securely; S2, using the AZVU method to perform speed correction and cumulative error correction on the motion information obtained in step S1; S3, using trajectory coding mapping technology to encode the corrected motion information to generate a trajectory coding map; S4. Build a TEM-VGG model based on the pre-trained convolutional layer, input the trajectory encoding map into the model, obtain the probability values ​​corresponding to different upper limb movement posture categories, and complete the measurement of upper limb movement.

2. The method for measuring upper limb movement in rehabilitation training based on deep trajectory coding according to claim 1, characterized in that: In step S1, a MEMS sensor is bound to the trainee's forearm to collect data, and the motion data includes time step, acceleration, angular velocity and Euler angle; The motion information includes coordinate values, acceleration information, angular velocity information, time information and speed information.

3. The method for measuring upper limb movement in rehabilitation training based on deep trajectory coding according to claim 1, characterized in that: In step S1, the SHA-256 hash encryption algorithm and data desensitization technology are used to protect the data.

4. The method for measuring upper limb movement in rehabilitation training based on deep trajectory coding according to claim 2, characterized in that: In step S2, speed correction includes the following: By calculating the characteristic values, the statistical characteristics of the trainee's upper limbs in motion and static states are obtained. The specific formula is: ； ； ； in, represents the acceleration vector norm at the ith time step, represents the angular velocity vector norm at the i-th time step, represents the statistical variance of the acceleration at the i-th time step, represents the component of acceleration on the x-axis at the i-th time step, represents the component of acceleration on the y-axis at the i-th time step, represents the component of acceleration on the z-axis at the ith time step, represents the component of the angular velocity on the x-axis at the i-th time step, represents the component of the angular velocity on the y-axis at the i-th time step, represents the component of the angular velocity on the z-axis at the i-th time step, represents the mean acceleration of the i-th time step, Represents the acceleration value of the kth time step; The motion state is distinguished based on the acceleration vector norm, angular velocity vector norm and statistical variance of acceleration. When the feature meets a specific threshold condition, that is, , indicating that the upper limbs of the trainee at the i-th time step are in a stationary state, and the magnitude of the velocity vector at the i-th time step is On the contrary, it indicates that the upper limbs of the trainee at the i-th time step are in motion. Update the modulus of the current velocity vector to complete the correction of the velocity value; in, represents the motion state of the i-th time step, represents the acceleration due to gravity, represents the speed increment, Represents the magnitude of the velocity vector at the i-1th time step.

5. The method for measuring upper limb movement in rehabilitation training based on deep trajectory coding according to claim 4, characterized in that: In step S2, the cumulative error correction includes the following: Calculate the angle between the y-axis and the gravity field at the initial position at the initial time step The specific formula is: ； in, represents the gravitational acceleration at a particular location, represents the value of the earth's gravitational field acting on the y-axis at zero velocity at the initial time step, and arccos represents the inverse cosine function; Calculate the angle during the motion. When the angle appears for the first time , then it is considered that the motion forms a closed loop; further verification of the closed loop is performed, if the angle The coordinates of the measured position are equal to those of the initial position, indicating that the closed loop has been achieved. If there is a deviation between the measured position and the initial position, the displacement of the measured position along the angle direction is subtracted from the coordinate difference between the measured position and the initial position to complete the correction of the cumulative error.

6. The method for measuring upper limb movement in rehabilitation training based on deep trajectory coding according to claim 5, characterized in that: In step S3, generating a trajectory coding map includes the following contents: The coordinate values ​​in the three-dimensional space are normalized and scaled, and mapped to a 128*128*128 three-dimensional matrix to obtain an intermediate trajectory encoding map; The specific expression of the position index of each three-dimensional coordinate point corresponding to the element in the three-dimensional matrix is: ； in, Represents the position index of the element in the three-dimensional matrix of the i-th time step, Max represents the maximum value, Represents the x-axis coordinate value of the i-th time step, Represents the y-axis coordinate value of the i-th time step, Represents the z-axis coordinate value of the i-th time step, Represents the x-axis coordinate value in three-dimensional space, Represents the y-axis coordinate value in three-dimensional space, Represents the z-axis coordinate value in three-dimensional space; Based on the RGB color space, the motion features in space, time and direction are mapped to the three channels of the intermediate trajectory encoding map, where the corrected speed information is adapted to the blue channel, the time information is adapted to the green channel, and the direction information is adapted to the red channel to obtain the trajectory encoding map. The specific expression is: ； ； ； in, Indicates the speed information of the i-th time step is mapped to the pixel value of the blue channel, Indicates that the time information of the i-th time step is mapped to the pixel value of the green channel, Indicates that the direction information of the i-th time step is mapped to the pixel value of the red channel, ceil() represents the upward rounding function, and m represents a constant. represents the modulus of the velocity vector at the i-th time step, V represents the velocity list set, represents the direction vector norm of the i-th time step, A represents the set of direction data, represents the i-th time step, Indicates the total number of time steps; The direction corresponding to the angle between the y-axis and the gravity field is the direction information.

7. The method for measuring upper limb movement in rehabilitation training based on deep trajectory coding according to claim 1, characterized in that: In step S4, the measurement of upper limb movement is completed including the following: The TEM-VGG model includes an input layer, a convolution layer, a ReLU activation function layer, a maximum pooling layer, a batch normalization layer, a fully connected layer, and an output layer connected in sequence. The convolution layer includes a set number of convolution blocks, and each convolution block includes a set number of 3×3 convolution kernels stacked together. The trajectory encoding map is input into the TEM-VGG model, transmitted to the convolution layer through the input layer, and the convolution block is used to extract features of the image to obtain the first feature map. The feature map passes through the ReLU activation function layer and is nonlinearly transformed using the ReLU function to obtain a nonlinear feature map. The feature map passes through the maximum pooling layer and is downsampled using the maximum pooling operation to obtain a downsampled feature map. The feature map passes through the batch normalization layer and is normalized using the batch normalization method to obtain a normalized feature map. The feature map is classified through the fully connected layer to obtain probability values ​​corresponding to different upper limb movement posture categories, and the result is output through the output layer.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method for measuring upper limb movement in rehabilitation training based on deep trajectory coding as described in any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for measuring upper limb movement in rehabilitation training based on deep trajectory coding according to any one of claims 1 to 7 is executed.

Images