An upper limb movement measurement method for rehabilitation training based on deep trajectory encoding

Abstract

The present invention discloses an upper limb movement measurement method for rehabilitation training based on depth trajectory encoding, which includes collecting movement data of the upper limb using a sensor, processing the data using an inertial navigation algorithm to obtain corresponding speed, direction, and displacement, and further obtaining the movement information of the upper limb in three-dimensional space; using the AZVU method to correct the speed and cumulative error of the movement information; using the trajectory encoding mapping technology to encode the corrected movement information to generate a trajectory encoding map; constructing a TEM-VGG model based on a pre-trained convolutional layer, inputting the trajectory encoding map into the model to obtain probability values corresponding to different upper limb movement posture categories, and completing the measurement of upper limb movement. The present invention has certain advantages in multiple indicators in the upper limb rehabilitation posture recognition task, can effectively reduce the confusion situation during classification, significantly improve the recognition accuracy, and has excellent performance in complex rehabilitation movement recognition tasks.

Description

Technical Field

[0001] The present invention relates to the technical field of rehabilitation therapy, and particularly relates to a method for measuring upper limb movement during rehabilitation training based on depth trajectory encoding. Background Art

[0002] With the increasing number of stroke patients, the need for upper limb hemiplegia rehabilitation is becoming increasingly urgent. Traditional inpatient rehabilitation is costly, and home-based rehabilitation training has gradually received attention. Currently, vision-based rehabilitation motion monitoring systems have privacy issues and cost limitations. Although inertial sensors have made progress, commercial products are expensive, low-cost solutions have large errors, and it is difficult for existing technologies to effectively extract rehabilitation posture features. Among existing solutions, for example, a markerless upper limb kinematics measurement system using a Kinect device does not require specific markers, but the patient video images collected involve personal privacy, and cost and scenarios limit its wide application. MEMS sensors can achieve effective motion position tracking and measurement in a rehabilitation system, but commercial products are relatively expensive, and low-cost sensors face challenges such as low measurement accuracy, data noise, and cumulative errors. In addition, effectively extracting upper limb movement features from wearable sensor data to achieve accurate identification remains a difficult problem, and existing technologies need to be further improved in these aspects. Summary of the Invention

[0003] The technical problem to be solved by the present invention is: to provide a method for measuring upper limb movement during rehabilitation training based on depth trajectory encoding, which corrects the errors accumulated during long-term tracking by combining the AZVU (Adaptive Zero Velocity Update) method and TEM (Trajectory Encoding Mapping), and at the same time uses anonymization technology to protect privacy data, providing a more accurate, efficient, and secure solution for upper limb rehabilitation motion monitoring.

[0004] To solve the above technical problems, the present invention adopts the following technical solutions:

[0005] A method for measuring upper limb movement during rehabilitation training based on depth trajectory encoding, comprising the following steps:

[0006] S1. Use a MEMS sensor to collect the movement data of the upper limb, process the movement data using an inertial navigation algorithm to obtain the corresponding speed, direction, and displacement, and then track the training actions of the upper limb to obtain the movement information of the upper limb in three-dimensional space. And use the SHA-256 hash encryption algorithm and data desensitization technology to protect the data.

[0007] S2. Use the AZVU method to correct the speed and cumulative error of the movement information obtained in step S1.

[0008] S3. Encode the corrected motion information using the trajectory encoding mapping technique to generate a trajectory encoding map.

[0009] S4. Construct a TEM-VGG model based on a pre-trained convolutional layer, input the trajectory encoding map into the TEM-VGG model, and obtain probability values corresponding to different upper limb movement posture categories to complete the measurement of upper limb movement.

[0010] Further, in step S1, the MEMS sensor is bound to the forearm of the trainer for data collection, and the motion data includes time step, acceleration, angular velocity, and Euler angle.

[0011] The motion information includes coordinate values, acceleration information, angular velocity information, time information, and velocity information.

[0012] Further, in step S2, the speed correction includes the following:

[0013] By calculating the eigenvalues, obtain the statistical characteristics of the trainer's upper limb in the motion state and the static state. The specific formula is:

[0014]

[0015] Among them, represents the acceleration vector norm at the i-th time step, represents the angular velocity vector norm at the i-th time step, represents the statistical variance of the acceleration at the i-th time step, represents the component of the acceleration at the i-th time step on the x-axis, represents the component of the acceleration at the i-th time step on the y-axis, represents the component of the acceleration at the i-th time step on the z-axis, represents the component of the angular velocity at the i-th time step on the x-axis, represents the component of the angular velocity at the i-th time step on the y-axis, represents the component of the angular velocity at the i-th time step on the z-axis, represents the acceleration mean at the i-th time step, represents the acceleration value at the k-th time step.

[0016] Distinguish the motion state based on the acceleration vector norm, angular velocity vector norm, and statistical variance of the acceleration. When the statistical characteristics meet specific threshold conditions, that is it indicates that the trainer's upper limb is in a static state at the i-th time step, and the modulus of the velocity vector at the i-th time step On the contrary, it indicates that the trainer's upper limb is in a motion state at the i-th time step. At this time, through Update the magnitude of the current velocity vector to complete the correction of the velocity value.

[0017] Among them, represents the motion state at the i-th time step, g represents the acceleration due to gravity, δ represents the velocity increment, represents the magnitude of the velocity vector at the (i - 1)-th time step.

[0018] Furthermore, in step S2, the cumulative error correction includes the following:

[0019] The specific formula for calculating the angle θ0 between the position on the y-axis in the zero-velocity state at the initial time step and the gravity field is:

[0020]

[0021] Among them, acc g represents the acceleration due to gravity in the area where the motion occurs, acc y0 represents the value of the Earth's gravitational field acting on the y-axis in the zero-velocity state at the initial time step, and arccos represents the inverse cosine function.

[0022] Calculate the angle during the motion. When the angle θ0 first appears during the motion, it is considered that the motion forms a closed loop at this time; further verify the closed loop. If the motion position where the angle θ0 first appears is equal to the coordinates of the position in the zero-velocity state, it indicates that the closed loop has been achieved and no cumulative error correction is required. If there is a deviation between the motion position and the coordinates of the position in the zero-velocity state, subtract the coordinate difference between the motion position and the position in the zero-velocity state from the displacement of the motion position along the angle direction to complete the correction of the cumulative error.

[0023] Furthermore, in step S3, generating the trajectory encoding map includes the following:

[0024] Perform normalization and scaling operations on the coordinate values in three-dimensional space and map them into a 128 * 128 * 128 three-dimensional matrix to obtain an intermediate trajectory encoding map.

[0025] The specific expression for the position index of each three-dimensional coordinate point corresponding to the element in the three-dimensional matrix is:

[0026]

[0027] Among them, represents the position index of the element in the three-dimensional matrix at the i-th time step, Max represents the maximum value, represents the x-axis direction coordinate value at the i-th time step, represents the y-axis direction coordinate value at the i-th time step, represents the z-axis direction coordinate value at the i-th time step, p xRepresents the x-axis coordinate value in three-dimensional space, p y Represents the y-axis coordinate value in three-dimensional space, p z Represents the z-axis coordinate value in three-dimensional space.

[0028] Based on the RGB color space, the motion features in space, time, and direction are mapped into three channels of the intermediate trajectory encoding map. Among them, the corrected velocity information is adapted to the blue channel, the time information is adapted to the green channel, and the direction information is adapted to the red channel to obtain the trajectory encoding map. The specific expression is:

[0029]

[0030] Among them, Represents the pixel value of the velocity information of the i-th time step mapped to the blue channel, Represents the pixel value of the time information of the i-th time step mapped to the green channel, R Ti Represents the pixel value of the direction information of the i-th time step mapped to the red channel, ceil() represents the ceiling function, m represents a constant, V represents the set of velocity lists, A Ti Represents the norm of the direction vector of the i-th time step, A represents the set of direction data, T i Represents the i-th time step, T total Represents the total number of time steps.

[0031] The direction corresponding to the angle between the y-axis and the gravitational field is the direction information.

[0032] Furthermore, in step S4, the measurement of upper limb movement includes the following contents:

[0033] The TEM-VGG (Trajectory Encoding Mapping-Visual Geometry Group Network) model includes an input layer, a convolutional layer, a ReLU activation function layer, a max pooling layer, a batch normalization layer, a fully connected layer, and an output layer connected in sequence. Among them, the convolutional layer includes a set number of convolutional blocks, and each convolutional block includes a set number of 3×3 convolutional kernels stacked together.

[0034] Input the trajectory encoding map into the TEM-VGG model, and transmit it to the convolutional layer through the input layer. Use the convolutional block to extract features from the trajectory encoding map to obtain the first feature map. The first feature map passes through the ReLU activation function layer and undergoes non-linear transformation processing using the ReLU function to obtain the non-linear feature map. The non-linear feature map passes through the max pooling layer and undergoes downsampling processing using the max pooling operation to obtain the downsampled feature map. The downsampled feature map passes through the batch normalization layer and undergoes normalization processing using the batch normalization method to obtain the normalized feature map, and then undergoes classification processing through the fully connected layer to obtain the probability values corresponding to different upper limb movement posture categories, and finally outputs the probability values corresponding to different upper limb movement posture categories through the output layer.

[0035] Furthermore, the present invention also proposes an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the upper limb movement measurement method for rehabilitation training based on depth trajectory encoding.

[0036] Furthermore, the present invention also proposes a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is run by the processor, it executes the upper limb movement measurement method for rehabilitation training based on depth trajectory encoding.

[0037] Compared with the prior art, the present invention adopts the above technical solutions and has the following technical effects:

[0038] The present invention effectively reduces the measurement error and realizes the relatively controllable measurement of upper limb movement information.

[0039] The present invention utilizes the advantages of the DCNN (Deep Convolutional Neural Network) model, realizes efficient abstract feature extraction, and has a high recognition accuracy. Description of the Drawings

[0040] Figure 1 is the overall implementation flowchart of the present invention.

[0041] Figure 2 is the closed-loop schematic diagram in the embodiment of the present invention.

[0042] Figure 3 is the coordinate system diagram of the trajectory encoding map in the embodiment of the present invention.

[0043] Figure 4 is the structural diagram of the TEM-VGG model of the present invention.

[0044] Figure 5It is the original velocity curve graph of the x, y, and z axes under arm forward extension and arm lateral extension in the embodiments of the present invention.

[0045] Figure 6 It is the velocity curve graph of the x, y, and z axes after using the AZVU method under arm forward extension, arm lateral extension, and forearm pronation in the embodiments of the present invention.

[0046] Figure 7 It is the original velocity curve graph of the x, y, and z axes under forearm pronation, touching the lumbar spine, and the initial state in the embodiments of the present invention.

[0047] Figure 8 It is the velocity curve graph of the x, y, and z axes after using the AZVU method under forearm pronation, touching the lumbar spine, and the initial state in the embodiments of the present invention.

[0048] Figure 9 It is the comparison graph of the displacement drift phenomenon under arm forward extension and arm lateral extension in the embodiments of the present invention without using and using the AZVU method for error correction.

[0049] Figure 10 It is the comparison graph of the displacement drift phenomenon under forearm pronation and touching the lumbar spine in the embodiments of the present invention without using and using the AZVU method for error correction.

[0050] Figure 11 It is the recognition performance graph obtained based on the trajectory coding graph of the node space motion trajectory in the embodiments of the present invention.

[0051] Figure 12 It is the recognition performance graph obtained based on the trajectory coding graph of the node space motion trajectory and completed color coding in the embodiments of the present invention. Detailed implementation manners

[0052] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and cannot be used to limit the protection scope of the present invention.

[0053] To achieve the above object, the present invention proposes a rehabilitation training upper limb motion measurement method based on depth trajectory coding, as Figure 1 shown, and the specific steps are as follows:

[0054] S1. Bind a MEMS sensor (such as Wit-Motion JY61) to the forearm of the trainer for data acquisition. The data includes time step, acceleration, angular velocity, and Euler angles. The MEMS sensor integrates a high-precision three-axis accelerometer and gyroscope and supports a sampling frequency of 100 Hz. Use the inertial navigation algorithm to process the data to obtain the corresponding speed, direction, and displacement, and then track the training movements of the upper limb to obtain the motion information of the upper limb in three-dimensional space. The motion information includes coordinate values, acceleration information, angular velocity information, time information, and speed information.

[0055] S2. Use the AZVU method to perform speed correction and cumulative error correction on the motion information obtained in step S1. The specific content is as follows:

[0056] The speed correction includes the following content:

[0057] By calculating the eigenvalues, obtain the statistical characteristics of the trainer's upper limb in the motion state and the static state. The specific formula is:

[0058]

[0059] Among them, represents the acceleration vector norm at the i-th time step (i.e., the acceleration value after gravity compensation), represents the angular velocity vector norm at the i-th time step, represents the statistical variance of the acceleration at the i-th time step, represents the component of the acceleration at the i-th time step on the x-axis, represents the component of the acceleration at the i-th time step on the y-axis, represents the component of the acceleration at the i-th time step on the z-axis, represents the component of the angular velocity at the i-th time step on the x-axis, represents the component of the angular velocity at the i-th time step on the y-axis, represents the component of the angular velocity at the i-th time step on the z-axis, represents the acceleration mean at the i-th time step, represents the acceleration value at the k-th time step.

[0060] Based on the acceleration vector norm, angular velocity vector norm, and statistical variance of acceleration to distinguish the motion state. When the statistical characteristics meet specific threshold conditions, that is it indicates that the trainer's upper limb is in a static state at the i-th time step, and the modulus of the velocity vector at the i-th time step On the contrary, it indicates that the trainer's upper limb is in a motion state at the i-th time step. At this time, through Update the magnitude of the current velocity vector to complete the correction of the velocity value.

[0061] Among them, represents the motion state at the i-th time step, g represents the gravitational acceleration, δ represents the velocity increment, represents the magnitude of the velocity vector at the (i - 1)-th time step.

[0062] The cumulative error correction includes the following:

[0063] During the long-term motion tracking process, drift errors may accumulate over time, resulting in the failure of motion tracking. Given that the upper limb rehabilitation training actions are repetitive and limited, a closed loop is defined to identify a complete action process, such as Figure 2 shown. During the upper limb rehabilitation exercise process, the complete motion process formed when the limb motion trajectory starts from the starting position, goes through a series of actions, and then returns to the starting position. Figure 2 (a) of Figure 2 is the starting position diagram of the closed loop, Figure 2 (b) of

[0064] is the closed loop motion process diagram, Figure 2 and (c) of

[0065]

[0066] is the diagram of returning to the starting position of the closed loop again. Among them, the blue is the x-axis, the red is the y-axis, and the green is the z-axis.

[0064] To avoid complex calculations, the specific formula for calculating the angle θ0 between the position on the y-axis in the zero-velocity state at the initial time step and the gravitational field (i.e., the yellow axis in Figure 2 ) is:

[0065]

[0066] Among them, acc g represents the gravitational acceleration in the area where the motion occurs, and acc y0 represents the value of the Earth's gravitational field acting on the y-axis in the zero-velocity state at the initial time step. arccos represents the inverse cosine function.

[0067] Calculate the angle during the motion process. When the angle θ0 first appears during the motion, it is considered that the motion forms a closed loop at this time; further verify the closed loop. If the motion position where the angle θ0 first appears is equal to the coordinates of the position in the zero-velocity state, it indicates that the closed loop has been achieved and no cumulative error correction is required. If there is a deviation between the motion position and the coordinates of the position in the zero-velocity state, it indicates that cumulative errors have occurred during the motion process. Subtract the coordinate difference between the motion position and the position in the zero-velocity state from the displacement of the motion position along the angle direction to complete the correction of the cumulative error.

[0068] S3. Use the trajectory coding mapping technology to encode the corrected motion information to generate a trajectory coding diagram, as Figure 3 shown. The specific content is:

[0069] Normalize and scale the coordinate values in three-dimensional space and map them into a three-dimensional matrix of 128*128*128 to obtain an intermediate trajectory encoding map.

[0070] Each three-dimensional coordinate point corresponds to the position index of an element in the three-dimensional matrix, and the specific expression is:

[0071]

[0072] where represents the position index of an element in the three-dimensional matrix at the i-th time step, Max represents the maximum value, represents the coordinate value in the x-axis direction at the i-th time step, represents the coordinate value in the y-axis direction at the i-th time step, represents the coordinate value in the z-axis direction at the i-th time step, p x represents the x-axis coordinate value in three-dimensional space, p y represents the y-axis coordinate value in three-dimensional space, p z represents the z-axis coordinate value in three-dimensional space.

[0073] Based on the RGB color space, map the motion features in space, time, and direction into three channels of the intermediate trajectory encoding map, where the corrected speed information is adapted to the blue channel, the time information is adapted to the green channel, and the direction information is adapted to the red channel to obtain a trajectory encoding map. The specific expression is:

[0074]

[0075]

[0076] where represents the pixel value of the speed information mapped to the blue channel at the i-th time step; represents the pixel value of the time information mapped to the green channel at the i-th time step; represents the pixel value of the direction information mapped to the red channel at the i-th time step; ceil() represents the ceiling function; m represents a constant, which is introduced when calculating and to prevent the denominator from being zero; V represents the set of speed lists; represents the norm of the direction vector at the i-th time step; A represents the set of direction data; T i represents the i-th time step; T total represents the total number of time steps.

[0077] The direction corresponding to the angle between the y-axis and the gravity field is the direction information.

[0078] FromFigure 3 It can be seen that each pixel in the image coordinate system has a unique coordinate. The coordinate system of the trajectory encoding map provides a reference framework for mapping the displacement and motion information in the three-dimensional space to the pixel values in the two-dimensional space image coordinate system. It is the key foundation for the construction of the trajectory encoding map. Its existence ensures the orderly mapping and representation of motion information on the two-dimensional image, enabling the trajectory encoding map to accurately contain the spatial, temporal, and dynamic characteristics of the motion. Figure 3 The coordinates of the blue square in it are (-3, -2), indicating that the displacement and motion information in the three-dimensional space coordinate system correspond to the pixel coordinates in the two-dimensional space image coordinate system.

[0079] S4. Inspired by the classic deep convolutional network VGG network, a TEM-VGG model is constructed based on the pre-trained convolutional layer. The trajectory encoding map is input into the TEM-VGG model to obtain the probability values corresponding to different upper limb movement posture categories, completing the measurement of upper limb movement. The specific content is as follows:

[0080] As Figure 4 shown, the TEM-VGG (Trajectory Encoding Mapping-Visual Geometry Group Network) model includes an input layer, a convolutional layer, a ReLU activation function layer, a max pooling layer, a batch normalization layer, a fully connected layer, and an output layer connected in sequence.

[0081] Among them, the convolutional layer includes multiple convolutional blocks, and each convolutional block includes two or three small 3×3-sized convolutional kernels stacked together, which can increase the depth of the feature extractor, thereby enabling automatic extraction of distinguishable features from the pose data.

[0082] In the training stage, it is difficult to retrain the deep convolutional network to automatically extract abstract features from the limited upper limb rehabilitation pose samples. Therefore, the TEM-VGG model loads the weight parameters learned from the large-scale image dataset ImageNet to initialize the weights, avoiding overfitting caused by training the feature extractor from the rehabilitation pose data to learn features from scratch.

[0083] Input the trajectory encoding map into the TEM-VGG model, and transmit it to the convolutional layer through the input layer. Use the convolutional block to extract features from the trajectory encoding map to obtain a first feature map containing rich motion features. The first feature map passes through the ReLU activation function layer and is processed by the ReLU function for non-linear transformation to obtain a non-linear feature map with stronger expressive ability, enhancing the model's learning ability for complex patterns. The non-linear feature map passes through the max pooling layer and is downsampled using the max pooling operation to obtain a downsampled feature map with reduced dimensions and retained main features, reducing the data dimensions and the model's computational complexity. The downsampled feature map passes through the batch normalization layer and is normalized using the batch normalization method to obtain a normalized feature map with a stable distribution, accelerating the stable convergence of the network, and then is classified through the fully connected layer to obtain the probability values corresponding to different upper limb movement posture categories, and finally the probability values corresponding to different upper limb movement posture categories are output through the output layer.

[0084] S5. Protect the data, and the specific content is as follows:

[0085] (a) Hash encryption: Use the SHA-256 hash encryption algorithm to generate a fixed-length hash value for sensitive information (such as ID, age, height, weight, etc.) in the patient's privacy data to ensure the irreversibility of the data.

[0086] (b) Data desensitization: Adopt data desensitization technology for information such as the patient's name, home address, and contact information, such as partially replacing it with "*" to protect the patient's privacy and security.

[0087] Embodiment:

[0088] In this embodiment, the upper limb postures at the initial position (i.e., the static state) at an initial time step and four upper limb training actions during the movement process are collected. These five actions include the other 23 relevant upper limb training actions used to remove the fine movements of the fingers in the Fugl-Meyer motor function assessment scale, as shown in Table 1.

[0089] Table 1 Training exercise definitions

[0090]

[0091] The trainer completes the above five actions as required, repeats each action 20 times, and takes a 10-second break after each training. The data collected from the upper limb rehabilitation training is transmitted through TTL (Transistor-Transistor Logic) and stored on the computer.

[0092] acc g The value is set to 9.80 (Nanjing, China).

[0093] Figure 5 It is the original x, y, and z axis velocity curve diagram for the arm forward and arm sideways movements. Figure 6 The velocity curves of x, y, and z axes after applying the AZVU method in the arm extension, arm side extension, and forearm pronation movements. Figure 7 The original x, y, and z axis velocity curves of the forearm pronating, touching the lumbar spine, and the initial state are shown. Figure 8 The velocity curves of x, y, and z axes after forearm pronation, touching the lumbar spine, and applying the AZVU method in the initial state are compared. Figure 5 , Figure 6 as well as Figure 7 , Figure 8 It can be seen that the velocity curve after applying the AZVU method is obviously smoother and more stable in the non-motion state, and the curve connecting the dynamic and non-dynamic states is smoother, indicating that the AZVU method can effectively correct the velocity curves of arm extension, arm lateral extension, forearm pronation, and lumbar touch in the non-motion state, avoiding the random walk of velocity caused by measurement noise.

[0094] Figure 9 Comparison of displacement drift phenomenon in arm forward extension and arm side extension movements without and with AZVU method for error correction; Figure 10 The figure shows the displacement drift phenomenon comparison between before and after using the AZVU method for error correction in the action of forearm pronation and touching the lumbar spine. The blue curve shows the displacement drift phenomenon without using the AZVU method for error correction, and the green curve shows the displacement drift phenomenon with the AZVU method for error correction. It can be seen that when the AZVU method is not used for error correction, the spatial displacement will accumulate drift errors over time due to the random walk of the speed. Long-term motion tracking will cause the deviation trend to become larger and larger, indicating that the AZVU method can significantly reduce the displacement drift phenomenon.

[0095] Figure 11 The recognition performance result diagram of the trajectory coding graph that retains the node spatial motion trajectory obtained in the motion measurement stage shows that the trajectory coding graph constructed only by relying on the spatial motion trajectory information has limited ability in distinguishing similar actions. Figure 12 In order to retain the spatial motion trajectory of the nodes obtained in the motion measurement stage and complete the recognition performance result diagram of the trajectory coding map with color coding, it can be seen that the trajectory coding map incorporating multivariate information can effectively improve the recognition ability of different upper limb rehabilitation movements, and the macro-precision index is improved by nearly 7%, indicating that the integration of multivariate information (speed, time and spatial feature information of limb nodes) can effectively improve the recognition effect.

[0096] Compare the results of the model proposed in the present invention (TEM-VGG-all-B+G+R-channels) with those of FCNN (Fully Connected Neural Network), DCNN (Deep Convolutional Neural Network), AlexNet (a classic deep convolutional neural network model), TEM-VGG-raw (Trajectory Encoding Map-Visual Geometry Group Network-raw version), and TEM-VGG-only-B+G-channels (Trajectory Encoding Map-Visual Geometry Group Network-only Blue and Green channels version) in terms of multiple metrics, as shown in Table 2.

[0097] Table 2 Experimental Results

[0098]

[0099] As can be seen from Table 2, after 128 epochs of learning, the FCNN model can only achieve a comprehensive performance of 83.56% F1 score, while the TEM-VGG model proposed in the present invention can achieve a comprehensive performance of more than 91% F1 score after 30 epochs of learning. The TEM-VGG model has far fewer training parameters than AlexNet, with obvious advantages in computing resources and time costs. Moreover, after 30 rounds of training, the TEM-VGG-all-B+G+R-channels model has better performance in multiple metrics than other models, with high accuracy and reliability in recognizing upper limb rehabilitation postures. At the same time, when generating the trajectory encoding map, the configuration with complete feature information (B+G+R channels) has the best performance, which can effectively utilize multi-source information to improve the recognition ability. Therefore, in the comparison of multiple models, the TEM-VGG model has obvious advantages in the task of recognizing upper limb rehabilitation postures. By combining the trajectory encoding map generation strategy and the pre-training and fine-tuning strategy, it can effectively improve the recognition accuracy, reduce the training parameters, and avoid overfitting.

[0100] An embodiment of the present invention further provides an electronic device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor. It should be noted that when the processor executes the computer program, it corresponds to the specific steps of the method provided by the embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. For the technical details not described in detail in this embodiment, reference can be made to the method provided by the embodiment of the present invention.

[0101] An embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program. It should be noted that when the computer program is run by the processor, it corresponds to the specific steps of the method provided by the embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. For the technical details not described in detail in this embodiment, reference can be made to the method provided by the embodiment of the present invention.

[0102] The above are only the preferred embodiments of the present invention. It should be pointed out that for those of ordinary skill in the art, without departing from the technical principle of the present invention, several improvements and deformations can be made, and these improvements and deformations should also be regarded as the protection scope of the present invention.

Claims

1. A method for measuring upper limb movement in rehabilitation training based on deep trajectory coding, characterized in that: include: S1. Collect motion data of upper limbs using MEMS sensors, process the motion data using inertial navigation algorithms, obtain corresponding speed, direction and displacement, and then track the training movements of the upper limbs, obtain motion information of the upper limbs in three-dimensional space, and perform security protection on the data; S2, using the AZVU method to perform speed correction and cumulative error correction on the motion information obtained in step S1; the specific content is: Speed ​​correction includes the following: By calculating the characteristic values, the statistical characteristics of the trainee's upper limbs in motion and static states are obtained. The specific formula is: in, represents the acceleration vector norm at the ith time step, represents the angular velocity vector norm at the i-th time step, represents the statistical variance of the acceleration at the i-th time step, represents the component of acceleration on the x-axis at the i-th time step, represents the component of acceleration on the y-axis at the i-th time step, represents the component of acceleration on the z-axis at the ith time step, represents the component of the angular velocity on the x-axis at the i-th time step, represents the component of the angular velocity on the y-axis at the i-th time step, represents the component of the angular velocity on the z-axis at the i-th time step, represents the mean acceleration of the i-th time step, Represents the acceleration value of the kth time step; The motion state is distinguished based on the acceleration vector norm, the angular velocity vector norm and the statistical variance of the acceleration. When the statistical features meet a specific threshold condition, that is, It indicates that the upper limb of the trainee at the i-th time step is in a stationary state, and the magnitude of the velocity vector at the i-th time step is On the contrary, it indicates that the upper limbs of the trainee at the i-th time step are in motion. Update the modulus of the current velocity vector to complete the correction of the velocity value; in, represents the motion state of the i-th time step, g represents the gravitational acceleration, δ represents the velocity increment, represents the magnitude of the velocity vector at the i-1th time step; Cumulative error correction includes the following: The specific formula for calculating the angle θ0 between the y-axis and the gravity field at the zero velocity state at the initial time step is: Among them, acc g Indicates the gravitational acceleration of the area during movement, acc y0 represents the value of the earth's gravitational field acting on the y-axis at zero velocity at the initial time step, and arccos represents the inverse cosine function; The angle during the motion process is calculated. When the angle θ0 appears for the first time, it is considered that the motion forms a closed loop. The closed loop is further verified. If the coordinates of the motion position where the angle θ0 appears for the first time are equal to the coordinates of the position at zero speed, it indicates that the closed loop has been achieved. If there is a deviation between the coordinates of the motion position and the position at zero speed, the displacement of the motion position along the angle direction is subtracted from the coordinate difference between the motion position and the position at zero speed to complete the correction of the accumulated error. S3, using trajectory coding mapping technology to encode the corrected motion information to generate a trajectory coding map; S4. Build a TEM-VGG model based on the pre-trained convolutional layer, input the trajectory coding map into the TEM-VGG model, obtain the probability values ​​corresponding to different upper limb movement posture categories, and complete the measurement of upper limb movement.

2. The method for measuring upper limb movement in rehabilitation training based on deep trajectory coding according to claim 1, characterized in that: In step S1, a MEMS sensor is bound to the trainee's forearm to collect data, and the motion data includes time step, acceleration, angular velocity and Euler angle; The motion information includes coordinate values, acceleration information, angular velocity information, time information and speed information.

3. The method for measuring upper limb movement in rehabilitation training based on deep trajectory coding according to claim 1, characterized in that: In step S1, the SHA-256 hash encryption algorithm and data desensitization technology are used to protect the data.

4. The method for measuring upper limb movement in rehabilitation training based on deep trajectory coding according to claim 1, characterized in that: In step S3, generating a trajectory coding map includes the following contents: The coordinate values ​​in the three-dimensional space are normalized and scaled, and mapped to a 128*128*128 three-dimensional matrix to obtain an intermediate trajectory encoding map; The specific expression of the position index of each three-dimensional coordinate point corresponding to the element in the three-dimensional matrix is: in, Represents the position index of the element in the three-dimensional matrix of the i-th time step, Max represents the maximum value, Represents the x-axis coordinate value of the i-th time step, Represents the y-axis coordinate value of the i-th time step, represents the z-axis coordinate value of the i-th time step, p x Represents the x-axis coordinate value in three-dimensional space, p y Represents the y-axis coordinate value in three-dimensional space, p z Represents the z-axis coordinate value in three-dimensional space; Based on the RGB color space, the motion features in space, time and direction are mapped to the three channels of the intermediate trajectory encoding map, where the corrected speed information is adapted to the blue channel, the time information is adapted to the green channel, and the direction information is adapted to the red channel to obtain the trajectory encoding map. The specific expression is: in, Indicates the speed information of the i-th time step is mapped to the pixel value of the blue channel, Indicates that the time information of the i-th time step is mapped to the pixel value of the green channel, Indicates that the direction information of the i-th time step is mapped to the pixel value of the red channel, ceil() represents the upward rounding function, and m represents a constant. represents the modulus of the velocity vector at the i-th time step, V represents the velocity list set, represents the direction vector norm of the i-th time step, A represents the set of direction data, T i represents the i-th time step, T total Indicates the total number of time steps; The direction corresponding to the angle between the y-axis and the gravity field is the direction information.

5. The method for measuring upper limb movement in rehabilitation training based on deep trajectory coding according to claim 1, characterized in that: In step S4, the measurement of upper limb movement is completed including the following: The TEM-VGG model includes an input layer, a convolution layer, a ReLU activation function layer, a maximum pooling layer, a batch normalization layer, a fully connected layer, and an output layer connected in sequence. The convolution layer includes a set number of convolution blocks, and each convolution block includes a set number of 3×3 convolution kernels stacked together. The trajectory coding map is input into the TEM-VGG model, transmitted to the convolution layer through the input layer, and the convolution block is used to extract features of the trajectory coding map to obtain a first feature map. The first feature map passes through the ReLU activation function layer and uses the ReLU function to perform nonlinear transformation processing to obtain a nonlinear feature map. The nonlinear feature map passes through the maximum pooling layer and uses the maximum pooling operation to perform downsampling processing to obtain a downsampled feature map. The downsampled feature map passes through the batch normalization layer and uses the batch normalization method to perform normalization processing to obtain a normalized feature map. The downsampled feature map passes through the fully connected layer for classification processing to obtain probability values ​​corresponding to different upper limb movement posture categories, and then the probability values ​​corresponding to different upper limb movement posture categories are output through the output layer.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method for measuring upper limb movement in rehabilitation training based on deep trajectory coding as described in any one of claims 1 to 5 are implemented.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for measuring upper limb movement in rehabilitation training based on deep trajectory coding according to any one of claims 1 to 5 is executed.

Images