Method for monitoring rehabilitation of meniscus injury patient based on calculation and simulation features

Abstract

The application discloses a method for monitoring rehabilitation of a meniscus injury patient based on calculation and simulation features, and relates to the field of gait analysis. The patient wears an inertial measurement unit and a plantar pressure sensing unit, collects relevant gait data, calculates actual relevant features, establishes a lower limb skeleton simulation model, and outputs corresponding simulation relevant features, and the actual and simulation data features are weighted and fused to expand the data set. The data set is used in a CNN-LSTM-MLP model to realize gait action recognition and prediction of the degree of meniscus injury, so as to realize the purpose of lower limb gait analysis and rehabilitation monitoring of the meniscus injury patient. The application is efficiently used in most existing gait analysis and monitoring technologies, and the gait data features of the patient are simulated and judged to determine the rehabilitation degree of the patient.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of gait analysis, and particularly relates to a meniscus injury patient rehabilitation monitoring method based on calculation and simulation characteristics. BACKGROUND

[0002] Walking is a kind of human action state based on muscles, bones and nervous system, and gait reflects the health degree and physical condition of human body to a certain extent. At present, most of the mainstream gait information collection devices are high in cost, large in size and complicated to use, so it is necessary to use a more portable and simple gait data collection device, and analyze the gait of the patient through a certain means, so as to achieve the purpose of judging the condition and rehabilitation of the injured person.

[0003] The meniscus is two crescent-shaped fibrocartilages located in the medial and lateral articular surfaces of the tibial plateau. Its main function is to maintain the stability of the knee joint and assist in transmitting body load. At present, meniscus injury has become one of the main causes of knee joint injury and has attracted widespread attention. Studies have shown that meniscus injury can cause the pelvis and lower limbs to be in an abnormal posture, thereby causing abnormal gait. Taking the gait analysis of meniscus injury patients as the research object, through the gait data collection device and the gait recognition and analysis method, the purpose of diagnosing meniscus injury can be effectively achieved. SUMMARY

[0004] There are methods for directly calculating related gait parameters according to sensor data on the market, but the reliability of the results obtained by only a single calculation method and a machine learning model is difficult to verify. In view of the problems and deficiencies existing in the prior art, the application provides a meniscus injury patient rehabilitation monitoring method based on calculation and simulation characteristics, and a meniscus injury patient gait analysis method realized by using a CNN-LSTM-MLP model. The actual test results show that the simulation data can realize the supplementary fusion and perfection of the data, improve the accuracy and reliability of the machine learning gait feature input, and cooperate with the determination of the rehabilitation degree, so as to provide a feasible solution for gait recognition and evaluation.

[0005] To achieve the above purpose, the technical scheme adopted by the application is as follows:

[0006] The meniscus injury patient rehabilitation monitoring method based on calculation and simulation characteristics comprises the following steps:

[0007] S1: establishing a human lower limb gait data collection system, the collection device comprising five inertial measurement units (IMUs) and a plantar pressure sensor, for collecting lower limb gait data of a subject;

[0008] S2: performing data processing and related filtering algorithm on the gait data obtained in S1 to obtain the actual calculated knee joint angle l1 , and r1and knee joint torque data M l1 , M r1 ;

[0009] S3: According to the gait data obtained in S1, an IMU-based simulation model of the lower limbs of the subject is established, and inverse kinematics processing and inverse dynamics processing are performed on the model to obtain simulation-processed knee joint angle l2 , theta r2 and knee joint torque data M l2 , M r2 ;

[0010] S4: The angle and torque calculation and simulation data obtained in S2 and S3 are periodically weighted data fusion to obtain a new data vector, which expands, corrects and perfects the training data set.

[0011] S5: For the gait data obtained in S4, a CNN-LSTM-MLP model is established to realize gait action recognition and prediction of meniscus injury degree, and finally the cosine similarity of the gait data vector is used to judge the rehabilitation degree of abnormal gait.

[0012] As a further improvement of the application, in step S3, the IMU data in S1 is converted into data available for the simulation model, the Euler angle data of the IMU is used to initialize the quaternion and convert it into a rotation matrix, a connection interface between the simulation software and matlab is set, the IMU data is used to establish a human lower limb bone simulation model, a plurality of scale factors are added to the model to achieve the purpose of model scaling, and the inverse kinematics processing and inverse dynamics processing are performed.

[0013] As a further improvement of the application, the angle and torque data obtained in S2 and S3 are periodically weighted data fusion, the similarity of each group of secondary period data samples is calculated using the Euclidean distance, and the weight is assigned respectively to reflect the contribution degree to the final gait parameter result, the feature fusion is performed, the obtained data vector is added to the IMU original data vector, and the IMU training data set is expanded.

[0014] As a further improvement of the application, the CNN-LSTM-MLP model in step S5 adopts the CNN-LSTM model to obtain the IMU data time sequence information and extract the data features, and performs action classification; the MLP model is used to obtain the function relationship between different action input and final meniscus injury degree, considering that there is a connection between the outputs of the two models, the result feedback is added and corrected by using the weighting idea.

[0015] As a further improvement of the present invention, after step S5 obtains the data of normal gait and abnormal gait, since the gait data characteristics belong to the data matrix, it is regarded as the superposition of the data vectors of each time frame. According to the cosine similarity of the vectors, the cosine similarity of each row vector of the normal matrix and the matrix to be tested is calculated respectively, so as to obtain the rehabilitation status characteristics.

[0016] Compared with the prior art, the present invention has the following beneficial effects:

[0017] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects: the present invention proposes a CNN-LSTM-MLP model, which uses the fusion of multiple algorithms to extract time series information and data features to separate types, and then predicts and estimates meniscus injury, which can effectively improve the performance of the motion recognition method, which is in sharp contrast to the traditional appearance-based or model-based methods, enabling the network to obtain richer information for characterization learning; inverse kinematics and inverse dynamics algorithms make posture solution and data processing simpler and more feasible, reducing possible errors; and a weighted fusion algorithm is used to normalize experimental calculation data and simulation data to improve the reliability of training data. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a process diagram of a method for monitoring rehabilitation of meniscus injury patients based on calculation and simulation features proposed by the present invention;

[0019] Figure 2 Schematic diagram of the weighted fusion algorithm of the present invention;

[0020] Figure 3 This is the framework diagram of the CNN-LSTM-MLP model of the present invention;

[0021] Figure 4 It is the algorithm flow chart of the present invention. DETAILED DESCRIPTION

[0022] The technical solution of the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0023] like Figure 1 The process diagram is shown as follows: Figure 4 As shown in the algorithm flow chart, the rehabilitation monitoring method for meniscus injury patients based on calculation and simulation features is characterized by the following specific steps:

[0024] Step 1, establish the human lower limb gait data acquisition system. The lower limb gait data acquisition system contains IMU nine-axis inertial measurement unit and plantar pressure sensor, using the scattered data acquisition method, five IMU elements are worn on the pelvis, left / right thigh and left / right lower leg of the subject, and the pressure sensor is placed on the bottom of the subject's foot or inside the shoe. The gait data collected includes nine-axis acceleration, angular velocity, magnetic force value, attitude angle, Euler angle, quaternion, plantar pressure and other data. Before testing, calibrate the inertial measurement unit and pressure sensor unit to reduce zero drift. The gait data obtained in S1 is processed and filtered to obtain the actual calculation of the knee joint angle θ l1 、θ r1 and knee joint torque data M l1 、M r1 .

[0025] Step 2, data conversion and establishment of simulation model, the IMU data in step 1 is converted into data available for the simulation model, this process is carried out using Python, mainly using the functions in the library scipy to achieve. First, get the Euler angle data of IMU, initialize the quaternion using the four data of Euler angle and the function from_quat(), then convert the quaternion to a rotation matrix using as_matrix() in scipy, finally convert the nine parameters of the rotation matrix into n rows and nine columns and write them into an excel file, then splice the acceleration data in IMU with the parameter data of the rotation matrix. Use the open source software OpenSim in biomechanics to establish a simulation model of human lower limb skeleton. Since the default standard skeleton model is generally inconsistent with the subject's body parameter data, scaling operation is needed, that is, adding multiple scale factors to the static upright human skeleton model and scaling, adjusting the coordinates of the marker points throughout the model, so that the height and weight of the model match the actual subject, and finally realize the scaling of the model. Perform inverse kinematics and inverse dynamics processing on the scaled simulation model to obtain knee joint angle and knee joint torque data. Inverse kinematics processing is used to obtain knee joint angle data, which uses weighted least squares optimization problem to minimize the angle marker error, in actual operation, the input model file, sensor direction file in quaternion and setting file can be used to get the output motion result; inverse dynamics processing is used to obtain knee joint torque data, based on the left and right knee joint angle data and the ground reaction force data obtained during the experiment, the net reaction force and net torque of the left and right knee joints are calculated according to the dynamic balance condition and boundary condition. In the actual operation process, input the motion file containing the time history of the generalized coordinates describing the model motion, external load data (i.e. ground reaction force, torque and center of pressure position) and scaled human model, and get the data output. Thus the simulation processed knee joint angle θ l2 、θr2 and knee joint torque data M l2 、M r2 ;

[0026] Step 3: For the data obtained in step 1 and step 2, use θ l1 and θ l2 For example, there are two methods to calculate the near-periodic primary sequence of the left knee joint angle changing with time. l1 and θ l2 Extract and segment into t sub-sequences respectively. For each sub-sequence, there are n frames in total, let i be the number of time frames, θ l1i and θ l2i That is, the joint angle data parameter of the i-th time frame. In order to express the similarity of each secondary sequence, let the Euclidean distance Where j indicates the secondary sequence, and the Euclidean distance of t secondary sequences is calculated. j The smaller the value, the closer the calculated angle is to the simulated angle. The more credible the data of the secondary sequence is, the greater the weight should be. j =1 / Get the final left knee joint gait parameter sequence: Similarly, the remaining three groups of right knee joint angle and knee joint torque gait parameter data are processed in the same way to obtain: After feature fusion, the obtained data vector is added to the IMU original data vector in parallel to expand the IMU training data set. The weighted fusion algorithm diagram is shown in the figure. Figure 2 shown.

[0027] Step 4: Use the CNN-LSTM-MLP model to perform action recognition and meniscus injury degree prediction on the output data. The CNN-LSTM-MLP model framework is shown in the figure below. Figure 3As shown, first through a layer of pretreatment, and then through the LSTM layer to obtain timing characteristics, assuming the input data timing length is T+1, then using T LSTM units to obtain information. The information obtained as input into the CNN layer again. The CNN layer has multiple convolution layers, multiple pooling layers and fully connected layers. The number of convolution kernels and the parameters of the convolution layer need to be optimized through experiments. The maximum pooling layer is used for pooling. The convolution layer to the fully connected layer uses the Softmax function to obtain a 1x6 vector {y(1), y(2), y(3), y(4), y(5), y(6)}, where y(1), y(2), y(3), y(4), y(5), y(6) represent an action respectively. The importance of the five test point data is analyzed, and the important features are selected as the input of the MLP and y(1), y(2), y(3), y(4), y(5), y(6) as the output of the function relationship with the output meniscus injury degree. The cross-entropy is used as the loss function for the CNN-LSTM part: CNN-LSTM = -∑ i Y' i log(Y i ), where Y i is the estimated gait pattern probability distribution, and Y' i is the actual gait pattern probability distribution. The Adam optimizer algorithm can be used to minimize the cross-entropy. The MSE can be used as the loss function for the MLP part: where y i is the actual value, is the predicted value. During parameter optimization, the local gradient descent method is used. Finally, output 2 and output 1 should have an interaction relationship, so output 2 should also be used as one of the conditions when setting the model parameters. A new parameter M_total is added to feed the result m of output 2 to the input of CNN-LSTM. The update operation of M_total and m is as follows: M_total = m*(1-w) + M_total*w, where w is the weight value, which determines the influence of M_total before updating in this update, and the value is [0, 0.5]. The purpose of this design is to make M_total have certain memory.

[0028] Step 5, based on the cosine similarity of the gait data vector to determine the rehabilitation degree of abnormal gait. After obtaining the normal gait and abnormal gait data, since the gait data features belong to data matrices, they are regarded as the superposition of each time frame data vector. Assuming that each type of gait data has m time frames, each gait data matrix contains m gait data vectors. For each gait data vector, let the vector obtained by using the standard reference normal gait data in the gait database be V nThe vector of gait data of the meniscus injury subject to be evaluated for the rehabilitation degree is set as V h According to the cosine similarity The cosine similarity of each row vector of the two matrices is calculated according to the above formula. In order to intuitively display the gait rehabilitation results, the cosine similarity results are processed by +1, and then the average value of each vector of the gait data matrix is taken. Let the gait rehabilitation degree be R After calculation, if R=0, it means that the two gaits are completely dissimilar, and the meniscus injury is serious. If R=1, it means that the two gaits are completely the same, that is, the gait is completely rehabilitated, which is consistent with the normal gait behavior characteristics. Thus, the meniscus injury patient rehabilitation monitoring method based on calculation and simulation characteristics is completed.

[0029] The above is only a preferred embodiment of the present application, not any other form of limitation on the present application, and any modification or equivalent change made according to the technical essence of the present application still falls within the scope of the present application.

Claims

1. A method for monitoring the rehabilitation of patients with meniscus injuries based on computational and simulation features, characterized by: The steps include: S1: Establish a human lower limb gait data acquisition system. The acquisition device includes five inertial measurement units (IMUs) and plantar pressure sensors to collect the subjects' lower limb gait data. S2: Process the gait data obtained in S1 and perform related filtering algorithms to obtain the actual calculated knee joint angle θ l1 ,θ r1 and knee joint torque data M l1 、M r1 ; S3: Based on the gait data obtained in S1, a simulation model of the human lower limb skeleton of the subject based on IMU is established. The model is processed by inverse kinematics and inverse dynamics respectively to obtain the simulated knee joint angle θ l2 ,θ r2 and knee joint torque data M l2 、M r2 ; S4: Perform weighted data fusion on the angle and torque calculations and simulation data obtained in S2 and S3 respectively according to the period to obtain new data vectors, and expand, correct and improve the training data set; S5: Based on the gait data obtained in S4, a CNN-LSTM-MLP model is established to realize gait movement recognition and prediction of the degree of meniscus injury. Finally, the cosine similarity of the gait data vector is used to determine the degree of recovery of abnormal gait.

2. The method for monitoring rehabilitation of patients with meniscus injuries based on calculation and simulation features according to claim 1, characterized in that: In order to realize data conversion and simulation model establishment, in step S3, the IMU data in S1 is exported into data that can be used by the simulation model, the quaternion is initialized using the Euler angle data of the IMU and converted into a rotation matrix, the connection interface between the simulation software and MATLAB is set up, and the IMU data is used to establish a simulation model of the human lower limb skeleton. Multiple scale factors are added to the model to achieve the purpose of model scaling for subsequent inverse kinematics processing and inverse dynamics processing.

3. The method for monitoring rehabilitation of patients with meniscus injuries based on calculation and simulation features according to claim 1, characterized in that: The angle and torque data obtained in S2 and S3 are weightedly fused according to the cycle. The similarity of each group of secondary cycle data samples is calculated using Euclidean distance and weighted respectively to reflect the contribution to the final gait parameter results. Feature fusion is performed and the obtained data vectors are added to the IMU original data vector in parallel to expand the IMU training data set.

4. The method for monitoring rehabilitation of patients with meniscus injuries based on calculation and simulation features according to claim 1, characterized in that: The CNN-LSTM-MLP model in step S5 uses the CNN-LSTM model to obtain the IMU data timing information and extract data features for motion classification. The MLP model is used to obtain the functional relationship between different motion inputs and the final meniscus injury degree. Considering the connection between the outputs of the two models, the weighted idea is used to add result feedback and make corrections.

5. The method for monitoring rehabilitation of patients with meniscus injuries based on calculation and simulation features according to claim 1, characterized in that: After obtaining the data of normal gait and abnormal gait in step S5, since the gait data characteristics belong to the data matrix, they are regarded as the superposition of the data vectors of each time frame. According to the cosine similarity of the vectors, the cosine similarity of each row vector of the normal matrix and the matrix to be tested is calculated respectively to obtain the recovery status characteristics.

Images