A method, system, device and medium for recognizing phase of crawling motion

Abstract

The invention discloses a crawling motion phase recognition method, system, device and medium, and relates to the technical field of biological signal recognition. The invention collects acceleration data and periodic pressure data of each joint in the crawling motion of a human body, uses the multi-joint collaborative motion theory and the principal component analysis method based on SVD to extract the multi-joint motion collaborative phase pattern in the crawling motion, separates the time recruitment signal, performs EMD decomposition and secondary reconstruction on the time recruitment signal to form a time recruitment curve, and then fuses the manually extracted time domain features and the features extracted by 1D-CNN to obtain the phase of the crawling motion; the invention introduces the multi-joint collaborative motion theory and the corresponding model into phase recognition, and at the same time, the collaborative phase pattern recruitment curve after secondary reconstruction is used as the object for feature extraction in phase recognition, so as to recognize the phase from the multi-joint motion coordination that is more in line with the human body motion control law, so that the phase recognition accuracy of the crawling motion is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of biological signal recognition, and in particular to a crawling motion phase recognition method, system, equipment and medium. Background Art

[0002] For children with cerebral palsy who have motor dysfunction caused by brain damage, their rehabilitation training usually includes comprehensive rehabilitation medicine, drug therapy, traditional Chinese medicine therapy and exercise therapy. Studies have shown that exercise therapy often has unexpected therapeutic effects. Exercise therapy is based on kinematics and neurophysiology, and uses instruments or the therapist's manual techniques or the child's own strength to restore and treat whole-body and local functions through active or passive movements. Crawling is an important training method for motor rehabilitation of children with cerebral palsy. Studies have shown that crawling has been proven to effectively promote the development of brain function in children with cerebral palsy, and it also has a positive effect on improving the stability of cerebellar movement.

[0003] Since crawling is a gross motor behavior involving alternating movements of the limbs, repetitive crawling training can effectively improve the limb coordination ability of patients with movement disorders; at the same time, long-term crawling training can stimulate the patient's neuromuscular system, thereby stimulating the recovery and reconstruction of their neuromuscular function, and ultimately improving the efficacy of motor rehabilitation; in a study on rehabilitation training for children with cerebral palsy, it was found that the treatment group combined with crawling training had significantly better rehabilitation effects than the control group using conventional rehabilitation programs; therefore, crawling has a positive role in promoting the recovery of motor function in patients with brain injury. For this reason, in recent years, many rehabilitation training devices with crawling training as the main function and form have appeared. Such devices usually use motor drive to passively guide patients to crawl, thereby achieving the effect of motor function rehabilitation training.

[0004] However, in actual application, it is impossible to accurately identify different movement phases of the limbs during crawling, such as landing, leaving the ground, and swinging. As a result, this method will cause a "drag feeling" to patients with motor function during rehabilitation training, and this method of control based on a predefined crawling pattern ignores the initiative and enthusiasm of human movement. At present, the identification of movement phases usually adopts image recognition or sensor measurement and recognition. Image recognition is to identify the movement direction of the human body by taking real-time movement images of the human body, but this recognition method has low accuracy and is not real-time. Sensor measurement and recognition is to judge the movement of the human body by placing sensors at different positions of the human body and then measuring the swing amplitude of different parts of the human body. However, this measurement method only focuses on the swing amplitude of the human body, and can often only identify the movement phase of the human body in a certain direction when it is moving. It is difficult to identify the multi-faceted and deep-level movement phases of the human body during crawling, which makes the phase recognition accuracy of crawling movement low. Summary of the invention

[0005] The embodiments of the present invention provide a method, system, device and medium for recognizing the phase of crawling motion, which can solve the problem in the prior art that the current recognition methods can only recognize the unilateral shallow motion phase of the human body during movement, and it is difficult to recognize the multi-faceted and deep motion phase of the human body during the crawling motion process, resulting in low accuracy in phase recognition of crawling motion.

[0006] The embodiment of the present invention provides a method for recognizing a crawling motion phase, comprising the following steps:

[0007] Collect acceleration data of each limb joint and pressure data of each limb during crawling movement;

[0008] The crawling motion process is divided into multiple crawling cycles, and the pressure data of each crawling cycle and the acceleration data of each crawling cycle are obtained;

[0009] The posture matrix of each limb joint is constructed according to the acceleration data of each crawling cycle. The principal component analysis method of singular value decomposition (SVD) is used to extract the motion coordination phase pattern of each limb joint in the posture matrix. The time recruitment signal reflecting the timing information of the human neuromuscular system regulation is separated from the motion coordination phase pattern. The extracted time recruitment signal is subjected to EMD decomposition and secondary reconstruction to obtain the time recruitment curve after secondary reconstruction.

[0010] Using the pressure data corresponding to the multiple crawling stages divided in each crawling cycle, the coordinated phase pattern of each limb joint movement is divided into multiple crawling movement phases;

[0011] The time domain feature data set of the time recruitment curve after secondary reconstruction in multiple crawling motion phases of each crawling cycle was extracted, and the ReliefF method was used to screen the time domain feature data set to obtain the first feature data set whose acceleration data weight coefficient was higher than the set threshold; the one-dimensional convolutional neural network 1D-CNN was used to extract the second feature data set of acceleration data in multiple crawling motion phases of each crawling cycle, and the first feature data set and the second feature data set were horizontally spliced ​​to form the final feature data set;

[0012] The final feature data set is classified and identified using a multi-class support vector machine (SVM) to obtain the phase of the crawling motion.

[0013] Preferably, the acquisition of acceleration data of joints of each limb of the human body and pressure data of each limb during the crawling movement includes:

[0014] The acceleration sensors are respectively set at the wrist joints, elbow joints, shoulder joints, knee joints and hip joints of the upper and lower limbs of the human body to collect the acceleration data of each limb joint during the crawling movement of the human body;

[0015] The pressure sensors are placed on each palm of the human body to collect the pressure data of each limb during the crawling movement of the human body.

[0016] Preferably, the acquisition of acceleration data of each crawling cycle includes:

[0017] The first and last time points of the pressure data of each crawling cycle are respectively corresponded to the time points of the acceleration, and the acceleration data of each crawling cycle in the acceleration data is identified and divided.

[0018] Preferably, the acquisition of the coordinated phase pattern of the limb joint movements includes:

[0019] Each crawling cycle contains the acceleration data at the left wrist, left elbow, left shoulder, left knee, right wrist, right elbow, right shoulder, right knee and hip joints. The acceleration data of each joint is set as the posture matrix V , the posture matrix is ​​extracted using principal component analysis based on singular value decomposition (SVD) V The posture coordination matrix in W and time recruitment matrix H , to form a coordinated phase pattern of movement of each limb joint; its posture matrix V It is expressed as:

[0020] ;

[0021] in: Indicates the number of joint acceleration data; Represents the number of principal components obtained after matrix decomposition; Indicates time;

[0022] When the cumulative explained variance of the principal components is higher than 90%, the number of principal components selected is and time recruitment matrix H The time recruitment signal represents the timing information of the human neuromuscular system regulation. Considering that the interference of neural noise will affect the quality of the time recruitment signal and subsequent phase recognition, the extracted time recruitment signal is subjected to EMD decomposition and secondary reconstruction to obtain the time recruitment curve after secondary reconstruction. The time recruitment curve after secondary reconstruction is used as the feature extraction object for subsequent phase recognition.

[0023] Preferably, the limb joint motion coordination phase pattern is divided into a plurality of crawling motion phases, including:

[0024] Each complete crawling cycle is divided into four crawling stages, namely S1, S2, S3 and S4; S1 represents the stage of left hand swinging and right hand supporting during crawling, S2 represents the stage of left hand and right hand supporting, S3 represents the stage of left hand supporting and right hand swinging, and S4 represents the stage of left hand and right hand supporting again;

[0025] According to the four crawling stages, the pressure data in each crawling cycle is divided into pressure data corresponding to the four crawling stages;

[0026] According to the correspondence between the pressure data in each crawling cycle and the data points of the coordinated phase pattern of each limb joint movement, the coordinated phase pattern time recruitment curve is divided into four phases using the pressure data corresponding to the four crawling stages divided in each crawling cycle.

[0027] Preferably, the acquisition of the final feature data set includes:

[0028] The peak value, peak-to-peak value, mean value, standard deviation, root mean square value and slope in the four phases of each crawling cycle are extracted to form the time domain feature data set M1. If the motion input of the segment joint acceleration is indivual;

[0029] The ReliefF method is used to perform feature screening on the time domain feature data set M1, screen the weight coefficients of the features in the time domain feature data set M1, and evaluate the importance of each feature to form a feature data set M2;

[0030] The four phases of each crawling cycle are input into a one-dimensional convolutional neural network 1D-CNN for feature extraction. The input layer size is 30×27. After two convolutional pooling layers and one ReLU activation layer, the feature data is flattened into one-dimensional data to obtain the feature data set M3.

[0031] The feature data set M2 and the feature data set M3 are horizontally spliced ​​to form the final feature data set.

[0032] Preferably, the step of obtaining the phase of the crawling motion comprises:

[0033] The classifiers in the multi-class support vector machine SVM are trained in a one-to-one manner, and a total of 6 classifiers are trained according to the 4 phase categories. The kernel function in the training uses a linear kernel, a polynomial kernel, and a Gaussian kernel to obtain the trained multi-class support vector machine SVM;

[0034] The final feature data set is input into the trained multi-class support vector machine (SVM) for classification and recognition to obtain the phase of the crawling movement.

[0035] The embodiment of the present invention further provides a crawling motion phase identification system, comprising:

[0036] Kinematic data signal acquisition module, used to collect acceleration data of joints of each limb of the human body and pressure data of each limb during crawling movement;

[0037] A collaborative phase pattern extraction module is used to divide the crawling motion process into multiple crawling cycles, and obtain the pressure data of each crawling cycle and the acceleration data of each crawling cycle;

[0038] The posture matrix of each limb joint is constructed according to the acceleration data of each crawling cycle. The principal component analysis method of singular value decomposition (SVD) is used to extract the motion coordination phase pattern of each limb joint in the posture matrix. The time recruitment signal reflecting the timing information of the human neuromuscular system regulation is separated from the motion coordination phase pattern. The extracted time recruitment signal is subjected to EMD decomposition and secondary reconstruction to obtain the time recruitment curve after secondary reconstruction.

[0039] A coordinated phase pattern phase division module is used to divide the coordinated phase pattern of each limb joint movement into a plurality of crawling movement phases using the pressure data corresponding to the plurality of crawling stages divided in each crawling cycle;

[0040] A multi-feature extraction module is used to extract a time-domain feature data set of the time recruitment curve after secondary reconstruction in multiple crawling motion phases in each crawling cycle, and use the ReliefF method to screen the time-domain feature data set to obtain a first feature data set whose acceleration data weight coefficient is higher than a set threshold; use a one-dimensional convolutional neural network 1D-CNN to extract a second feature data set of acceleration data in multiple crawling motion phases in each crawling cycle, and horizontally splice the first feature data set and the second feature data set to form a final feature data set;

[0041] The phase recognition module is used to classify and recognize the final feature data set using a multi-category support vector machine (SVM) to obtain the phase of the crawling motion.

[0042] An embodiment of the present invention further provides an electronic device, including a memory and a processor;

[0043] The memory is used to store computer programs;

[0044] The processor is used to implement the steps of the above-mentioned method for identifying the phase of crawling movement when executing the computer program stored in the memory.

[0045] An embodiment of the present invention further provides a computer-readable storage medium for storing a computer program, which, when executed by a processor, implements the steps of the crawling motion phase identification method as described above.

[0046] The embodiments of the present invention provide a method, system, device and medium for recognizing a crawling motion phase. Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0047] The present invention collects acceleration data of each joint of the upper and lower limbs and periodic pressure data during the crawling process of the human body through multiple sensors, extracts the multi-joint motion cooperative phase mode during the crawling process by using the multi-joint cooperative motion theory and the principal component analysis method based on singular value decomposition (SVD), separates the multi-joint time recruitment curve in the multi-joint motion cooperative phase mode, and further performs EMD decomposition and secondary reconstruction on the extracted time recruitment signal to obtain the time recruitment curve after secondary reconstruction. In this stage, the multi-joint time recruitment curve after secondary reconstruction of multiple joints during the crawling movement is extracted, and then the time domain features manually extracted from the multi-joint motion input cooperative phase mode and the features extracted by 1D-CNN are fused, and the phase of the crawling movement is obtained after classification and recognition. In this process, the multi-joint cooperative motion theory and the corresponding model are introduced into the phase recognition, and the cooperative phase mode recruitment curve after secondary reconstruction is used as the object for feature extraction in the phase recognition, so as to recognize the phase of the multi-joint motion coordination from multiple aspects and deeper levels, so that the phase recognition accuracy of the crawling movement is greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of the overall process of a crawling motion phase identification method provided by an embodiment of the present invention;

[0049] Figure 2 A schematic diagram of sensor module signal transmission of a crawling motion phase identification method and system provided by an embodiment of the present invention;

[0050] Figure 3 A schematic diagram of crawling phase division of a crawling motion phase identification method provided by an embodiment of the present invention;

[0051] Figure 4 A schematic diagram of motion coordination phase pattern extraction of a crawling motion phase identification method provided by an embodiment of the present invention;

[0052] Figure 5 A schematic diagram of phase division of a motion coordination phase pattern of a crawling motion phase identification method provided by an embodiment of the present invention;

[0053] Figure 6 A schematic diagram of a flow chart of secondary reconstruction of a time recruitment curve of a crawling motion phase identification method provided by an embodiment of the present invention;

[0054] Figure 7A schematic diagram of a secondary reconstruction process of a time recruitment curve of a cooperative phase pattern based on EMD in a crawling motion phase identification method provided by an embodiment of the present invention;

[0055] Figure 8 A schematic diagram of time domain feature extraction and screening of a crawling motion phase identification method provided by an embodiment of the present invention;

[0056] Fig. 9 A schematic diagram of 1D-CNN feature extraction of a crawling motion phase recognition method provided by an embodiment of the present invention;

[0057] Fig.10 A schematic diagram of feature fusion of a crawling motion phase recognition method provided by an embodiment of the present invention;

[0058] Fig.11 A schematic diagram of the accuracy of a training set and a test set after accumulatively training 10 times under the condition of directly extracting features according to a method for recognizing the phase of a crawling motion provided by an embodiment of the present invention;

[0059] Fig.12 A schematic diagram of the accuracy of a training set and a test set after accumulating 10 training times under the condition of extracting a coordinated phase pattern and then extracting features in a crawling motion phase recognition method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention, so the present invention is not limited by the specific embodiments disclosed below.

[0061] See also Figure 1 The embodiment of the present invention provides a crawling motion phase recognition method, specifically a motion phase recognition method and system for compliant control of a crawling rehabilitation device for children with cerebral palsy, which mainly realizes the acquisition and storage functions of acceleration signals and pressure signals of upper and lower limb joints of the human body during crawling motion, and realizes phase recognition of motion coordination phase mode, mainly including:

[0062] 1. Kinematic data signal acquisition.

[0063] The MUP6050 sensors were placed on the wrist joints (ulnar styloid processes), elbow joints (upper part of the humerus near the lateral epicondyle), shoulder joints (intertubercular groove), knee joints (near the patellar femoral surface), and hip joints (posterior superior iliac spine) of the volunteers in the crawling experiment. They were mainly used to collect acceleration signals of the upper and lower limb joints during crawling. The thin-film pressure sensor was placed on the right palm of the volunteers in the crawling experiment to collect pressure signals during crawling, which were used to divide the collected acceleration signals into periods and phases later. Each MPU6050 sensor and thin-film pressure module were integrated with the ESP-01S WIFI communication module, and then connected to the router. Finally, the collected joint triaxial acceleration signals and pressure signals were transmitted to the PC in real time for storage, as shown in Figure 2. Figure 2 shown.

[0064] 2. Data preprocessing.

[0065] Since the saved three-axis acceleration data is the output value of the IMU sensor, it needs to be converted into actual physical quantities. The accelerometer measures the acceleration of an object in each axis, and its output value is usually a digital signal proportional to the acceleration. Assume that the range of the accelerometer is ( is the acceleration due to gravity, about 9.8m / s 2 ), the output data is Data (data high eight bits << 8 | data low eight bits), the actual acceleration The calculation formula is , where n is the number of bits of output data. So when the range is , when the data is 16 bits, Then, the three-axis acceleration signal and pressure signal are filtered using a fourth-order low-pass Butterworth filter with a denoising and cutoff frequency of 20Hz; the time from the limb leaving the ground to the limb leaving the ground again is a crawling cycle. Taking the left hand as an example, the corresponding left hand departure point pressure threshold is set to 1, and the pressure signal of the crawling movement is divided into multiple complete crawling cycle pressure signals, and the first and last time points of each cycle pressure signal correspond to the time points of the collected joint three-axis acceleration, so as to obtain the corresponding single cycle joint three-axis acceleration signal data, and finally the single cycle pressure signal and the single cycle joint three-axis acceleration signal are normalized and re-interpolated into 101 data points (corresponding to 0% to 100% cycle). And a complete crawling stage can be divided into four stages, namely S1, S2, S3, and S4; such as Figure 3As shown in the figure, S1 represents the left hand swinging and right hand supporting stage during crawling; S2 represents the left hand and right hand supporting stage; S3 represents the left hand supporting and right hand swinging stage; S4 represents the left hand and right hand supporting stage again; according to the corresponding relationship between the divided single cycle pressure and the single secondary reconstructed time recruitment curve data point, the extracted secondary reconstructed time recruitment curve is divided into four corresponding phases: TS1, TS2, TS3, TS4 using the four stages of the single cycle pressure subdivision in the crawling process; thus, the secondary reconstructed time recruitment curve of each cycle is divided into four phases, each phase is classified into one category, and there are four types of data sets in total, such as Figure 5 shown.

[0066] 3. Extraction of coordinated phase patterns during crawling movement cycle.

[0067] The present invention adopts the principal component analysis method based on SVD, which can represent a relatively complex original data matrix by multiplying several smaller and simpler sub-matrices. These sub-matrices describe the important characteristics of the matrix. If X represents the number of variables n The data set obtained by performing m observations is:

[0068] .

[0069] Then the PCA implementation is as follows:

[0070] After centering the data set X, the singular value decomposition can be obtained:

[0071] .

[0072] Where: X represents the original matrix of a single action; represents the mean matrix for centering X; Represents a principal component of the action, i.e., the coordinated phase mode; the eigenvector The postural synergy relationship representing the action reveals the linear proportional relationship between joint accelerations during the movement; Represents the matrix ( ) singular values; vector Recruitment signal representing the time required for joint movement within a movement cycle; Indicates the number of data points in a single crawling cycle; Indicates the number of joint three-axis acceleration data.

[0073] First, the coordinated phase pattern of the joint triaxial acceleration signal data of a single cycle is extracted, such as Figure 4As shown in the figure, each cycle contains the three-axis acceleration data of 9 joints, including the left wrist, left elbow, left shoulder, left knee, right wrist, right elbow, right shoulder, right knee, and hip joint. Therefore, the three-axis acceleration signal data of each joint is a 101×27 data matrix, which is called the posture matrix V. The principal component analysis method based on SVD is used to extract the coordinated phase pattern achieved by the upper and lower limbs, and the posture coordination relationship matrix is ​​extracted from the original matrix V and time recruitment matrix , as shown below:

[0074] .

[0075] in: Indicates the number of joint triaxial acceleration data (set in the present invention ); Represents the number of principal components obtained after matrix decomposition; Indicates time (standardized as a fixed crawling cycle in the present invention, i.e., 0% to 100%); It means that the original matrix V is decomposed into principal components (i.e., collaborative phase patterns), each of which contains part of the information in the original posture matrix in order from high to low, and its importance is judged by the size of the explained variance.

[0076] When the cumulative explained variance of the principal components exceeds 90%, it indicates that the number of principal components selected is too large. Or the selected temporal recruitment can represent the main movement trajectory mode of the movement, because the temporal recruitment matrix It is the embodiment of the behavior of movement in time and space, so it is used as the main data for subsequent phase division. Each time recruitment matrix is ​​101× The matrix form contains Time recruitment curves of three-axis acceleration data.

[0077] 4. Secondary reconstruction of the temporal recruitment curve of the cooperative phase pattern.

[0078] Considering the extracted synergistic phase pattern time recruitment curve that reflects the timing information of human neuromuscular system regulation There may be interference such as neural noise that will affect the quality of the signal and the effect of subsequent phase recognition, so EMD decomposition is used and secondary reconstruction is performed to remove each The role of internal interference information in the secondary reconstruction stage is to extract the time recruitment matrix of the cooperative phase pattern Each time recruitment curve in Perform EMD decomposition to obtain One IMF frequency signal, repeatedly update the number n (n < k) of the selected IMF frequency signals to perform a secondary reconstruction on the time recruitment curve, thereby forming a time recruitment matrix after the secondary reconstruction, and calculate the original time recruitment matrix and the time recruitment matrix after the secondary reconstruction of the reconstruction error to determine the finally obtained time recruitment matrix The core of EMD decomposition is the sifting process, and IMFs are extracted through iteration. As Figure 6 shown in the schematic diagram of the secondary reconstruction process of the time recruitment curve, as Figure 7 shown in the schematic diagram of the secondary reconstruction of the time recruitment curve of the cooperative phase pattern based on EMD. Specifically, taking the first time recruitment curve signal as an example, the sifiting process can be expressed as:

[0079] 1. Calculate the upper envelope and the lower envelope of the signal .

[0080] : Use spline interpolation to connect all local maximum points to form the upper envelope of the signal.

[0081] : Similarly, use spline interpolation to connect all local minimum points to form the lower envelope of the signal.

[0082] 2. Calculate the mean line : .

[0083] 3. Extract the detail component : .

[0084] 4. Check whether it meets the conditions of IMF. If it meets, then is an IMF; otherwise, take as the new signal and continue to iterate the above steps.

[0085] The overall process is expressed as: .

[0086] Among them, is the signal of the k-th iteration, is the corresponding mean line.

[0087] The Sifting process needs to be terminated under a certain condition to avoid infinite iteration. Commonly used termination conditions include:

[0088] 1. Number of extreme points: The extracted component There are a sufficient number of maxima and minima.

[0089] 2. Change of mean line: In two consecutive iterations, the change of the mean line is less than the preset threshold ,Right now: .

[0090] 3. Orthogonality of IMF: The inner product of the extracted IMF and the residual signal is close to zero.

[0091] By meeting any of the above conditions, the sifting process can be terminated, ensuring the effectiveness and independence of the IMF.

[0092] 5. Multi-feature extraction.

[0093] The multi-feature extraction module is divided into manual extraction of time domain features, ReliefF screening of time domain features and 1D-CNN extraction of features; specifically:

[0094] 1. Manually extract time domain features.

[0095] The four phases of the time recruitment curve of the coordinated phase pattern of each cycle above are divided into peak, peak-to-peak, mean, standard deviation, root mean square, slope and other time domain features. Since each phase contains time recruitment curves, so each time domain feature of each phase will be extracted The formula is:

[0096] Peak: .

[0097] Peak-to-peak value: .

[0098] Mean: .

[0099] Standard Deviation: .

[0100] RMS: .

[0101] Slope: .

[0102] 2. ReliefF screens time domain features.

[0103] Through manual time domain feature extraction, the peak value, peak-to-peak value, mean value, standard deviation, root mean square, slope, etc. of the time recruitment curve of the coordinated phase pattern can be obtained for each phase. characteristic values; Figure 8As shown in the figure, the phases of all extracted features are combined into a phase sample set and the phase labels 1, 2, 3, and 4 are added after the corresponding samples to form a time domain feature set M1. Then, the ReliefF algorithm is used for feature screening. Each time, a sample R is randomly selected from all training sample sets, and then the k nearest neighbor samples of R are found from the sample sets of the same type as R, and the k nearest neighbor samples are found from the sample sets of different types of each R. Then, the weight of each feature is updated. According to the weight size, the importance of the feature for classification can be compared, and the appropriate weight threshold is set to screen the features. Finally, the phase sample set M2 after screening the features is obtained.

[0104] 3. 1D-CNN extracts features.

[0105] Unify the time step of each phase, such as Fig. 9 As shown, it is then input into a one-dimensional convolutional neural network for feature extraction. The input layer size is 30×27. After two convolutional pooling layers and one ReLU activation layer, the feature data is flattened into one-dimensional data, and finally the feature data set M3 is obtained.

[0106] 6. Feature fusion.

[0107] like Fig.10 As shown, the manually extracted time domain feature dataset M1 is filtered by ReliefF to obtain the feature dataset M2, and then the feature dataset M2 is horizontally spliced ​​with the feature dataset M3 extracted by 1D-CNN to form a new feature dataset M.

[0108] 7. Phase identification.

[0109] A multi-class support vector machine algorithm was selected for the above-mentioned crawling collaborative phase classification problem. In this algorithm, a one-to-one method was used to train the classifier. Since there were four phase categories, a total of 6 classifiers needed to be trained. First, the phase sample geometry M was divided into a training set and a test set in a ratio of 8:2. Then, linear kernel, polynomial kernel and Gaussian kernel were selected for the kernel function, and C and γ were selected for the parameters. Then, 10-fold cross validation was used for the training set to select the best model kernel function and parameters. Finally, the test set data was input into the best classification model to obtain the corresponding phase recognition accuracy. Fig.11 This is a diagram showing the accuracy of the training set and the test set after 10 cumulative trainings under the condition of direct feature extraction. Fig.12 This is a diagram showing the accuracy of the training set and the test set after 10 cumulative trainings under the condition of extracting the coordinated phase pattern and then extracting the features.

[0110] The present invention collects acceleration data of each joint of the upper and lower limbs and periodic pressure data during the crawling process of the human body through multiple sensor modules, and extracts the coordinated phase pattern of multi-joint motion during the crawling movement, and separates the time recruitment signal reflecting the timing information of the human neuromuscular system regulation. In addition, considering that interference such as neural noise may affect the signal quality and even the subsequent phase recognition effect, the method further performs EMD decomposition and secondary reconstruction on the extracted time recruitment signal to obtain the time recruitment curve after secondary reconstruction, and at the same time, the coordinated phase pattern time recruitment curve after secondary reconstruction is used as the object of feature extraction in phase recognition, so as to identify the phase from the coordinated multi-joint motion that is more in line with the law of human motion control. This can make the crawling movement more accurate in phase recognition, and further provide a reference for the flexible control of the crawling rehabilitation training robot.

[0111] The present invention introduces the theory of multi-joint collaborative motion and the corresponding model into phase recognition. At the same time, the collaborative phase pattern time recruitment curve that has undergone secondary reconstruction is used as the object for feature extraction in phase recognition, so as to identify the phase from the multi-joint motion coordination that is more in line with the laws of human motion control, thereby greatly improving the phase recognition accuracy of crawling motion.

[0112] An embodiment of the present invention further provides an electronic device, including a memory and a processor.

[0113] The memory is used to store computer programs.

[0114] When the processor is used to execute the computer program stored in the memory, the steps of the above-mentioned method for identifying the phase of crawling movement are implemented.

[0115] An embodiment of the present invention further provides a computer-readable storage medium for storing a computer program, which implements the steps of the above-mentioned method for identifying the phase of crawling motion when the computer program is executed by a processor.

[0116] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A crawling motion phase identification method, characterized in that: The following steps are involved: Collect acceleration data of each limb joint and pressure data of each limb during crawling movement; The crawling motion process is divided into multiple crawling cycles, and the pressure data of each crawling cycle and the acceleration data of each crawling cycle are obtained; The posture matrix of each limb joint is constructed according to the acceleration data of each crawling cycle. The principal component analysis method of singular value decomposition (SVD) is used to extract the motion coordination phase pattern of each limb joint in the posture matrix. The time recruitment signal reflecting the timing information of the human neuromuscular system regulation is separated from the motion coordination phase pattern. The extracted time recruitment signal is subjected to EMD decomposition and secondary reconstruction to obtain the time recruitment curve after secondary reconstruction. Using the pressure data corresponding to the multiple crawling stages divided in each crawling cycle, the coordinated phase pattern of each limb joint movement is divided into multiple crawling movement phases; Extract the time domain feature data set of the time recruitment curve after secondary reconstruction in multiple crawling motion phases of each crawling cycle, and use the ReliefF method to screen the time domain feature data set to obtain the first feature data set whose acceleration data weight coefficient is higher than the set threshold; A one-dimensional convolutional neural network 1D-CNN is used to extract a second feature data set of acceleration data in multiple crawling motion phases in each crawling cycle, and the first feature data set and the second feature data set are horizontally spliced ​​to form a final feature data set; The final feature data set is classified and identified using a multi-class support vector machine (SVM) to obtain the phase of the crawling motion.

2. A crawling motion phase identification method according to claim 1, characterized in that: The acquisition of acceleration data of joints of each limb of the human body and pressure data of each limb during the crawling movement includes: The acceleration sensors are respectively set at the wrist joints, elbow joints, shoulder joints, knee joints and hip joints of the upper and lower limbs of the human body to collect the acceleration data of each limb joint during the crawling movement of the human body; The pressure sensors are placed on each palm of the human body to collect the pressure data of each limb during the crawling movement of the human body.

3. The crawling motion phase identification method according to claim 1, characterized in that: The acquisition of acceleration data of each crawling cycle includes: The first and last time points of the pressure data of each crawling cycle are respectively corresponded to the time points of the acceleration, and the acceleration data of each crawling cycle in the acceleration data is identified and divided.

4. The method for recognizing a crawling motion phase according to claim 1, characterized in that: The acquisition of the coordinated phase pattern of the limb joint movements includes: Each crawling cycle contains the acceleration data at the left wrist, left elbow, left shoulder, left knee, right wrist, right elbow, right shoulder, right knee and hip joints. The acceleration data of each joint is set as the posture matrix V , the posture matrix is ​​extracted using principal component analysis based on singular value decomposition (SVD) V The posture coordination matrix in W and time recruitment matrix H , to form a coordinated phase pattern of movement of each limb joint; its posture matrix V It is expressed as: ； in: Indicates the number of joint acceleration data; Represents the number of principal components obtained after matrix decomposition; Indicates time; When the cumulative explained variance of the principal components is higher than 90%, the number of principal components selected is and time recruitment matrix H The time recruitment signal represents the timing information of the human neuromuscular system regulation. Considering that the interference of neural noise will affect the quality of the time recruitment signal and subsequent phase recognition, the extracted time recruitment signal is subjected to EMD decomposition and secondary reconstruction to obtain the time recruitment curve after secondary reconstruction. The time recruitment curve after secondary reconstruction is used as the feature extraction object for subsequent phase recognition.

5. The method for recognizing a crawling motion phase according to claim 1, characterized in that: The limb joint movement coordination phase pattern is divided into a plurality of crawling movement phases, including: Each complete crawling cycle is divided into four crawling stages, namely S1, S2, S3 and S4; S1 represents the stage of left hand swinging and right hand supporting during crawling, S2 represents the stage of left hand and right hand supporting, S3 represents the stage of left hand supporting and right hand swinging, and S4 represents the stage of left hand and right hand supporting again; According to the four crawling stages, the pressure data in each crawling cycle is divided into pressure data corresponding to the four crawling stages; According to the correspondence between the pressure data in each crawling cycle and the data points of the coordinated phase pattern of each limb joint movement, the coordinated phase pattern time recruitment curve is divided into four phases using the pressure data corresponding to the four crawling stages divided in each crawling cycle.

6. A crawling motion phase identification method according to claim 5, characterized in that: The acquisition of the final feature data set includes: The peak value, peak-to-peak value, mean value, standard deviation, root mean square value and slope in the four phases of each crawling cycle are extracted to form the time domain feature data set M1. If the motion input of the segment joint acceleration is indivual; The ReliefF method is used to perform feature screening on the time domain feature data set M1, screen the weight coefficients of the features in the time domain feature data set M1, and evaluate the importance of each feature to form a feature data set M2; The four phases of each crawling cycle are input into a one-dimensional convolutional neural network 1D-CNN for feature extraction. The input layer size is 30×27. After two convolutional pooling layers and one ReLU activation layer, the feature data is flattened into one-dimensional data to obtain the feature data set M3. The feature data set M2 and the feature data set M3 are horizontally spliced ​​to form the final feature data set.

7. The method for recognizing a crawling motion phase according to claim 1, characterized in that: The step of obtaining the phase of the crawling motion comprises: The classifiers in the multi-class support vector machine SVM are trained in a one-to-one manner, and a total of 6 classifiers are trained according to the 4 phase categories. The kernel function in the training uses a linear kernel, a polynomial kernel, and a Gaussian kernel to obtain the trained multi-class support vector machine SVM; The final feature data set is input into the trained multi-class support vector machine (SVM) for classification and recognition to obtain the phase of the crawling movement.

8. A crawling motion phase recognition system, characterized in that: include: Kinematic data signal acquisition module, used to collect acceleration data of joints of each limb of the human body and pressure data of each limb during crawling movement; A collaborative phase pattern extraction module is used to divide the crawling motion process into multiple crawling cycles, and obtain the pressure data of each crawling cycle and the acceleration data of each crawling cycle; The posture matrix of each limb joint is constructed according to the acceleration data of each crawling cycle. The principal component analysis method of singular value decomposition (SVD) is used to extract the motion coordination phase pattern of each limb joint in the posture matrix. The time recruitment signal reflecting the timing information of the human neuromuscular system regulation is separated from the motion coordination phase pattern. The extracted time recruitment signal is subjected to EMD decomposition and secondary reconstruction to obtain the time recruitment curve after secondary reconstruction. A coordinated phase pattern phase division module is used to divide the coordinated phase pattern of each limb joint movement into a plurality of crawling movement phases using the pressure data corresponding to the plurality of crawling stages divided in each crawling cycle; A multi-feature extraction module is used to extract a time domain feature data set of a time recruitment curve after secondary reconstruction in multiple crawling motion phases in each crawling cycle, and use the ReliefF method to filter the time domain feature data set to obtain a first feature data set whose acceleration data weight coefficient is higher than a set threshold; A one-dimensional convolutional neural network 1D-CNN is used to extract a second feature data set of acceleration data in multiple crawling motion phases in each crawling cycle, and the first feature data set and the second feature data set are horizontally spliced ​​to form a final feature data set; The phase recognition module is used to classify and recognize the final feature data set using a multi-category support vector machine (SVM) to obtain the phase of the crawling motion.

9. An electronic device, characterized in that: include: Memory and processor; The memory is used to store computer programs; The processor is used to implement the steps of a crawling motion phase identification method as described in any one of claims 1 to 7 when executing the computer program stored in the memory.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, which, when executed by a processor, implements the steps of a crawling motion phase identification method as described in any one of claims 1 to 7.

Images