A method for establishing a human posture recognition model and a method for human posture recognition
By combining the two-way long and short-term memory network and the support vector machine classifier model, the problems of insufficient feature extraction and low recognition accuracy in human posture recognition are solved, and efficient posture recognition is achieved, especially the excellence on small sample data sets performance.
Patent Information
- Application Number
- CN202210565324.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-23
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2042-05-23
AI Technical Summary
The existing human posture recognition algorithms have problems such as insufficient feature extraction, low recognition accuracy, complex extraction process and high filtering delay.
A model combining a bidirectional long and short-term memory network and a support vector machine classifier is adopted to collect human motion signals through intelligent wearable devices for preprocessing and training, extract subtle features of bidirectional timing before and after time, and improve the accuracy and efficiency of posture recognition.
It significantly improves the recognition accuracy and efficiency of human posture recognition, especially the classification performance on small sample data sets, achieving an accuracy of 95.76%, which is better than traditional machine learning methods and other neural network design models.
Smart Images

Figure CN114897025B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer application technology, and in particular to a method for establishing a human body posture recognition model and a method for human body posture recognition. Background Art
[0002] Human posture recognition is widely used in health monitoring, sports competition analysis, and human-computer interaction scenarios. There are two types of traditional human posture recognition algorithms. One is the statistical machine learning method. For example, the original signal is classified through models such as threshold analysis, support vector machine or perception vector machine. There are problems such as insufficient feature extraction and low recognition accuracy. The other is the combination of shallow machine learning algorithms and signal processing technology. For example, signal features are extracted through wavelet transform, empirical mode decomposition or fast Fourier transform and then input into the machine learning model for classification. Although the classification effect is acceptable, there are problems such as complex extraction process, high filtering delay, and incomplete feature extraction. Summary of the invention
[0003] In view of the deficiencies in the prior art, the present invention provides a human body posture recognition model, a human body posture recognition method and device, and a storage medium.
[0004] In a first aspect, the present invention provides a method for establishing a human posture recognition model, the method comprising the following steps:
[0005] Acquire human motion signals in different postures, wherein the source of the human motion signals includes a smart wearable device worn on a human body;
[0006] Preprocessing the human body motion signal to obtain a model input signal;
[0007] The preset model is trained based on the model input signal to obtain a human posture recognition model, and the human posture recognition model is used to output a posture classification result, wherein the preset model includes a bidirectional long short-term memory network and a support vector machine classifier connected in sequence, and the model input signal is used to train the bidirectional long short-term memory network.
[0008] In a second aspect, the present invention provides a device for establishing a human posture recognition model, which includes a memory and a processor; the memory is used to store a computer program; the processor is used to implement the human posture recognition model establishment method as described above when executing the computer program.
[0009] In a third aspect, the present invention provides a method for human body posture recognition, the method comprising the following steps:
[0010] Acquire human motion signals;
[0011] The human body motion signal is input into a human body posture recognition model established according to the human body posture recognition model establishment method as described above to obtain a posture classification result.
[0012] In a fourth aspect, the present invention provides a human body posture recognition device, which includes a memory and a processor; the memory is used to store a computer program; the processor is used to implement the human body posture recognition method as described above when executing the computer program.
[0013] In a fifth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the human body posture recognition model establishment method or the human body posture recognition method as described above is implemented.
[0014] In the present invention, human motion signals can be collected through smart wearable devices such as smart belts, so that they can be used in various scenarios that require real-time posture recognition. After preprocessing the human motion signals, they can be used to train a preset model, wherein the preset model includes a bidirectional long short-term memory network and a support vector machine classifier. The bidirectional long short-term memory network can be used to perform deep bidirectional feature extraction, for example, subtle features of bidirectional time series before and after time can be extracted. The support vector machine classifier can ensure the classification performance in a small sample data set, and ultimately significantly improve the recognition accuracy and efficiency of the model in posture recognition tasks. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0016] Figure 1 A schematic diagram of a flow chart of a method for establishing a human posture recognition model according to an embodiment of the present invention;
[0017] Figure 2 A schematic diagram of a smart wearable device according to an embodiment of the present invention;
[0018] Figure 3 A schematic diagram of the structure of a human body posture recognition model according to an embodiment of the present invention;
[0019] Figure 4 Schematic diagram of the process of the human body posture recognition method according to an embodiment of the present invention. DETAILED DESCRIPTION
[0020] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.
[0021] like Figure 1 As shown, a method for establishing a human posture recognition model according to an embodiment of the present invention comprises the following steps:
[0022] Human body motion signals in different postures are acquired, wherein the source of the human body motion signals includes a smart wearable device worn on a human body.
[0023] Specifically, the smart wearable device can be a smart belt to ensure the comfort of wearing and the consistency of human-machine movement, which is conducive to accurate data collection. The main sensing devices it carries include accelerometers and gyroscopes, among which the accelerometer can be used to collect acceleration data, and the gyroscope can be used to collect angular velocity data and angle data as human motion signals. In addition, human behavior postures usually include walking, running, going upstairs, going downstairs, standing and sitting down. The human motion signals corresponding to each posture are different. The human motion signals under specific postures can be collected to form a corresponding relationship, or the corresponding behavior posture can be determined through random human motion signals.
[0024] The human body motion signal is preprocessed to obtain a model input signal.
[0025] Specifically, after obtaining human motion signals in different postures through, for example, a smart belt, information mining can be performed according to the characteristics of the data, or noise reduction processing such as filtering can be performed, so that the pre-processed signals are more conducive to the establishment of an efficient and accurate classification model.
[0026] The preset model is trained based on the model input signal to obtain a human posture recognition model, and the human posture recognition model is used to output a posture classification result, wherein the preset model includes a bidirectional long short-term memory network and a support vector machine classifier connected in sequence, and the model input signal is used to train the bidirectional long short-term memory network.
[0027] Specifically, the motion signals can be used as samples and the corresponding postures can be used as corresponding labels to train the bidirectional long short-term memory network (BLSTM). The trained BLSTM is used as a signal feature extractor, which together with the support vector machine (SVM) classifier constitutes a human posture recognition model.
[0028] In this embodiment, human motion signals can be collected through smart wearable devices such as smart belts, so that they can be used in various scenarios that require real-time posture recognition. After preprocessing the human motion signals, they can be used to train a preset model, wherein the preset model includes a bidirectional long short-term memory network and a support vector machine classifier. The bidirectional long short-term memory network can be used to perform deep bidirectional feature extraction. For example, subtle features of bidirectional time series before and after time can be extracted. The support vector machine classifier can ensure the classification performance in a small sample data set, and ultimately significantly improve the recognition accuracy and efficiency of the model in posture recognition tasks.
[0029] Optionally, acquiring human body motion signals in different postures includes:
[0030] The human motion signal is obtained in walking, running, going upstairs, going downstairs, standing and sitting postures, wherein the human motion signal includes three-dimensional acceleration data, three-dimensional angular velocity data and three-dimensional angle data collected by the smart wearable device.
[0031] Specifically, Figure 2 As shown, when the user wears the above-mentioned smart belt, the up and down direction of the view is used as the X-axis direction, the left and right direction of the view is used as the Y-axis direction, and the direction perpendicular to the view is used as the Z-axis direction. At this time, the smart belt can collect acceleration data, angular velocity data and angle data in three dimensions of the X-axis, Y-axis and Z-axis, including a total of 9 motion signals, namely, acceleration data in the X-axis direction, acceleration data in the Y-axis direction, acceleration data in the Z-axis direction, angular velocity data in the X-axis direction, angular velocity data in the Y-axis direction, angular velocity data in the Z-axis direction, angle data in the X-axis direction (roll angle) data, angle data in the Y-axis direction (pitch angle) data and angle data in the Z-axis direction (yaw angle). When collecting data for model training, users can be allowed to perform actions in walking, running, going upstairs, going downstairs, standing and sitting postures to obtain human motion signals corresponding to different postures.
[0032] Optionally, the preprocessing of the human body motion signal to obtain a model input signal includes:
[0033] Three-dimensional gravity acceleration data and three-dimensional linear acceleration data are obtained according to the three-dimensional acceleration data.
[0034] Specifically, the acceleration data collected above is the superposition of gravity acceleration and linear acceleration. Considering the high-frequency characteristics of body linear acceleration and low-frequency characteristics of gravity acceleration during exercise, the original acceleration data is separated by first-order low-pass filtering, which can be calculated by the following formula.
[0035] g i (n) = αg i (n-1)+(1-α)Ai (n) i = x, y, z;
[0036] a i (n) = A i (n)-g i (n) i = x, y, z;
[0037] Among them, n is the time, A i (n) is the total acceleration at time n, g i (n) is the gravitational acceleration component at time n, a i (n) is the linear acceleration at time n, α is a parameter related to the sampling period and time constant, α can be set to 0.8, x, y, z represent the X-axis, Y-axis, and Z-axis components respectively.
[0038] The model input signal is obtained based on the three-dimensional acceleration data, the three-dimensional angular velocity data, the three-dimensional angle data, the three-dimensional gravity acceleration data and the three-dimensional linear acceleration data.
[0039] Specifically, in the posture of going upstairs, the angle between the direction of gravity and the direction of the Z axis is greater than 90°, so the component of the acceleration data in the direction of the Z axis is negative; in the posture of going downstairs, the angle between the direction of gravity and the direction of the Z axis is less than 90°, so the component of the acceleration data in the direction of the Z axis is positive. Decomposing the acceleration data into gravity acceleration data and linear acceleration data does not require the introduction of external parameters, but the latter two can more accurately reflect the relevant characteristics of different motion postures. Therefore, incorporating the latter two into the model input signal will help to further improve the accuracy of the training model. Therefore, at this time, it is equivalent to expanding the motion signals from 9 to 15.
[0040] Optionally, the preprocessing of the human motion signal to obtain a model input signal further includes:
[0041] Perform moving mean filtering on the human motion signal.
[0042] Specifically, to ensure the smoothness of signal changes, the original signal must first be filtered and denoised. Compared with filtering methods such as median filtering and Kalman filtering, the moving mean filtering method has the advantages of small computational complexity and less distortion of processed signals, and is more suitable for processing motion sensor sequence signals. Therefore, the moving mean filter is used to denoise the human motion signal, and its filtering process can be calculated by the following formula.
[0043]
[0044] Among them, x i is the original signal value recorded at the i-th sample point, y iis the moving mean of the sample point, k represents the window size of the mean filter, and y i The new sequence formed is the signal sequence after noise reduction processing.
[0045] All signals are processed by moving mean filtering and the optimal filter window size is set to complete the signal denoising. The noise signal can be effectively removed after moving mean filtering, and at the same time, the main motion information in the original signal can be better preserved.
[0046] Performing data standardization processing on the human body motion signal.
[0047] Specifically, since the sensor device is embedded in the smart belt, it may be slightly displaced as the belt shakes during intense exercise, and the collected data will have slight deviations. At the same time, considering the large differences in the amplitudes of acceleration, angular velocity, and angle, Min-Max standardization is used to process the above 15 signal data, which can be calculated by the following formula.
[0048]
[0049] Among them, x i Represents the value of each signal, min represents the minimum value of the sample data taken, and max represents the maximum value of the sample data taken.
[0050] In addition, data overlap sampling can be performed on human motion signals.
[0051] Optionally, the bidirectional long short-term memory network includes a first fully connected layer, a forward LSTM layer, a reverse LSTM layer and a second fully connected layer. The two outputs of the first fully connected layer are respectively connected to the forward LSTM layer and the reverse LSTM layer. The outputs of the forward LSTM layer and the reverse LSTM layer are connected to the second fully connected layer after splicing, and the output of the second fully connected layer is used to access the support vector machine classifier.
[0052] Specifically, Figure 3 As shown, the bidirectional long short-term memory network (BLSTM) includes a first fully connected layer, a forward LSTM layer, a reverse LSTM layer and a second fully connected layer, wherein the forward LSTM layer and the reverse LSTM layer each include two LSTM units.
[0053] Bidirectional long short-term memory network is used as a motion signal feature extractor. In the model training stage, the above 15 types of motion signals can be used as samples, and their corresponding postures can be used as corresponding labels. BLSTM can perform deep bidirectional feature extraction, in which the subtle features of the bidirectional time series of motion behavior in the forward and backward time are respectively mined through the forward LSTM layer and the reverse LSTM layer.
[0054] After training the human posture recognition model, or in other words, in the classification and recognition stage, after the human motion signal passes through the trained BLSTM, the outputs of the forward LSTM layer and the reverse LSTM layer at the last moment are spliced, and the extracted feature information is input into the SVM classifier through the fully connected layer. In the SVM classifier, the multi-classification task is realized through the soft interval and kernel function, and the posture classification result is finally output to complete the posture recognition. Among them, the nonlinear SVM classifier replaces the Softmax layer in the neural network model as the behavior posture classifier, and the good classification performance of SVM in low-dimensional features is used to improve the accuracy of behavior posture recognition.
[0055] Optionally, the training a preset model based on the model input signal includes:
[0056] The outputs of the forward LSTM layer and the reverse LSTM layer at the last moment are concatenated and combined with the Softmax classifier, and the parameters of each internal layer of the neural network are updated through the time back propagation algorithm, and the training of the preset model is completed by minimizing the loss function.
[0057] Specifically, during the model training process, the BLSTM and Softmax intermediate layers are combined to output the probability values of 6 types of behavioral postures. The corresponding indexes are output as the posture classification results through maximum likelihood estimation, and the parameters of each internal layer of the neural network are updated through the back propagation through time (BPTT) algorithm. The model training is completed by minimizing the loss function and saving the model parameters.
[0058] The calculation of the Softmax function is shown below.
[0059]
[0060] Among them, i is the behavior posture number, y i is the probability of the output.
[0061] BLSTM is trained by updating the weights of each internal network layer. For the multi-classification task of posture recognition, the cross entropy function is set as the loss function, and its calculation is shown in the following formula.
[0062]
[0063] in, For each training sample label, y i is the output value of the classification result, or the output probability.
[0064] In the actual training process, it is necessary to set the model's hyperparameters to complete the model training and tuning. Specifically, the grid search method can be used to determine the best hyperparameters, including the dropout coefficient setting and the number of neurons. Due to the complex structure of the neural network, overfitting often occurs during the training process. In addition to setting regularization in the loss function, it can also be alleviated by adding a dropout layer and setting the probability of neuron dropout. After comparison, when the dropout layer is added to each intermediate layer and the dropout probability is set to 0.2, the recognition accuracy of the model is relatively stable and has a good effect in avoiding overfitting. In addition, the number of neurons has a significant impact on the feature extraction ability of the model, which not only directly affects the training accuracy of the model, but also affects the training time of the model. Considering the accuracy, overfitting and computational time caused by the number of model parameters, the number of neurons in each LSTM layer is set to 64. In order to obtain the best training effect, the hyperparameters are traversed and combined by grid search, and the best hyperparameter combination is determined according to the training effect. The hyperparameter settings of the human posture recognition model are shown in Table 1.
[0065] Table 1
[0066]
[0067] The design and training of the human posture recognition model can be completed by adjusting the model structure and tuning the hyperparameters. In order to verify the training effect of the model, the human motion signals collected by the above-mentioned smart belt and the UCI-HAR dataset can be used for verification. Among them, the UCI-HAR dataset is a public dataset of the University of California, including sensor signal data of 30 volunteers in 6 types of behavioral postures: walking, going upstairs, going downstairs, sitting, standing and lying. After the original signal is preprocessed accordingly, it can be input into the above-mentioned human posture recognition model for comparative verification.
[0068] Since the recognition of human posture is a classification task, Precision, Recall, F1-score, and Accurancy can be used as evaluation indicators, which are calculated through four sample statistics, namely:
[0069] (1) True Positive (TP): a sample whose classification result is positive.
[0070] (2) True Negative (TN): A negative sample whose classification result is negative.
[0071] (3) False Positive (FP): A negative example whose classification result is positive.
[0072] (4) False Negative (FN): A positive example whose classification result is negative.
[0073] Calculate the sample statistics of each posture and obtain the evaluation indicators as shown in the following formula:
[0074]
[0075]
[0076]
[0077]
[0078] For human posture classification, traditional machine learning methods include MLP, random forest, decision tree, KNN and SVM, etc. In order to more intuitively compare the effects of various methods, the same UCI-HAR data set is used to compare the above methods and the method of the present invention, and finally the average accuracy, average precision and average recall are taken. The corresponding results are shown in Table 2.
[0079] Table 2
[0080]
[0081] It can be seen that the human posture recognition model of the present invention has the best performance in various evaluation indicators, with average precision, average recall and average accuracy all exceeding 95%, and its performance on the UCI-HAR data set is better than traditional machine learning methods in all aspects. This is because the model deeply mines the subtle feature differences in the timing of each behavior motion signal through a bidirectional recurrent neural network, while traditional machine learning methods rely on a single feature extraction method and only capture local feature information of sensor signals, resulting in average classification performance. At the same time, after completing the training, the human posture recognition model of the present invention can directly identify human postures through motion timing signal fragments, achieving end-to-end input and output, which is better than machine learning methods in all aspects.
[0082] Furthermore, the human posture recognition model of the present invention is compared with the posture recognition model designed based on neural network on the UCI-HAR dataset by researchers at home and abroad. The comparison results are shown in Table 3.
[0083] Table 3
[0084]
[0085] It can be seen that the human posture recognition model of the present invention has the highest recognition accuracy, reaching 95.76%, and its classification effect is better than the effect shown by the researchers in the corresponding literature in Table 3. The human posture recognition model of the present invention mines the time series signal features of the previous moment, the current moment and the next moment during the feature learning process, while CNN and LSTM do not consider the signal features of the next moment. In addition, in classification and discrimination, it can fully combine the excellent performance of BLSTM in time series signal feature extraction and the classification performance advantage of SVM classifier in small sample data sets, thereby significantly improving the recognition accuracy of the model in posture recognition tasks.
[0086] In another embodiment of the present invention, a human posture recognition model building device includes a memory and a processor; the memory is used to store a computer program; the processor is used to implement the human posture recognition model building method as described above when executing the computer program.
[0087] It should be noted that the device may be a computer device such as a server or a mobile terminal.
[0088] In another embodiment of the present invention, a computer program is stored on a computer-readable storage medium, and when the computer program is executed by a processor, the method for establishing a human body posture recognition model as described above is implemented.
[0089] like Figure 4 As shown, a human body posture recognition method according to an embodiment of the present invention comprises the following steps:
[0090] Acquire human motion signals.
[0091] Specifically, the user's human body motion signal can be obtained through a smart wearable device such as a smart belt. Optionally, the human body motion signal is pre-processed accordingly and then input into the human body posture recognition model.
[0092] The human body motion signal is input into a human body posture recognition model established according to the human body posture recognition model establishment method as described above to obtain a posture classification result.
[0093] Specifically, after the human motion signal is input into the human posture recognition model, BLSTM is used as a signal feature extractor and SVM is used as a classifier to output the posture classification result.
[0094] In another embodiment of the present invention, a human body posture recognition device includes a memory and a processor; the memory is used to store a computer program; the processor is used to implement the human body posture recognition method as described above when executing the computer program.
[0095] It should be noted that the device may be a computer device such as a server or a mobile terminal.
[0096] In another embodiment of the present invention, a computer program is stored on a computer-readable storage medium, and when the computer program is executed by a processor, the human body posture recognition method as described above is implemented.
[0097] The reader should understand that in the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.
[0098] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.
Claims
1. A method for establishing a human posture recognition model, characterized in that: include: Acquire human motion signals in different postures, wherein the source of the human motion signals includes a smart wearable device worn on a human body; Preprocessing the human body motion signal to obtain a model input signal; Based on the model input signal, a preset model is trained to obtain a human posture recognition model, and the human posture recognition model is used to output a posture classification result, wherein the preset model includes a bidirectional long short-term memory network and a support vector machine classifier connected in sequence, and the model input signal is used to train the bidirectional long short-term memory network; The bidirectional long short-term memory network includes a first fully connected layer, a forward LSTM layer, a reverse LSTM layer, and a second fully connected layer, wherein two outputs of the first fully connected layer are respectively connected to the forward LSTM layer and the reverse LSTM layer, and the outputs of the forward LSTM layer and the reverse LSTM layer are connected to the second fully connected layer after being spliced, and the output of the second fully connected layer is used to connect to the support vector machine classifier; The training of the preset model based on the model input signal comprises: The outputs of the forward LSTM layer and the reverse LSTM layer at the last moment are concatenated and combined with the Softmax classifier, and the parameters of each internal layer of the neural network are updated through the time back propagation algorithm, and the training of the preset model is completed by minimizing the loss function.
2. The method for establishing a human posture recognition model according to claim 1, characterized in that: The obtaining of human motion signals in different postures comprises: The human motion signal is obtained in walking, running, going upstairs, going downstairs, standing and sitting postures, wherein the human motion signal includes three-dimensional acceleration data, three-dimensional angular velocity data and three-dimensional angle data collected by the smart wearable device.
3. The method for establishing a human posture recognition model according to claim 2, characterized in that: The preprocessing of the human body motion signal to obtain a model input signal comprises: Obtaining three-dimensional gravity acceleration data and three-dimensional linear acceleration data according to the three-dimensional acceleration data; The model input signal is obtained based on the three-dimensional acceleration data, the three-dimensional angular velocity data, the three-dimensional angle data, the three-dimensional gravity acceleration data and the three-dimensional linear acceleration data.
4. The method for establishing a human posture recognition model according to claim 3, characterized in that: The preprocessing of the human body motion signal to obtain a model input signal further comprises: Performing moving mean filtering on the human motion signal; Performing data standardization processing on the human body motion signal.
5. A human body posture recognition model building device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to implement the method for establishing a human posture recognition model as described in any one of claims 1 to 4 when executing the computer program.
6. A human body posture recognition method, characterized in that: include: Acquire human motion signals; The human body motion signal is input into a human body posture recognition model established according to the human body posture recognition model establishment method according to any one of claims 1 to 4 to obtain a posture classification result.
7. A human body posture recognition device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to implement the human body posture recognition method as claimed in claim 6 when executing the computer program.
8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the method for establishing a human posture recognition model as described in any one of claims 1 to 4 or the method for recognizing human posture as described in claim 6 is implemented.
