Human activity recognition method based on combination of wavelet transform and convolutional neural network

Abstract

The present invention discloses a human activity recognition method based on a combination of wavelet transform and convolutional neural network, belonging to the human activity recognition technology field of pattern recognition. In this method, first, the waveform data of the multi-channel sensor is segmented by window sliding, and then decomposed into low-frequency and high-frequency components by wavelet transform, and the two are combined as the input data of the network model; then, the convolutional neural network with different convolution kernels is used to efficiently extract multi-dimensional features, and the interference noise caused by unconscious shaking of the human body is filtered by the maximum pooling layer; finally, after the output classification of the fully connected layer, a human activity recognition model is constructed to realize accurate recognition of human activity status. This method optimizes the original data input method, avoids the drawbacks of heavy reliance on artificial feature extraction based on traditional pattern recognition methods, improves the classification performance of the model, has higher activity recognition ability, and has strong generalization ability and robustness.

Description

Technical Field

[0001] The present invention relates to the technical field of human activity recognition related to pattern recognition for recognizing specific movements or actions of a person based on sensor data, and in particular to a human activity recognition method based on a combination of wavelet transform and convolutional neural network. Background Art

[0002] With the development of the Internet of Things, artificial intelligence, and human-computer interaction applications, human activity recognition technology plays an important role in people's daily lives. Its goal is to use data collected by different devices (such as multimodal sensors, cameras, etc.) to extract features from human daily activities, automatically classify the types of human activities, and obtain the information that the human body needs to convey, thereby providing a basis for other intelligent applications. Therefore, the field of human activity recognition technology has become one of the hottest research topics. So far, human activity recognition technology has been widely used in various fields of human life, such as: in medical diagnosis, human activity recognition technology is used to track the elderly for fall detection and disease prevention; in road traffic, it identifies driving activities and guides humans to travel safely; in smart cities, it is used to identify and monitor family activities and energy-saving solutions, create smart home environments, etc.

[0003] However, the human activity recognition method based on traditional pattern recognition is highly dependent on manual feature extraction and can often only capture the shallow features of the original input signal, which greatly limits the accuracy and performance improvement of traditional machine learning methods in the classification of human activity recognition technology. Therefore, given the successful application of deep learning models in image classification, speech recognition, natural language processing and other fields, the use of deep learning methods in the field of human activity recognition is a new research direction in pattern recognition. Summary of the invention

[0004] In view of this, the present invention proposes a human activity recognition method based on wavelet transform and convolutional neural network combination. The method has strong robustness and higher activity recognition ability.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] A human activity recognition method based on a combination of wavelet transform and convolutional neural network comprises the following steps:

[0007] (1) Preprocess the data of multi-channel sensors and segment them by window sliding;

[0008] (2) Decomposing the data processed in step (1) by wavelet transform to serve as input data of the network model;

[0009] (3) constructing a convolutional neural network, inputting the input data obtained in step (2) into the convolutional neural network and training it to obtain a trained convolutional neural network model; the output of the convolutional neural network model is an activity recognition classification result;

[0010] (4) Input the data from the terminal’s built-in sensors into the trained convolutional neural network model to identify human activities.

[0011] Furthermore, the specific method of step (1) is:

[0012] (101) Filling missing values ​​in the collected multi-channel sensor data using a linear interpolation algorithm;

[0013] (102) Use a sliding window to segment the data processed in step (101).

[0014] Furthermore, the fixed length of the sliding window is 24, the overlap rate is 50%, and the step size is set to 12.

[0015] Furthermore, the wavelet transform decomposition in step (2) is specifically as follows:

[0016] The data processed in step (1) is transformed using Haar wavelet transform. high and h low Decompose into low-frequency and high-frequency components, extract temporal features, and then combine the temporal features of the low-frequency and high-frequency components.

[0017] Furthermore, the convolutional neural network includes two convolutional layers, a maximum pooling layer, a convolutional layer, a maximum pooling layer, a convolutional layer and two fully connected layers in sequence; ReLU is used to calculate feature mapping, and the maximum pooling layer uses Dropout to prevent model overfitting.

[0018] Furthermore, the convolutional neural network is trained in step (3) as follows:

[0019] All waveform data are divided into multiple different training sets and test sets. The training set is used to train the convolutional neural network, and the test set is used to test the trained convolutional neural network. The parameters of the convolutional neural network are tuned according to the test results. After iterative training and testing, a trained convolutional neural network is obtained.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] (1) The method of the present invention can solve the problems of large fluctuations and severe noise in sensor data.

[0022] (2) The present invention combines the constructed input data and proposes a human activity recognition method based on the combination of wavelet transform and convolutional neural network, which can realize accurate recognition of human activity status through convolutional neural network.

[0023] (3) Compared with the human activity recognition method based on traditional pattern recognition, the present invention does not need to rely on manual feature extraction, can extract deeper multi-dimensional features, and has stronger generalization ability. Moreover, since the present method uses wavelet transform for data decomposition, the number of parameters is small, the model converges faster, and has stronger robustness and higher activity recognition ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Flow chart of a method for identifying human activities in an embodiment of the present invention.

[0025] Figure 2 Schematic diagram of window sliding segmentation in an embodiment of the present invention.

[0026] Figure 3 This is a convolutional neural network model diagram in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0028] A human activity recognition method based on a combination of wavelet transform and convolutional neural network. In this method, first, the waveform data of the multi-channel sensor is segmented by window sliding, and then decomposed into low-frequency and high-frequency components by wavelet transform. The two are combined as the input data of the network model; then, the convolutional neural network with different convolution kernels is used to efficiently extract multi-dimensional features, and the maximum pooling layer is used to filter the interference noise caused by the unconscious shaking of the human body; finally, the output classification is carried out through the fully connected layer to achieve accurate recognition of human activity status. The method flow is as follows Figure 1 shown.

[0029] The method embodiment specifically includes the following steps:

[0030] Step 1: Select the measured data of wireless sensors worn on subjects from the OPPORTUNITY dataset. Missing data are represented by NaN / 0, and a linear interpolation algorithm is used to fill the missing values.

[0031] Step 2: After preprocessing, the short time series data extracted from the original sensor data is segmented using a sliding window with a fixed length of 24 and an overlap rate of 50%, that is, the step size is set to 12. Figure 2 As shown, the data of different sensor channels are segmented after being segmented by a sliding window to obtain more sample data.

[0032] Step 3: Use Haar wavelet to apply h to the waveform data obtained in step 2 high and h low The transform is decomposed into low-frequency and high-frequency components, which effectively extracts the temporal feature information from the signal data and combines the low-frequency and high-frequency components as the input data of the network model.

[0033] Step 4: Build a multi-layer convolutional neural network model consisting of convolutional layers, maximum pooling layers, and fully connected layers, use ReLU to calculate feature maps, and use Dropout to prevent the model from overfitting. The network model settings are as follows: Figure 3 As shown in the figure, it includes two convolutional layers, a maximum pooling layer, a convolutional layer, a maximum pooling layer, a convolutional layer, and two fully connected layers; its input is a combination of low-frequency and high-frequency components after wavelet transform, and its output is the activity recognition classification result.

[0034] The convolutional layers with different convolution kernels are used to efficiently extract multi-dimensional features, and the maximum pooling layer is used to filter the interference noise caused by unconscious shaking of the human body. Finally, the classification model is output through the fully connected layer.

[0035] Step 5: Use cross-validation to divide the input data of the network model in step 3 into different training sets and test sets, train the constructed multi-layer convolutional neural network model and perform parameter tuning, and store the trained positioning model.

[0036] Step 6: Input the data from the mobile phone’s built-in sensors into the network model trained in step 5 to identify human activities.

[0037] This method uses wavelet transform to locally transform the original sensor data in space (time) and frequency, effectively extracts fine-grained representative information from the signal data, and uses convolutional neural network to learn deep features to achieve accurate recognition of human activity status. The wavelet transform uses Haar wavelet, which has fewer parameters and has a good effect on improving the accuracy of the network model. As a feature extraction method in the deep learning method, the convolutional neural network learns the frequency characteristics of waveform data of sensors in different channels and is used to build a human activity recognition model.

[0038] In summary, the present invention avoids the problem that the human activity recognition method based on traditional pattern recognition can only capture the shallow features of the original input signal, resulting in limitations in the accuracy and performance improvement of the classification of human activity recognition technology. A human activity recognition method based on a combination of wavelet transform and convolutional neural network is proposed. In this method, wavelet transform is used to locally transform the input data in space and frequency, effectively extract fine-grained representative information from the signal data, and optimize the original data input method. Using a multi-layer convolutional neural network, the deep representative features are fully learned, and a human activity recognition model is trained to achieve accurate human activity classification. This method is verified by experiments.

[0039] The results show that the method of the present invention simplifies the structure of the model, improves the classification performance of the model, has higher activity recognition ability, and has strong generalization ability and robustness compared with traditional machine learning for human activity recognition.

Claims

1. A human activity recognition method based on a combination of wavelet transform and convolutional neural network, characterized in that: The following steps are involved: (1) Preprocess the data of multi-channel sensors and segment them by window sliding; (2) Decomposing the data processed in step (1) by wavelet transform as input data of the network model; (3) constructing a convolutional neural network, inputting the input data obtained in step (2) into the convolutional neural network and training it to obtain a trained convolutional neural network model; the output of the convolutional neural network model is an activity recognition classification result; (4) Input data from the terminal’s built-in sensors into the trained convolutional neural network model to identify human activities; The specific method of step (1) is as follows: (101) Using a linear interpolation algorithm to fill in the missing values ​​in the collected multi-channel sensor data; (102) segmenting the data processed in step (101) using a sliding window; Among them, the fixed length of the sliding window is 24, the overlap rate is 50%, and the step size is set to 12; The wavelet transform decomposition in step (2) is as follows: The data processed in step (1) is transformed using Haar wavelet transform. and Decompose into low-frequency and high-frequency components, extract temporal features, and then combine the temporal features of the low-frequency and high-frequency components.

2. The human activity recognition method based on the combination of wavelet transform and convolutional neural network according to claim 1, characterized in that: The convolutional neural network includes two convolutional layers, a maximum pooling layer, a convolutional layer, a maximum pooling layer, a convolutional layer and two fully connected layers in sequence; using Calculate feature maps, the maximum pooling layer uses Function to prevent model overfitting.

3. The human activity recognition method based on the combination of wavelet transform and convolutional neural network according to claim 2 is characterized in that: The way to train the convolutional neural network in step (3) is: All waveform data are divided into multiple different training sets and test sets. The training set is used to train the convolutional neural network, and the test set is used to test the trained convolutional neural network. The parameters of the convolutional neural network are tuned according to the test results. After iterative training and testing, a trained convolutional neural network is obtained.

Images