An Activity Recognition Method and System Based on an Ensemble Learning Strategy

Abstract

The present invention discloses an activity recognition method and system based on an integrated learning strategy. The method comprises the following steps: S1. Install sensors on key parts of the human body to collect real-time physiological signals related to the human body; S2. Fuse the collected continuous physiological signals through information fusion; S3. Based on feature selection, select an integrated feature selection method suitable for activity recognition; S4. For the multi-classification tasks involved in activity recognition, select a classification system based on a decomposition strategy and integrated learning technology for automatic recognition of activity types. The present invention has been tested by an activity recognition data set, and can not only screen out useful features, but also improve the recognition accuracy compared with the prior art, and can more effectively solve the activity recognition problem in practical applications.

Description

Technical Field

[0001] The present invention relates to the technical field of activity recognition and data mining, and particularly to an activity recognition method and system based on an ensemble learning strategy. Background Art

[0002] Recently, human activity recognition has received a great deal of attention and has a wide range of applications in fields such as elderly care, emotion monitoring, and human-computer interaction. For example, real-time activity monitoring of diabetic patients can enable doctors to detect risks at an early stage for treatment, music software can sense the current activity state of users to recommend specific types of music suitable for that activity, and surveillance and security departments can use activity recognition technology to respond to terrorist threats. In addition, intelligent conference rooms and intelligent hospitals also rely on activity recognition to provide multimodal interaction. Traditional activity recognition monitoring uses cameras or some large monitoring devices, which are limited by cost and technology and cannot achieve continuous and effective monitoring. In the past decade or so, technology has developed rapidly, and mobile devices such as sensors have shown unprecedented characteristics. Their low cost and portability enable them to continuously measure physiological signals such as electrocardiogram (ECG) and thoracic electrical bioimpedance (TEB). These data information contains information about the physiological and emotional states of the human body and can be more effectively applied to activity recognition.

[0003] Currently, a single physiological signal acquisition strategy is mainly adopted, and there are few practices of activity recognition that fuse multiple physiological signal sources. On the other hand, the types of activities are rich and diverse, so activity recognition is a complex multi-classification problem rather than a binary classification problem that existing research focuses on. In addition, existing practices pay more attention to activity recognition data acquisition strategies and ignore the construction of classifiers.

[0004] As can be seen from the above, although there are many existing methods for activity recognition, these studies are troubled by the curse of dimensionality of features and the low performance of classifiers. Therefore, in view of the deficiencies of existing research, it is necessary to propose an activity recognition system to solve the activity recognition problem in practical applications. Based on this situation, the present invention mainly uses the OVO decomposition strategy, screens features using ensemble feature selection technology, and constructs a powerful activity recognition classifier system using ensemble learning technology, so as to more accurately solve the activity recognition problem and meet the actual needs of people's lives. Summary of the Invention

[0005] In view of the deficiencies of the prior art, the present invention proposes an activity recognition method and system based on an ensemble learning strategy. Physiological signals of the human body are measured by wearable small sensors, continuous signals are processed by a discretization method, features of the data set are preliminarily screened by ensemble feature selection, and the classification problem of multiple activities is decomposed into multiple classification sub-problems of two activities by the OVO decomposition strategy. Then, for each sub-problem, a certain proportion of features are randomly selected to divide the training set and the validation set. Each time, the activity classifier with the highest accuracy rate on the validation set is selected and repeated a certain number of times to construct a classifier ensemble for each sub-problem, and finally combined into a multi-class activity classification system.

[0006] To achieve the above objectives, the present invention adopts the following technical solutions:

[0007] An activity recognition technology based on an ensemble learning strategy, comprising:

[0008] S1. Data acquisition: Install sensors on key parts of the human body to collect relevant physiological signals of the human body in real time;

[0009] S2. Data preprocessing: Fuse the collected continuous physiological signals through information fusion;

[0010] S3. Ensemble feature selection: Based on feature selection, select an ensemble feature selection method suitable for activity recognition;

[0011] S4. Construct an activity automatic classification system: For the multi-classification task involved in activity recognition, select a classification system based on the decomposition strategy and ensemble learning technology for automatic recognition of activity types.

[0012] Preferably, the step S1 specifically includes:

[0013] S11. Trigger different activity states of the subject;

[0014] S12. Continuously measure the physiological signals of the human body.

[0015] Preferably, the step S2 specifically includes:

[0016] S21. Discretization of physiological signals;

[0017] S22. Feature extraction of physiological signals.

[0018] Preferably, the step S3 specifically includes:

[0019] S31. Calculate the weight of each feature in each method;

[0020] S32. Aggregation of feature selection results.

[0021] Preferably, the step S4 specifically includes:

[0022] S41. Classify the problem of classifying multiple activities into two sub-problems of activity classification;

[0023] S42. Obtain an activity classifier;

[0024] S43. Construct a multi-class activity classification system;

[0025] S44. Predict the activity category;

[0026] Furthermore, the specific steps of step S11 include:

[0027] Four activities are used to trigger four activity states of the subject: the neutral activity is triggered by watching a documentary, the emotional activity is triggered by watching a movie clip, the mental activity is triggered by playing a game, and the physical activity is triggered by exercising.

[0028] Furthermore, the specific steps of step S12 include:

[0029] When in different activity states, the subject wears a set of sensing clothing, uses a sensing glove and a bracelet to measure the electrodermal activity (EDA) data, and uses a recorder ECGZ2 to measure the electrocardiogram (ECG) and thoracic electrical bioimpedance (TEB) data. During the measurement, a device GSR is used to record the EDA measurement data, and a sensing vest is used to record the ECG and TEB data. Through the above method, the sensing system records the physiological data of each activity state of each subject, and the original data is obtained.

[0030] Furthermore, the specific steps of step S21 include:

[0031] After obtaining the original data, the present invention performs discretization processing on the continuous physiological data, takes a discrete point every few seconds from the continuous signal to obtain discrete data. At the same time, the present invention extracts features related to activity recognition from the measurement data through a wide literature review, and common ones include the standard statistical parameter set (SSSP) and the baseline, etc.

[0032] Furthermore, the specific steps of step S22 include:

[0033] The present invention extracts features related to activity recognition from the measurement data through a wide literature review, and common ones include the standard statistical parameter set (SSSP) and the baseline, etc.

[0034] Furthermore, the specific steps of step S32 include:

[0035] The present invention adopts a simple aggregation strategy, and each base feature selector screens out several important features and takes the union to obtain a preliminarily processed feature subset.

[0036] Furthermore, the specific steps of step S41 include:

[0037] The one - versus - one (OVO) method is a decomposition strategy used to solve multi - activity classification problems, which decomposes a classification problem of m activities into m(m - 1) / 2 classification sub - problems of two activities.

[0038] Further, the step S42 specifically includes:

[0039] The key part of the activity automatic classification system is to generate complementary, accurate, and diverse activity classifiers. To ensure the diversity of classifiers, for each sub - problem, the present invention randomly selects a certain proportion of features each time to obtain multiple feature subsets, and uses them to train the base learners to obtain an activity classifier pool. After obtaining the classifier pool, in order to select the best candidate classifier from it, the classifier with the highest activity classification accuracy is selected on the validation set each time.

[0040] Further, the step S43 specifically includes:

[0041] Assume that each activity classification sub - problem repeats step S42 for T times. Then each activity classification sub - problem ensemble contains T classifiers. The present invention aggregates the m(m - 1) / 2 sub - problem ensembles together to form the final multi - class ensemble, so that a multi - class activity classification system composed of T*m(m - 1) / 2 classifiers is finally formed.

[0042] Further, the step S44 specifically includes:

[0043] S441. Construct a confidence matrix

[0044] For any activity data, the activity automatic classification system submits each sample in it to each binary classifier system MCS ab to obtain a confidence level r ab , where the confidence level is determined by the number of votes of the classifier for the a - th type of activity. For the input sample O, the confidence level can be calculated according to the following formula

[0045]

[0046] where |MCS ab (a|O)| represents the number of classifiers in the activity recognition classifier of the binary classifier system MCS ab that classify the input sample O into the a - th type of activity, and |MCS ab | represents the number of activity recognition classifiers in the binary classifier system MCS ab .

[0047] Finally, a score matrix R is obtained:

[0048]

[0049] S442 Determine the classification result

[0050] The present invention follows the relative majority voting method, and the activity category with the most votes is used as the final output of the automatic classification system.

[0051]

[0052] The present invention also discloses an activity recognition system based on an ensemble learning strategy, including the following modules:

[0053] Data acquisition module: Real-time collect human-related physiological signals through sensors installed at key parts of the human body;

[0054] Data preprocessing module: Fuse the collected continuous physiological signals through information fusion;

[0055] Feature selection module: Based on feature selection, select an ensemble feature selection method suitable for activity recognition;

[0056] Activity automatic recognition system construction module: For the multi-classification tasks involved in activity recognition, select a classification system based on a decomposition strategy and ensemble learning technology for automatic recognition of activity types.

[0057] The present invention has been tested with an activity recognition data set, and can not only screen out useful features, but also improve the recognition accuracy compared with the prior art, and can more effectively solve the activity recognition problem in practical applications. Description of the Drawings

[0058] Figure 1 It is a specific flowchart of data acquisition provided by the preferred embodiment;

[0059] Figure 2 It is a specific flowchart of the activity recognition technology;

[0060] Figure 3 It is a confusion matrix of the recognition accuracy of each activity category;

[0061] Figure 4 It is a block diagram of the activity recognition system based on the ensemble learning strategy provided by the preferred embodiment. Detailed Embodiments

[0062] The following will describe in detail the specific embodiments of the present invention with reference to the drawings.

[0063] In this embodiment, a wearable small sensor is used to measure the electrocardiogram (ECG), thoracic electrical bioimpedance (TEB), and electrodermal activity (EDA) of 40 researchers, and useful features are extracted. The continuous signal is processed by discretization, the features are preliminarily screened by integrated feature selection, and a multi-class activity classification system is established by constructing an integration for each two-activity classification sub-problem to predict the sample activity category.

[0064] Specifically as follows:

[0065] An activity recognition method based on an integrated learning strategy in this embodiment, the specific process is as Figure 2 shown, including the following steps:

[0066] Step 1: Data collection; specifically as follows:

[0067] Step 11: Trigger different activity states;

[0068] The research objects selected in the present invention are 40 students and mountaineers aged 20 to 49 years old, and four activity states of the subjects are triggered by four events shown in Table 1.

[0069] Table 1 Activities and descriptions triggering four physiological states

[0070]

[0071]

[0072] Step 12: Continuously measure the physiological signals of the human body;

[0073] For each activity state of each subject, 280 seconds of continuous physiological signals are recorded in this experiment.

[0074] Step 2: Data preprocessing; specifically as follows:

[0075] Step 21: Discretization of physiological signals;

[0076] In this embodiment, a discrete point is taken from the physiological signal every 10 seconds. Therefore, 4480 discrete points, that is, 4480 pieces of data, can be obtained from the physiological signals of 40 subjects.

[0077] Step 22: Feature extraction of physiological signals;

[0078] Through extensive literature reviews, a total of 533 features are finally extracted, including 174 ECG signals, 151 TEB signals, 104 hand EDA signals, and 104 arm EDA signals. The specific method is asFigure 1 as shown

[0079] Step 3: Integrated feature selection; specifically as follows:

[0080] Step 31: Calculate the weight of each feature in each method;

[0081] In this embodiment, 13 basic feature selectors are selected, as follows:

[0082] 1. Information gain (IG) represents the degree of reduction in information uncertainty under certain conditions and is often used for feature selection. For any feature X in the feature set S k and the class label Y, the calculation equation is as follows:

[0083] IG(X k ,Y) = H(X k ) - H(X k |Y), X k ∈S (3.1)

[0084] where the information entropy H(X) represents the uncertainty of the random variable X

[0085]

[0086] where x p represents a specific value of the feature X, and P(x p ) represents the probability of taking the value x p on X

[0087] At the same time, the conditional entropy H(X k |Y) represents the uncertainty of the feature X k under the condition that the class label Y is known. Its calculation formula is as follows:

[0088]

[0089] where y q represents a specific value of the class label Y, P(y q ) represents the probability of taking the value y q on Y, and P(x p |y q ) represents the probability of x q occurring under the condition that y p occurs.

[0090] 2. Information gain ratio (GR) compensates for the bias of information gain by dividing by the intrinsic value. It prefers features with fewer values. The calculation is as follows

[0091]

[0092] 3. Symmetric uncertainty (SU) uses information entropy and conditional entropy to calculate the dependence of features. It also prefers features with fewer values. The calculation is as follows

[0093]

[0094] 4. Chi-square test (χ 2 ) determines the correctness of the theoretical value by the deviation between the observed value and the theoretical value. If the correctness is high, the theoretical value is considered correct. The calculation is as follows

[0095]

[0096] where n p,q refers to the number of samples when feature X k takes the p-th value and Y takes the q-th value, and E p,q is the expected value, NV(*) represents the number of possible values of *.

[0097] 5. The correlation coefficient is used to detect the degree of linear correlation between two variables. The larger the absolute value, the higher the degree of linear correlation. The most common one is the Pearson correlation coefficient. The calculation is as follows

[0098]

[0099] where cov(X k , Y) represents the covariance of features X k and Y, and var(*) represents the variance of *.

[0100] 6. The original Relief method randomly selects a sample I r from the sample set, and then finds t nearest neighbors from the positive and negative classes to obtain the sample sets M e (R) and M e (C), and updates the weights of each feature according to the calculation, repeating M times. ReliefF is an extension of the Relief algorithm in multi-class classification. The basic difference between it and Relief is that it selects t nearest neighbors from all classes, and the weight of the class is calculated by the proportion of the number of samples in each class to the total number of samples in the data set. The weight W[X k of feature X k is calculated as follows

[0101]

[0102] where class(I r ) represents the class of sample I r , p(c) represents the proportion of the number of samples in the c-th class to the total number, and the function diff(X k, I1, I2) Calculate the difference in the values of feature X for two samples I1 and I2 k on.

[0103]

[0104] 7. The Fisher Score is an effective feature selection method. Its main idea is to identify features with similar eigenvalue within the same class while having a large difference in eigenvalues between different classes. The calculation formula is as follows:

[0105]

[0106] where n i represents the number of samples in the i-th class, u k represents the mean of feature X k , u ki represents the mean of feature X in the i-th class of samples k , represents the variance value of feature X in the i-th class of samples k .

[0107] 8. The criterion for the Laplacian Score to judge a good feature is that the eigenvalue changes little among samples in the same class, and the degree of variation in eigenvalues is large among different samples. Therefore, the smaller the numerical fluctuation of similar samples and the larger the numerical fluctuation of different samples, the higher the Laplacian score of this feature.

[0108] First, it constructs the weight matrix S1(g, h). If samples I g and I h have the same label, then otherwise S1(g, h) = 0; then, D1 is defined as D1 = diag(S11), that is, the diagonal matrix of S11, 1 = [1, 1, 1...1] T then the Laplacian matrix L = D1 – S1. The calculation is as follows:

[0109]

[0110] where X k T represents the matrix transpose of X k ,

[0111] 9. Minimum Redundancy Maximum Relevance (MRMR) can use mutual information, correlation, or distance / similarity scores to select features. The purpose is to penalize the correlation of features through the redundancy of features in the presence of other selected features.

[0112] Among them, the correlation between each feature in the feature set S and Y is defined by the average value of all mutual information values between each feature X k and Y, as follows:

[0113]

[0114] where F(X k , Y) represents the F-statistical test

[0115] The redundancy value of all features in the feature set S is the average value of all information gains between feature X k and feature X j :

[0116]

[0117] where C(X k , X j ) represents the correlation coefficient between X k and X j .

[0118] The correlation between the MRMR criterion and the MRMR criterion is a combination of the above two measures, defined as follows:

[0119]

[0120] 10. The Gini value is an index to measure data impurity, and its definition is as follows:

[0121]

[0122] where P(x o ) represents the proportion of the o-th value in the data set D v , and NV(D v ) represents the number of possible values on the data set subset D v .

[0123] Assume that the data set D can be divided into m data set subsets according to the label value, D 1 , D 2 ,... D m . The Gini index of the feature X k is defined as:

[0124]

[0125] 11. The criterion of mutual information feature selection (MIFS) is that the feature should not only be strongly correlated with the class label, but also should not be highly correlated with the remaining features with each other. In other words, the correlation between features should be minimized. Therefore, during the feature selection stage, both feature correlation and feature redundancy are considered. The score of the feature X k can be expressed as:

[0126]

[0127] Penalty term Select a feature X with a high mutual information with the current feature j to minimize feature redundancy. Among them, IG(X k , X j ) is the information gain of feature X k and X j , and β is between 0 and 1.

[0128] 12. The idea of Conditional Mutual Information Feature Extraction (CIFE) is that as long as the feature redundancy given the class label is stronger than the within-feature redundancy, feature selection will be negatively affected. Using this idea, the score of the new unselected feature X k is

[0129]

[0130] where IG(X j , X k |Y) represents the conditional information gain of X j , X k when the third variable Y is given, and its calculation formula is as follows:

[0131] IG(X, Y|Z) = H(X|Z) - H(X|Y, Z) (3.19)

[0132] 13. Dual-Input Symmetric Relevance (DISR) uses a normalization technique to normalize the mutual information:

[0133]

[0134] where H(X k X j Y) represents the joint entropy of the three, that is, the degree of certainty of the three occurring together.

[0135] IG(X k X j , Y) = IG(X j , Y) + IG(X k , Y|X j ) (3.21)

[0136] Step 32: Aggregation of feature selection results;

[0137] Calculate the weights through these feature selection methods, select the top 10 important attributes and take the union to obtain more than 50 preliminarily screened features. Among them, Table 2 only shows the top 15 features with the most selection times.

[0138] Table 2 Selected Features (Partial)

[0139]

[0140]

[0141] Step 4: Construction of the multi-classifier system; specifically as follows:

[0142] Step 41: Divide the multi-class problem into binary-class sub-problems;

[0143] Since this data set is a classification problem of 4 activities, it is divided into 6 binary-activity classification sub-problems according to the OVO method.

[0144] Step 42: Obtaining the activity classifier pool;

[0145] The base learners used in the present invention include six activity classifiers, and each classifier and its introduction are as follows:

[0146] · Support Vector Machine (SVM): SVM is a binary-classification model. Its basic model is a linear classifier that defines the maximum margin in the feature space. The use of the kernel function enables it to solve non-linear problems.

[0147] · Decision Tree (DT): DT is a method for approximating discrete function values. It uses an induction algorithm to process data to generate readable rules and decision trees, and then uses the tree to analyze new data.

[0148] · Naive Bayes Classifier (NB): NB is a method based on Bayes' rule. It assumes that the feature conditions are independent of each other and learns the joint probability distribution from input to output through the given training set.

[0149] · k-Nearest Neighbor (kNN): For an input sample, kNN finds the k instances in the training set that are closest to the instance and classifies the input instance into the class that belongs to the majority of these instances.

[0150] · Logistic Regression (LR): LR is a generalized linear regression. It uses maximum likelihood to estimate parameters to make different types of samples linearly separable in the classification problem.

[0151] · Neural Network (NN): NN is an algorithmic mathematical model that mimics the behavioral characteristics of animal neural networks and performs distributed parallel information processing. It adjusts the relationships between a large number of internal nodes to achieve the purpose of processing information.

[0152] This embodiment uses 40 base learners to select the best classifier. The specific parameter configurations of each base learner are shown in Table 3, and all experiments are carried out using the third-party module sklearn in Python.

[0153] Table 3 Parameter Configuration of the Base Learner

[0154]

[0155]

[0156] Step 43: Construct a multi-class activity classifier system;

[0157] Step 42 was repeated 40 times for each sub-problem, resulting in an activity automatic classification system composed of 240 classifiers.

[0158] Step 44: Predict the activity category;

[0159] Using the constructed activity automatic classification system, the activity categories of the obtained data set were predicted. The present invention uses ten-fold cross-validation ten times to compensate for the risks of a single experiment. According to the construction and prediction of the activity automatic classification system, the final average accuracy rate and confusion matrix are as Figure 3 shown.

[0160] The present invention uses the Kappa index to evaluate the performance. The specific accuracy rate and Kappa value for each time are shown in Table 4

[0161] Table 4 Accuracy Rate and Kappa Value for Each Ten-Fold Cross-Validation

[0162]

[0163]

[0164] As Figure 4 shown, this embodiment discloses an activity recognition system based on an ensemble learning strategy, including the following modules connected in sequence:

[0165] Data acquisition module: Real-time acquisition of human-related physiological signals through sensors installed at key parts of the human body;

[0166] Data preprocessing module: Fusion of the collected continuous physiological signals through information fusion;

[0167] Feature selection module: Based on feature selection, select an integrated feature selection method suitable for activity recognition;

[0168] Activity automatic recognition system construction module: For the multi-classification tasks involved in activity recognition, select a classification system based on the decomposition strategy and ensemble learning technology for automatic recognition of activity types.

[0169] The specific embodiments described herein are merely illustrative of the spirit of the present invention. Those skilled in the art can combine the above embodiments, or make various modifications or supplements to different data sets and the specific embodiments described, or use similar ways to substitute, but will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

Claims

1. An activity recognition method based on an ensemble learning strategy, characterized in that: Follow the steps below: S1. Install sensors on key parts of the human body to collect relevant physiological signals of the human body in real time; S2. Through information fusion, fuse the collected continuous physiological signals; S3. Based on feature selection, select an integrated feature selection method suitable for activity recognition; S4. For the multi-classification tasks involved in activity recognition, select a classification system based on the decomposition strategy and integrated learning technology for automatic recognition of activity types; Step S4 is specifically as follows: S41. Adopt a one-versus-one decomposition strategy to decompose m activity recognition tasks into m(m - 1) / 2 pairwise activity recognition subtasks; S42. For each activity recognition subtask, randomly select a certain proportion of feature subsets, train a candidate classifier set on this feature subset, and then select the optimal classifier among them. Repeat the above steps T times until T sub-classifiers are obtained, and finally form a multi-class activity automatic recognition system composed of T*m(m - 1) / 2 classifiers; S43. In the stage of automatically identifying the activity type, submit the physiological signal to the system, and each sub-classifier system MCS ab calculates the corresponding output confidence r ab , where the confidence is determined by the number of votes of the classifier for the activity of the a-th class; for the input sample O, the confidence is calculated according to the following formula: where |MCS ab (a|O) | represents the number of classifiers in the binary classifier system MCS that classify the input sample O into class a activities, |MCS ab and |MCS ab | represents the number of activity recognition classifiers in the binary classifier system MCS; ab ​ Finally, obtain the score matrix R: Follow the plurality voting method, and the activity category with the most votes is used as the final output of the activity automatic classification system; 2. The activity recognition method based on an integrated learning strategy according to claim 1, wherein: Step S1 is specifically as follows: S11. The human body wears a sensing garment, uses a sensing glove and bracelet to measure skin electrical activity data, uses a recorder ECGZ2 to measure electrocardiogram and thoracic impedance data; uses a device GSR to record EDA measurement data, and uses a sensing vest to record ECG and TEB data; S12. By changing the state of the human body, use sensors to collect physiological data of each human body in different activity states.

3. The activity recognition method based on an integrated learning strategy according to claim 2, wherein: Step S2 is specifically as follows: S21. After obtaining the original physiological data, take a discrete point from the continuous physiological signal every few seconds to obtain discretized data; S22. Adopt methods including median, standard deviation, and baseline indicators to extract feature data related to activity recognition from the measurement data.

4. The activity recognition method based on an integrated learning strategy according to claim 3, characterized in that: Step S3 is specifically as follows: S31. Calculate the importance of each feature for activity recognition based on the feature importance measurement method; S32. Sort according to the feature importance under different feature measurement methods, screen out important feature subsets according to the rules, and then perform a merging operation on the obtained feature subsets to complete the feature selection operation.

5. An activity recognition system based on an ensemble learning strategy for implementing an activity recognition method based on an ensemble learning strategy according to any one of claims 1-4, characterized in that, The activity recognition system based on the integrated learning strategy includes the following modules: Data acquisition module: Through sensors installed on key parts of the human body, collect relevant physiological signals of the human body in real time; Data preprocessing module: Through information fusion, fuse the collected continuous physiological signals; Feature selection module: Based on feature selection, select an integrated feature selection method suitable for activity recognition; Activity automatic recognition system construction module: For the multi-classification tasks involved in activity recognition, select a classification system based on the decomposition strategy and integrated learning technology for automatic recognition of activity types.

Images