A method for motion recognition

Abstract

The present invention belongs to the field of human-computer interaction technology, and relates to a method for motion recognition, comprising the following steps: collecting multiple motion signals as an original input set; performing activity segment detection on each motion signal in the original input set one by one to obtain a valid signal; performing feature extraction based on the valid signal to obtain a valid sample feature set; constructing a multi-layer motion recognition model based on the valid sample feature set; collecting the motion signal to be tested, sending the motion signal to be tested into the constructed multi-layer motion recognition model, performing motion recognition, and outputting the recognition result. The present invention is based on the fusion of multiple classifiers, and proposes a rejection decision algorithm of multi-layer decision fusion. For irregular operations or non-target actions, secondary discrimination is performed. In the case where two layers of classifiers and two confidence decisions are rejected, the test sample feature will be identified as a non-target action. It has strong practicality and is worth promoting.

Description

Technical Field

[0001] The invention belongs to the technical field of human-computer interaction, and in particular relates to a motion recognition method. Background Art

[0002] In recent years, with the advancement of science and technology, human-computer interaction based on motion recognition has become more and more diversified, and its application scope has become more and more extensive. Although the current human motion recognition system has achieved a high recognition accuracy rate using various feature extraction algorithms and classifier algorithms, there are still certain problems. First, there is still room for improvement in the recognition accuracy of actions. Secondly, for actions that are not predefined in the classifier, they will still be forcibly classified into predefined categories and identified as target actions. In addition, for actions that are not operated in a standardized manner, there is a high probability that they will be mistakenly identified as other actions. If the motion recognition system is actually applied, there is a possibility of misoperation of the application object, leading to accidents. Summary of the invention

[0003] In view of this, the present invention provides a motion recognition method to solve the above-mentioned technical problem.

[0004] The technical solution of the present invention is:

[0005] A method for motion recognition comprises the following steps:

[0006] Collect multiple action signals as original input sets;

[0007] Perform activity segment detection on each action signal in the original input set one by one to obtain valid signals;

[0008] Extract features based on effective signals to obtain effective sample feature sets;

[0009] Based on the effective sample feature set, a multi-layer action recognition model is constructed;

[0010] The motion signal to be tested is collected, and the motion signal to be tested is sent to the constructed multi-layer motion recognition model to perform motion recognition, and the recognition result is output.

[0011] Preferably, the motion signal is collected based on the motion sensor signal or electromyographic signal of the wearable device.

[0012] Preferably, extracting features based on effective signals to obtain effective sample feature sets includes the following steps:

[0013] The effective signal of each action is divided into windows for feature extraction, sub-features are obtained, and the sub-features are combined to obtain the effective sample feature set of the action.

[0014] Preferably, the multi-layer action recognition model includes: a k-nearest neighbor classification model and a Gaussian mixture model classifier, wherein the k-nearest neighbor classification model is the first decision layer and the Gaussian mixture model classifier is the second decision layer.

[0015] Preferably, the construction of the k nearest neighbor classification model comprises the following steps:

[0016] Assume that the training sample feature set X is {(X1,Y1),(X2,Y2),...,(X C ,Y C )}, where C is the number of defined action categories; c = 1, 2, ..., C is the c-th type of hand action; X c is the signal feature set of the c-th type of hand action; Y c is the category label of the cth type of hand action;

[0017] The training sample feature set X is stored offline according to the sample category.

[0018] Preferably, the construction of the Gaussian mixture model classifier comprises the following steps:

[0019] Assume that the training sample feature set X is {(X1,Y1),(X2,Y2),...,(X C ,Y C )}, where C is the number of defined action categories; c = 1, 2, ..., C is the c-th type of hand action; X c is the signal feature set of the c-th type of hand action; Y c is the category label of the cth type of hand action;

[0020] For the C-type hand movements, the training sample feature set X c ={x1,x2,...,x N}, c = 1, 2, ..., C respectively train C Gaussian mixture model classifiers.

[0021] Preferably, the training method of the Gaussian mixture model classifier is as follows:

[0022] 1) Using the signal feature set Xc of the c-th hand action, an initial value estimation is performed on the unknown parameters in the Gaussian mixture model classifier of the c-th action. The initial value estimation method includes the following steps:

[0023] a) From the signal feature set X of the c-th hand action c Randomly select K samples as the center μ of K Gaussian components k (k=1, 2, ..., K);

[0024] b) Using formula (1), we can sequentially extract the signal feature set X of the c-th hand action. cFor each sample x in i (i=1,2,..,N), calculate its center μ with the Gaussian component k (k=1, 2, ..., K), and then sort the calculated N distance values ​​from large to small, select the smallest distance value, and mark it as the component with the closest distance;

[0025]

[0026] c) The center μ of each Gaussian component k (k=1, 2, ..., K) is updated to the mean of all samples belonging to this component;

[0027] d) Return to b) and execute sequentially until the number of iterations N-1 is met or the clustering results of each sample no longer change, so that the center μ of each Gaussian component can be obtained. k ;

[0028] e) Calculate the covariance matrix ∑ using formula (2) k ,

[0029]

[0030] in,

[0031] x i is the signal feature set X of the c-th hand action c Each sample in

[0032] μ k is the center of the Gaussian component;

[0033] N k is the number of samples classified into the kth Gaussian component;

[0034] f) Use formula (3) to calculate the weight w of each Gaussian component j ,

[0035]

[0036] in,

[0037] w j is the weight of the Gaussian component;

[0038] N is the total number of samples of category c;

[0039] N j is the total number of samples belonging to the jth Gaussian component, satisfying

[0040] 2) Re-estimating the unknown parameters in the Gaussian mixture model classifier of the c-th action, including the following steps:

[0041] For the kth Gaussian component, use formula (4) to re-estimate its weight π k , mean μ k and covariance estimate∑ k ,

[0042]

[0043] in,

[0044]

[0045] γ(i,k) is the i , which is the probability generated by the kth Gaussian component;

[0046] π k is the weight, μ k is the mean, ∑ k is the covariance estimate;

[0047] Return to step 2) and continue until the maximum number of iterations is reached or the error is less than a preset threshold, thereby obtaining the parameters of C GMM classifiers;

[0048] 3) Calculate the probability density function of the Gaussian mixture model classifier of the c-th action according to formula (5):

[0049]

[0050] in,

[0051]

[0052] w j is the weight of the jth Gaussian component, satisfying

[0053] N(x i |μ j ,∑ j ) is the probability density function of the jth Gaussian component;

[0054] K is the number of Gaussian components in the Gaussian mixture model classifier.

[0055] Preferably, the motion signal to be tested is subjected to motion recognition, using a multi-layer rejection decision algorithm.

[0056] Preferably, the multi-layer rejection decision algorithm comprises the following steps:

[0057] 3) Initial judgment of k nearest neighbor classification

[0058] For the test sample feature X test , respectively calculate and test sample features X testThe category of the training sample with the closest distance to the M data points in the dataset is assigned, and the decision confidence value r is calculated according to the following formula (6):

[0059]

[0060] in,

[0061] N i For X test Belongs to category ω i The number of sample points;

[0062] N test For X test The number of data points;

[0063] r is the decision confidence value;

[0064] Sort the multiple decision confidence values ​​r calculated and select the largest decision confidence value r max , determine the maximum decision confidence value r max Categoryω i , r max Compared with the predefined threshold δ1, if r max is greater than the predefined threshold δ1, then the test sample feature X test Belongs to category ω i , otherwise continue to execute;

[0065] 4) GMM secondary discrimination

[0066] When the parameters λ of the C group of Gaussian mixture model classifiers (c = 1, 2, ..., C) are known, the new input sample feature X test , calculate the posterior probability P(X test |λ c ),

[0067]

[0068] According to formula (8), the calculated multiple posterior probabilities P(X test |λ c ) to sort and select the largest posterior probability P(X test |λ c ), X test The category belongs to

[0069]

[0070] If ω i =ω j , then the test sample feature X test Belongs to category ω iOr j , otherwise continue to execute; 3) Differential posterior probability judgment

[0071] The posterior probability P_(X test |λ c ),

[0072]

[0073] Where P = [P (X test |λ1)P(X test |λ2),...,P(X test |λ C )] T ;

[0074] The posterior probability P_(X test |λ c ) are sorted from large to small to obtain the sorted posterior probability vector P_d, and the first two elements of the posterior probability vector P_d are taken out and the difference is calculated. When the difference satisfies formula (10), the test sample feature X test belongs to category j, otherwise, it belongs to the rejection category ω0,

[0075] |P_d(1)-P_d(2)|>δ2 (10)

[0076] in,

[0077] δ2 is a predefined threshold;

[0078] P_d(1) is the first element after sorting the posterior probability vector P_d from large to small;

[0079] P_d(2) is the second element after sorting the posterior probability vector P_d from large to small.

[0080] The present invention provides an action recognition method, which can detect actions that are not predefined in the classifier during the training stage, or action signals generated due to improper operation of the operator, with a high accuracy through a set multi-layer rejection decision algorithm, thereby avoiding the possibility of misoperation. First of all, the present invention proposes a rejection decision algorithm of multi-layer decision fusion based on the fusion of multiple classifiers. For each layer of classifiers, starting from the classifier's own algorithm, the concept of decision confidence value r is proposed according to the decision result, and for more standardized target actions, it can be accurately detected at a faster speed. For irregular operations or non-target actions, secondary discrimination is performed. In the case of two layers of classifiers, when both confidence decisions are rejected, the test sample feature will be identified as a non-target action, which is highly practical and worthy of promotion. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 is a flow chart of identifying the characteristics of a test sample according to the present invention;

[0082] Figure 2 It is a method for constructing an action recognition model of the present invention;

[0083] Figure 3 It is a flow chart of the multi-layer rejection decision recognition algorithm of the present invention. DETAILED DESCRIPTION

[0084] The present invention provides an action recognition method, which is combined with Figure 1 to Figure 2 The present invention is described with reference to the structural schematic diagram of FIG.

[0085] Example 1

[0086] A method for motion recognition comprises the following steps:

[0087] Collect multiple action signals as original input sets;

[0088] Perform activity segment detection on each action signal in the original input set one by one to obtain valid signals;

[0089] Extract features based on effective signals to obtain effective sample feature sets;

[0090] Based on the effective sample feature set, a multi-layer action recognition model is constructed;

[0091] The motion signal to be tested is collected, and the motion signal to be tested is sent to the constructed multi-layer motion recognition model to perform motion recognition, and the recognition result is output.

[0092] Among them, feature extraction based on effective signals to obtain effective sample feature sets includes the following steps:

[0093] The effective signal of each action is divided into windows for feature extraction, sub-features are obtained, and the sub-features are combined to obtain the effective sample feature set of the action.

[0094] The multi-layer action recognition model includes: a k-nearest neighbor classification model and a Gaussian mixture model classifier, wherein the k-nearest neighbor classification model is the first decision layer and the Gaussian mixture model classifier is the second decision layer.

[0095] This method is applicable to motion sensing signals or electromyographic signals based on wearable devices and is divided into two parts:

[0096] The first part is the construction of a multi-layer action recognition model, including the offline storage of training sample feature sets applied to the k-nearest neighbor classification model and the parameter estimation of the Gaussian mixture model classifier, as shown in the attached figure. Figure 2 As shown; the second part is the decision-making and discrimination of the test sample features based on the multi-layer action recognition model, as shown Figure 1 As shown in FIG. 1 and 2, both the first and second parts involve data collection of original signals, activity segment detection and feature extraction.

[0097] Among them, the construction of the k nearest neighbor classification model includes the following steps:

[0098] Assume that the training sample feature set X is {(X1,Y1),(X2,Y2),...,(X C ,Y C )}, where C is the number of defined action categories; c = 1, 2, ..., C is the c-th type of hand action; X c is the signal feature set of the c-th type of hand action; Y c is the category label of the cth type of hand action;

[0099] The training sample feature set X is stored offline according to the sample category.

[0100] Among them, the construction of the Gaussian mixture model classifier includes the following steps:

[0101] Assume that the training sample feature set X is {(X1,Y1),(X2,Y2),...,(X C ,Y C )}, where C is the number of defined action categories; c = 1, 2, ..., C is the c-th type of hand action; X c is the signal feature set of the c-th type of hand action; Y c is the category label of the cth type of hand action;

[0102] For the C-type hand movements, the training sample feature set X c ={x1,x2,...,x N}, c = 1, 2, ..., C respectively train C Gaussian mixture model classifiers.

[0103] The training method of the Gaussian mixture model classifier is as follows:

[0104] 1) Using the signal feature set Xc of the c-th hand action, an initial value estimation is performed on the unknown parameters in the Gaussian mixture model classifier of the c-th action. The initial value estimation method includes the following steps:

[0105] a) From the signal feature set X of the c-th hand action c Randomly select K samples as the center μ of K Gaussian components k(k=1, 2, ..., K);

[0106] b) Using formula (1), we can sequentially extract the signal feature set X of the c-th hand action. c For each sample x in i (i=1,2,..,N), calculate its center μ with the Gaussian component k (k=1, 2, ..., K), and then sort the calculated N distance values ​​from large to small, select the smallest distance value, and mark it as the component with the closest distance;

[0107]

[0108] c) The center μ of each Gaussian component k (k=1, 2, ..., K) is updated to the mean of all samples belonging to this component;

[0109] d) Return to b) and execute sequentially until the number of iterations N-1 is met or the clustering results of each sample no longer change, so that the center μ of each Gaussian component can be obtained. k ;

[0110] e) Calculate the covariance matrix ∑ using formula (2) k ,

[0111]

[0112] in,

[0113] x i is the signal feature set X of the c-th hand action c Each sample in

[0114] μ k is the center of the Gaussian component;

[0115] N k is the number of samples classified into the kth Gaussian component;

[0116] f) Use formula (3) to calculate the weight w of each Gaussian component j ,

[0117]

[0118] in,

[0119] w j is the weight of the Gaussian component;

[0120] N is the total number of samples of category c;

[0121] N j is the total number of samples belonging to the jth Gaussian component, satisfying

[0122] 2) Re-estimating the unknown parameters in the Gaussian mixture model classifier of the c-th action, including the following steps:

[0123] For the kth Gaussian component, use formula (4) to re-estimate its weight π k , mean μ k and covariance estimate∑ k ,

[0124]

[0125] in,

[0126]

[0127] γ(i,k) is the i , which is the probability generated by the kth Gaussian component;

[0128] π k is the weight, μ k is the mean, ∑ k is the covariance estimate;

[0129] Return to step 2) and continue until the maximum number of iterations is reached or the error is less than a preset threshold, thereby obtaining the parameters of C GMM classifiers;

[0130] 3) Calculate the probability density function of the Gaussian mixture model classifier of the c-th action according to formula (5):

[0131]

[0132] in,

[0133]

[0134] w j is the weight of the jth Gaussian component, satisfying

[0135] N(x i |μ j ,∑ j ) is the probability density function of the jth Gaussian component;

[0136] K is the number of Gaussian components in the Gaussian mixture model classifier.

[0137] The second part is the decision-making and judgment of the test sample features based on the multi-layer action recognition model, such as Figure 2 As shown in Figure 1, after completing the signal acquisition of the action, it is also necessary to perform activity segment detection and feature extraction on the signal to obtain the feature representation X of the action. test , then Figure 3 As shown, the multi-layer rejection decision algorithm of the present invention is used to test The method for category judgment is as follows:

[0138] 1) Initial judgment of k nearest neighbor classification

[0139] For the test sample feature X test , respectively calculate and test sample features X test The category of the training sample with the closest distance to the M data points in the dataset is assigned, and the decision confidence value r is calculated according to the following formula (6):

[0140]

[0141] in,

[0142] N i For X test Belongs to category ω i The number of sample points;

[0143] N test For X test The number of data points;

[0144] r is the decision confidence value;

[0145] Sort the multiple decision confidence values ​​r calculated and select the largest decision confidence value r max , determine the maximum decision confidence value r max Categoryω i , r max Compared with the predefined threshold δ1, if r max is greater than the predefined threshold δ1, then the test sample feature X test Belongs to category ω i , otherwise continue to execute;

[0146] 2) GMM secondary discrimination

[0147] When the parameters λ of the C group of Gaussian mixture model classifiers (c = 1, 2, ..., C) are known, the new input sample feature X test , calculate the posterior probability P(X test |λ c ),

[0148]

[0149] According to formula (8), the calculated multiple posterior probabilities P(X test |λ c ) to sort and select the largest posterior probability P(X test |λc ), X test The category belongs to

[0150]

[0151] If ω i =ω j , then the test sample feature X test Belongs to category ω i Or j , otherwise continue to execute;

[0152] 3) Differential posterior probability discrimination

[0153] The posterior probability P_(X test |λ c ),

[0154]

[0155] Where P = [P (X test |λ1)P(X test |λ2),...,P(X test |λ C )] T ;

[0156] The posterior probability P_(X test |λ c ) are sorted from large to small to obtain the sorted posterior probability vector P_d, and the first two elements of the posterior probability vector P_d are taken out and the difference is calculated. When the difference satisfies formula (10), the test sample feature X test belongs to category j, otherwise, it belongs to the rejection category ω0,

[0157] |P_d(1)-P_d(2)|>δ2 (10)

[0158] in,

[0159] δ2 is a predefined threshold;

[0160] P_d(1) is the first element after sorting the posterior probability vector P_d from large to small;

[0161] P_d(2) is the second element after sorting the posterior probability vector P_d from large to small.

[0162] The present invention is based on the fusion of multiple classifiers and proposes a rejection decision algorithm of multi-layer decision fusion. For irregular operations or non-target actions, a secondary discrimination is performed according to the confidence value r. In the case of two-layer classifiers and two confidence decisions are rejected, the test sample feature will be identified as a non-target action. The invention has strong practicality and is worthy of promotion.

[0163] What is disclosed above is only a preferred specific embodiment of the present invention; however, the embodiments of the present invention are not limited thereto, and any changes that can be conceived by those skilled in the art should fall within the protection scope of the present invention.

Claims

1. A method for motion recognition, characterized in that: The following steps are involved: Collect multiple action signals as original input sets; Perform activity segment detection on each action signal in the original input set one by one to obtain valid signals; Extract features based on effective signals to obtain effective sample feature sets; Based on the effective sample feature set, a multi-layer action recognition model is constructed; Collect the motion signal to be tested, send the motion signal to the constructed multi-layer motion recognition model, perform motion recognition, and output the recognition result; The multi-layer action recognition model includes: a k-nearest neighbor classification model and a Gaussian mixture model classifier, wherein the k-nearest neighbor classification model is the first decision layer and the Gaussian mixture model classifier is the second decision layer; Perform action recognition on the action signal to be tested, using a multi-layer rejection decision algorithm; The multi-layer rejection decision algorithm comprises the following steps: 1) Initial judgment of k nearest neighbor classification For the test sample feature X test , respectively calculate and test sample features X test The category of the training sample with the closest distance to the M data points is assigned, and the decision confidence value r is calculated according to the following formula (6): in, N i For X test Belongs to category ω i The number of sample points; N test For X test The number of data points; r is the decision confidence value; Sort the multiple decision confidence values ​​r calculated and select the largest decision confidence value r max , determine the maximum decision confidence value r max Categoryω i , r max Compared with the predefined threshold δ1, if r max is greater than the predefined threshold δ1, then the test sample feature X test Belongs to category ω i , otherwise continue to execute; 2) GMM secondary discrimination When the parameters λ of the C group of Gaussian mixture model classifiers (c = 1, 2, ..., C) are known, the new input sample feature X test , calculate the posterior probability P(X test |λ c ), According to formula (8), the calculated multiple posterior probabilities P(X test |λ c ) to sort and select the largest posterior probability P(X test |λ c ), X test The category belongs to If ω i =ω j , then the test sample feature X test Belongs to category ω i Or j , otherwise continue to execute; 3) Differential posterior probability judgment The posterior probability P_(X test |λ c ), where P = [P(X test |λ1)P(X test |λ2),...,P(X test |λ C )] T ; The posterior probability P_(X test |λ c ) Sort from large to small to get the sorted posterior probability vector P _ d, take out the posterior probability vector P _ The first two elements of d are taken and their difference is calculated. When the difference satisfies formula (10), the test sample feature X test belongs to category j, otherwise, it belongs to the rejection category ω0, |P_d(1)-P_d(2)|>δ2 (10) in, δ2 is a predefined threshold; P_d(1) is the posterior probability vector P _ d is the first element after sorting from largest to smallest; P_d(2) is the posterior probability vector P _ d is the second element after sorting from largest to smallest.

2. The action recognition method according to claim 1, characterized in that: The action signals are collected based on the motion sensor signals or electromyographic signals of wearable devices.

3. The method for motion recognition according to claim 1, characterized in that: Feature extraction based on effective signals and obtaining effective sample feature sets include the following steps: The effective signal of each action is divided into windows for feature extraction, sub-features are obtained, and the sub-features are combined to obtain the effective sample feature set of the action.

4. The method for motion recognition according to claim 1, characterized in that: The construction of the k nearest neighbor classification model includes the following steps: Assume that the training sample feature set X is {(X1,Y1),(X2,Y2),...,(X C ,Y C )}, where C is the number of defined action categories; c = 1, 2, ..., C is the c-th type of hand action; X c is the signal feature set of the c-th type of hand action; Y c is the category label of the cth type of hand action; The training sample feature set X is stored offline according to the sample category.

5. The method for motion recognition according to claim 4, characterized in that: The construction of the Gaussian mixture model classifier includes the following steps: Assume that the training sample feature set X is {(X1,Y1),(X2,Y2),...,(X C ,Y C )}, where C is the number of defined action categories; c = 1, 2, ..., C is the c-th type of hand action; X c is the signal feature set of the c-th type of hand action; Y c is the category label of the cth type of hand action; For the C-type hand movements, the training sample feature set X c ={x1,x2,...,x N }, c = 1, 2, ..., C respectively train C Gaussian mixture model classifiers.

6. The method for motion recognition according to claim 5, characterized in that: The training method of the Gaussian mixture model classifier is as follows: 1) Using the signal feature set Xc of the c-th hand action, an initial value estimation is performed on the unknown parameters in the Gaussian mixture model classifier of the c-th action. The initial value estimation method includes the following steps: a) From the signal feature set X of the c-th hand action c Randomly select K samples as the center μ of K Gaussian components k (k=1, 2, ..., K); b) Using formula (1), we can sequentially extract the signal feature set X of the c-th hand action. c For each sample x in i (i=1,2,..,N), calculate its center μ with the Gaussian component k (k=1, 2, ..., K), and then sort the calculated N distance values ​​from large to small, select the smallest distance value, and mark it as the component with the closest distance; c) The center μ of each Gaussian component k (k=1, 2, ..., K) is updated to the mean of all samples belonging to this component; d) Return to b) and execute sequentially until the number of iterations N-1 is met or the clustering results of each sample no longer change, so that the center μ of each Gaussian component can be obtained. k ; e) Calculate the covariance matrix ∑ using formula (2) k , in, x i is the signal feature set X of the c-th hand action c Each sample in μ k is the center of the Gaussian component; N k is the number of samples classified into the kth Gaussian component; f) Use formula (3) to calculate the weight w of each Gaussian component j , in, w j is the weight of the Gaussian component; N is the total number of samples of category c; N j is the total number of samples belonging to the jth Gaussian component, satisfying 2) Re-estimating the unknown parameters in the Gaussian mixture model classifier of the c-th action, including the following steps: For the kth Gaussian component, use equation (4) to re-estimate its weight π k , mean μ k and covariance estimate∑ k , in, γ(i,k) is the i , which is the probability generated by the kth Gaussian component; π k is the weight, μ k is the mean, Σ k is the covariance estimate; Return to step 2) and continue until the maximum number of iterations is reached or the error is less than a preset threshold, thereby obtaining the parameters of C GMM classifiers; 3) Calculate the probability density function of the Gaussian mixture model classifier of the c-th action according to formula (5): in, w j is the weight of the jth Gaussian component, satisfying N(x i |μ j ,∑ j ) is the probability density function of the jth Gaussian component; K is the number of Gaussian components in the Gaussian mixture model classifier.

Images