An improved label consistent based multi-channel air gesture dictionary learning method

Abstract

The application relates to a multi-channel air gesture dictionary learning method based on improved label consistency, and belongs to the technical field of human-computer interaction. Multi-channel data representing multi-class gesture actions are collected and acquired, data preprocessing is performed on the data, a multi-channel air gesture dictionary learning model based on improved label consistency is constructed, a training set is input into the model to train the model, and model parameters are adjusted through a test set, optimal recognition model is utilized to complete experimental evaluation of the multi-channel air gesture dictionary learning recognition model based on improved label consistency, and the best recognition rate of the model for different gestures is output. The application has the advantages that the real-time requirement of three-dimensional air gesture recognition can be better met, the problem of low model robustness and low gesture label recognition rate caused by insufficient feature extraction of a single sensor channel can be effectively avoided by increasing the diversity of collected gesture data features.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of human-computer interaction, in particular to a method for processing multi-channel data by using a dictionary learning classification method based on improved label consistency, wherein the multi-channel data includes three-dimensional ultrasonic coordinate data, three-axis acceleration data and a quaternion. BACKGROUND

[0002] Three-dimensional air pen gesture writing recognition technology is a new human-computer interaction technology, which aims to input or control by recognizing the pen gesture or hand gesture of the user writing in the air. Air pen gesture writing improves the degree of freedom of human-computer interaction and improves the human-computer interaction experience, but at the same time, it puts forward higher requirements for the accuracy and efficiency of its interaction. Efficient and flexible three-dimensional interaction technology, as the main development direction of future human-computer interaction, attracts the attention of many researchers. As one of the important human-computer interaction methods, three-dimensional pen interaction has been widely used in various fields with the maturation of computer technology, and pen gesture recognition research has become a hot research direction.

[0003] With the increasing interest in virtual reality and three-dimensional space human-computer interaction, unlike the traditional two-dimensional user graphical interface, the mouse and keyboard input signal realization system control, three-dimensional space virtual reality human-computer interaction input tends to increase the user's real experience, and the amount of information provided by the user as the input mapping to the virtual space is more massive, accompanied by a large amount of data storage and calculation. As an input source, the user currently mainly uses the hand and head mapping. In the process of using the user's hand as input, the pen gesture recognition can provide more accurate information mapping such as handwriting and graphics than hand gesture recognition and is more in line with human natural writing habits. Three-dimensional air pen interaction has the characteristics of three-dimensional space, natural interaction and efficient interaction, but the existing pen gesture recognition method puts forward many restrictions on the user in the interaction process and cannot fully play the characteristics of three-dimensional pen interaction, resulting in a significant decrease in the user's actual interaction experience. In addition, the current pen gesture recognition research mainly obtains pen gesture data through a single channel, which is easily affected by device errors and external noise interference, and cannot guarantee high robustness in the recognition system. For example, the method of using an accelerometer, an ultrasonic sensor, etc. as a single channel to complete pen gesture recognition has a single feature attribute, and when completing a three-dimensional space pen gesture action with multiple attributes, it cannot guarantee the robustness of the method. The multi-channel pen gesture recognition method combines three-dimensional ultrasonic coordinate data and acceleration data that can describe the trajectory shape feature of the pen gesture action, and adds a quaternion to represent the pen gesture, effectively improving the robustness of the system. SUMMARY

[0004] The application provides a multi-channel air gesture dictionary learning method based on improved label consistency to solve the problems of inaccurate feature extraction and slow recognition speed in a three-dimensional gesture recognition method.

[0005] The technical scheme adopted by the application comprises the following steps:

[0006] Step 1. Multi-class gesture data samples are collected by holding an electronic pen to complete a predetermined gesture action, and then multi-channel data representing the multi-class gesture action are obtained;

[0007] Step 2. The multi-channel data corresponding to the multi-class gestures are preprocessed;

[0008] Step 3. An improved label consistency-based multi-channel air gesture dictionary learning model is constructed;

[0009] Step 4. The training set is input to train the model, and the model parameters are adjusted through the test set;

[0010] Step 5. The optimal recognition model is used to complete the experimental evaluation of the improved label consistency-based multi-channel air gesture dictionary learning recognition model, and the best recognition rate of the model for different gestures is output.

[0011] The data collection method in step 1 of the application is as follows:

[0012] The experimenter holds an electronic pen equipped with a nine-axis inertial sensor to perform a predetermined gesture action, presses the start button to start recording gesture data and transmits the gesture data to the upper computer, presses the end button to complete the collection of a single gesture data sample, and repeats the above steps to complete the collection of the entire gesture data set.

[0013] The data preprocessing method in step 2 of the application is as follows:

[0014] Step 2.1: Gesture data sample screening. Since the collected gesture data has problems of data point redundancy and data pollution, and the gesture feature information contained in the too few sampling points is insufficient, it is necessary to remove duplicate points and eliminate gesture samples with too short length sequences;

[0015] First, all gesture sample sequences are read in a loop, and the gesture sequence without data preprocessing is set as D[m] = d1, d2,.., d m , where m is the sequence length, the i-th sampling point is d i , and then it is judged whether the i-th sampling point d i and the i+1-th sampling point d i+1 are equal. If they are equal, the d i+1 data point is deleted, and then the gesture sequence after removing the duplicate points is D[k] = d1, d2,.., d kwhere D[k] represents the de-duplication stroke sequence, k≤m;

[0016] Finally, determine the relationship between the length K of the de-duplication stroke sequence and the sampling point threshold. If K is greater than the sampling point threshold, retain the stroke sequence sample. If K is less than the sampling point threshold, delete the stroke sequence sample.

[0017] Step 2.2: Stroke data resampling. In order to solve the problems of non-uniform stroke sample sequence length and information redundancy caused by excessively long stroke sample sequence length, and to meet the needs of subsequent dictionary learning and sparse representation, the stroke data is resampled by linear interpolation.

[0018] Determine the fixed length of the resampled stroke sequence, i.e., the number of sampling points n. Since the number of sampling points of the original stroke sequence is not equal and is less than k according to step 2.1, the number of sampling points after resampling is n, which is calculated as follows:

[0019]

[0020] where x represents the current position sequence after target resampling, y represents the coordinate value corresponding to the resampled sequence, x1 and x2 are the horizontal coordinates of the two adjacent sampling points in the original stroke data sequence close to the current sampling point x, and y2 and y1 are the coordinate values corresponding to x1 and x2 in the original stroke data sequence.

[0021] Step 2.3: Stroke data smoothing. In order to remove noise interference and smooth the sharp edges of the data, a moving average filter is used to smooth the stroke data, which is calculated as follows:

[0022]

[0023] where H′ r [n] is the stroke sequence after step 2.1 and step 2.1 processing before filtering, H b [n] is the data sequence after moving average filtering, and the above formula is used to average filter the three-dimensional ultrasonic coordinate data and three-axis acceleration data in the stroke data, respectively. N is the number of sampling points in the moving average filter.

[0024] Step 2.4: Ultrasonic feature extraction. After step 2.3, the H b [n] data sequence contains ultrasonic data, acceleration, and quaternion data, and the ultrasonic coordinate sequence S[n] = S1, S2, …, S i ,…,S n ,1≤i≤n is taken as S i = (x i ,y i ,z i) is the three-axis ultrasonic coordinate of three-dimensional pen gesture for any point, n is the sequence length thereof;

[0025] calculating three-dimensional coordinate S of trajectory center point c = (x c ,y c ,z c ) :

[0026]

[0027] calculating the Euclidean distance of each group of ultrasonic coordinates s i = (x i ,y i ,z i ) in three-dimensional pen gesture ultrasonic coordinate sequence S[n] to trajectory center point S c = (x c ,y c ,z c ) in turn:

[0028]

[0029] obtaining trajectory center distance encoding sequence d n , d[n] = d1, d2, …, d n , …, d N ;

[0030] calculating trajectory full length L, and accumulating the sum of the spatial straight line distance of adjacent two points in pen gesture sequence as the full length of pen gesture trajectory:

[0031]

[0032] normalizing the trajectory center encoding by using the trajectory full length to obtain normalized trajectory center distance encoding sequence C[n]:

[0033]

[0034] Step 2.5: Pen gesture data normalization. Since the amplitude deviation between different pen gesture data measured by the pen gesture data acquisition device is usually not uniform, in order to normalize the amplitude of the pen gesture motion signal, the normalized method is used to process and scale the data processed in the above steps to the interval [-1, 1], and the calculation method is as follows:

[0035]

[0036] where i represents the i-th sampling point of the pen gesture sequence, H n [i] is the normalized pen gesture sequence, H[i] is the pen gesture sequence before normalization, H max represents the maximum value in the current pen gesture sequence, H minrepresents the minimum value in the current gesture sequence.

[0037] The step 3 of the application is to construct a multi-channel air gesture dictionary learning recognition model based on improved tag consistency, which includes a gesture feature extraction module, a dictionary learning module, and a classifier parameter output module.

[0038] According to the input of the air gesture data set after data preprocessing in step 2, the input gesture data contains eight dimensions, which are normalized trajectory center distance encoding sequence C[n], three-axis acceleration data A n x y z T n T ; wherein the collected gesture data needs to be divided into training set and test set after data preprocessing, and the training set and test set are divided according to the ratio of a:b, wherein a>b; a, n∈N * * N is a natural number;

[0039] The gesture feature extraction module adopts PCA feature extraction, which completes the feature extraction of the gesture data in the time dimension;

[0040] Suppose there are c types of gestures, and each type of gesture has p in the training sample. After the above preprocessing, the size of each gesture feature matrix is N=n×8. Each gesture feature matrix is expanded into an N×1 dimensional column vector, and let M=c×p be the number of training set samples. Then the total spread matrix of the training set is:

[0041]

[0042] where x i represents the i-th gesture sample in a certain class of samples in c classes of samples, and u represents the average vector of all gesture features in the training sample, which is calculated as follows:

[0043]

[0044] The training sample matrix X is constructed, and the average value u of the vector sample matrix is calculated.

[0045] The covariance matrix C=E[(x-u)(x-u) T ] is calculated, where E represents the expectation, and the eigenvalues and eigenvectors are calculated; the eigenvalues are arranged from large to small, and the first k largest eigenvalues corresponding to U i are selected as the change matrix W, and the training sample is changed y=W T ​​​​​​​X gets the dimension-reduced training sample matrix y, where W T represents the dimension-reduced matrix;

[0046] The dictionary learning module adopts an improved label consistent dictionary learning algorithm to learn a dictionary with label features, adds a label consistency constraint and an "optimal" classification performance standard of a "discriminative sparse coding error" improved by a cosine similarity to a target function, and uses a K-SVD algorithm to optimize;

[0047] The label consistent dictionary learning algorithm is as follows:

[0048]

[0049] The improved label consistent dictionary learning algorithm model is as follows:

[0050]

[0051] In the formula, parameters α and β represent the weight values of the discriminative sparse coding error, Q represents the discriminative sparse coding of the input sample, D represents the size of the dictionary, X is the sparse coefficient, Y is the total number of training samples, W represents the classifier weight parameter, A represents the linear transformation matrix, and T is the sparse threshold;

[0052] On the basis of the above formula, the initialization of the discriminative sparse coding Q value is changed by using the cosine similarity, and the dimension-reduced matrix is trained simultaneously in the dictionary learning process to better extract the discriminative information in the sample, and the specific algorithm is as follows:

[0053]

[0054] In the formula, L is the dimension-reduced matrix used to dimension-reduce the sample, Y is the training sample, D is the dictionary, X is the sparse coefficient, Q represents the discriminative sparse coding, A represents the linear transformation matrix, W is the classifier parameter, and T is the sparse threshold; parameters α, β, and γ represent the weight values of each term, control the relative contribution coefficients, and are used to adjust the proportions of the three terms in the model;

[0055] The dimension to which the sample is reduced is set, and necessary variables are initialized. First, the PCA algorithm is used to obtain the initialized dimension-reduced matrix L of the sample, M = LY is set, at this time the sample is M, the total number of categories is C, and then the size of the dictionary is set. Then, the KSVD algorithm is used to train the class dictionary of each category, and after obtaining the class dictionary of each category, the initialized dictionary D is composed of the class dictionaries (0) , and then the sparse coefficient X of the sample is obtained by using the l1 or l2 norm method according to the dictionary;

[0056] A multivariate ridge regression model is used to obtain the initialized transformation matrix A (0) , as follows:

[0057]

[0058] The following solution is obtained:

[0059] A (0) = QX T (XX T + λ1I) -1

[0060] Similarly, for the classifier parameter W (0) , the following solution can be obtained using the multivariate ridge regression model again:

[0061] W (0) = HX T (XX T + λ2I) -1

[0062] where λ1, λ2 are positive numbers, and I is the identity matrix;

[0063] After initializing the parameters, the cosine similarity is used to change the initialization of the discriminative sparse coding Q value based on the label consistent dictionary learning algorithm, and the dimension reduction matrix is trained simultaneously in the dictionary learning process to better extract the discriminative information in the samples. The specific algorithm is:

[0064]

[0065]

[0066] Keep the initialized dictionary D (0) , the classifier parameter W (0) , and the dimension reduction matrix L unchanged, and let M = LY. Then the above formula changes to:

[0067]

[0068] When updating the dictionary and the sparse coefficient, the above formula remains unchanged except for the dictionary D and the sparse coefficient X, which can be converted to:

[0069]

[0070] Using the initialized parameters, the updated dictionary model parameters D new and X new are obtained, which are:

[0071]

[0072] The above formula is introduced into the algorithm formula to convert it into a solution:

[0073]

[0074] Get the dictionary D, the transformation matrix A and the classifier parameter W meeting the classification requirements from the updated dictionary, specifically:

[0075]

[0076] Where d1 is the first column of the dictionary D, d n is the nth column; a1 is the first column of the transformation matrix A, a n is the nth column of the transformation matrix A; w1 is the first column of the classifier parameter W, w n is the nth column of the classifier parameter W;

[0077] Wherein the classifier parameter output module, the label l corresponding to the maximum element in the classifier parameter matrix is the output recognition label:

[0078]

[0079] Where W is the classifier parameter matrix, X j is the sparse coefficient.

[0080] The specific operation method of the step 4 for training the model for the input training set and adjusting the model parameters through the test set is as follows:

[0081] The pen gesture sequence corresponding to the test set pen gesture is taken as the input to test the air pen gesture recognition model, the trained multi-channel air pen gesture recognition model is used to test the pen gesture recognition accuracy, the training set is iterated multiple times on the air pen gesture recognition model by adjusting the hyperparameters, and the air pen gesture recognition model is optimized on the test set.

[0082] Wherein the specific adjustable parameters include the sampling point number k in the step 2 data preprocessing, the dimension reduction matrix W T in the step 3, the training set and test set ratio a:b, the dictionary atom Z, the iteration number R and the assignment parameters alpha, beta and gamma, and the multiple optimization parameters make the air pen gesture recognition model optimal on the test set.

[0083] The step 5 is specifically to use the optimized recognition model obtained by the step 4 to complete the experimental evaluation of the multi-channel air pen gesture recognition model based on sparse representation, and output the optimal recognition rate of the model for different pen gestures.

[0084] The advantages of the present application are:

[0085] (1) The present application establishes a multi-channel air pen gesture recognition model based on sparse representation, and the pen gesture data of different channels increases the diversity of pen gesture features and improves the robustness of system recognition.

[0086] (2) The multi-channel air gesture recognition model based on sparse representation established by the application uses PCA feature extraction to complete preliminary feature extraction, avoiding the problems of gesture feature data redundancy and long processing time.

[0087] (3) The multi-channel air gesture recognition model based on sparse representation established by the application uses the method of dictionary learning to extract the main features of the gesture and perform classification and recognition, thereby improving the recognition speed on the basis of ensuring the recognition rate. BRIEF DESCRIPTION OF DRAWINGS

[0088] Figure 1 is a flowchart of the application;

[0089] Figure 2 is a schematic diagram of the host computer of the embodiment of the application;

[0090] Figure 3 is a schematic diagram of the internal structure of the electronic pen of the embodiment of the application;

[0091] Figure 4 is a schematic diagram of the multi-channel air gesture collection of the embodiment of the application;

[0092] Figure 5 is a diagram showing the influence of the dictionary scale and sparsity parameters on gesture recognition of the embodiment of the application;

[0093] Figure 6 is a diagram showing the influence of the sample dimension on gesture recognition of the embodiment of the application. DETAILED DESCRIPTION

[0094] Referring to Figure 1 , the method comprises the following steps:

[0095] Step 1. Collecting multi-class gesture data samples by holding the electronic pen to complete the predetermined gesture action, and then obtaining multi-channel data representing multi-class gesture actions;

[0096] Step 2. Data preprocessing is performed on the multi-channel data corresponding to the multi-class gestures;

[0097] Step 3. Constructing a multi-channel air gesture dictionary learning model based on improved label consistency;

[0098] Step 4. Inputting the training set to train the model and adjusting the model parameters through the test set;

[0099] Step 5. Using the optimized recognition model to complete the experimental evaluation of the multi-channel air gesture dictionary learning recognition model based on improved label consistency, and outputting the best recognition rate of the model for different gestures.

[0100] The data collection method in step 1 of the application is as follows:

[0101] The experimenter holds an electronic pen with a nine-axis inertial sensor to perform a prescribed gesture action, presses a start button to start recording gesture data and transmitting to the upper computer, and presses an end button to complete the collection of a single gesture data sample, and repeats the above steps to complete the collection of the entire gesture data set.

[0102] The data preprocessing method in step 2 is specifically as follows:

[0103] Step 2.1: Gesture data sample screening, since the collected gesture data has problems of data point redundancy and data pollution, and the gesture feature information contained by too few sampling points is insufficient, it is necessary to remove duplicate points and eliminate gesture samples with too short length sequences;

[0104] First, all gesture sample sequences are read in a loop, and the gesture sequence without data preprocessing is set as D[m] = d1, d2,.., d m , where m is the sequence length, and the i-th sampling point is d i , then it is judged whether the i-th sampling point d i and the i+1-th sampling point d i+1 are equal, if they are equal, the d i+1 data point is deleted, then the gesture sequence after removing the duplicate points is D[k] = d1, d2,.., d k , where D[k] represents the gesture sequence after removing the duplicate points, and k≤m;

[0105] Finally, it is judged whether the length K of the gesture sequence after removing the duplicate points and the sampling point threshold value are related, if K is greater than the sampling point threshold value, the gesture sequence sample is retained, if it is less than the sampling point threshold value, the gesture sequence sample is deleted;

[0106] Step 2.2: Gesture data resampling, in view of the problems of non-uniform length of gesture sample sequences and information redundancy of too long gesture sample sequences, and the need for subsequent dictionary learning and sparse representation, the gesture is resampled by linear interpolation;

[0107] The fixed length of the resampled gesture sequence is determined, that is, the number of sampling points n, because the number of sampling points of the original gesture sequence is not equal and is less than k according to step 2.1, so the number of sampling points after resampling is n, and the calculation method is as follows:

[0108]

[0109] Where x represents the current position sequence after target resampling, y represents the coordinate value corresponding to the resampled sequence, x1 and x2 are the horizontal coordinates of the two adjacent sampling points of the original gesture data sequence close to the current sampling point x, and y2 and y1 are the coordinate values corresponding to x1 and x2 in the original gesture data sequence.

[0110] Step 2.3: Smoothing the gesture data. To remove noise interference and smooth the tip burrs of the data, a moving average filter is used to smooth the gesture data. The calculation method is as follows:

[0111]

[0112] where H′ r [n] is the gesture sequence after processing in steps 2.1 and 2.1 before filtering, H b [n] is the data sequence after moving mean filtering. The three-dimensional ultrasonic coordinate data and the three-axis acceleration data in the gesture data are mean filtered using the above formula. N is the number of sampling points in the moving mean filter.

[0113] Step 2.4: Ultrasonic feature extraction, after step 2.3, H is obtained b The [n] data sequence includes ultrasonic data, acceleration and quaternion data, where the ultrasonic coordinate sequence is S[n] = S1, S2, ..., S i ,…,S n ,1≤i≤n; where s i =(x i ,y i ,z i ) is the three-axis ultrasonic coordinate of the three-dimensional gesture at any point, and n is the sequence length;

[0114] Calculate the three-dimensional coordinates S of the trajectory center point c =(x c ,y c ,z c ):

[0115]

[0116] Calculate each set of ultrasonic coordinates s in the three-dimensional ultrasonic coordinate sequence S[n] in sequence i =(x i ,y i ,z i ) to the center point S of the trajectory c =(x c ,y c ,z c )’s Euclidean distance:

[0117]

[0118] Get the trajectory center distance encoding sequence d n , d[n]=d1,d2,…,d n ,…,d N ;

[0119] The total length L of the trajectory is calculated, and the sum of the spatial straight line distances between adjacent two points of the stroke sequence is accumulated as the total length of the stroke trajectory:

[0120]

[0121] The total length of the trajectory is used to normalize the trajectory center coding to obtain a normalized trajectory center distance coding sequence C[n]:

[0122]

[0123] Step 2.5: Normalization of stroke data, since the amplitudes of different stroke data measured by the stroke data acquisition device are usually not uniform, in order to normalize the amplitudes of the stroke motion signals, the normalized method is used to process and scale the data processed in the above steps to the interval [-1, 1], and the calculation method is as follows:

[0124]

[0125] Where i represents the i-th sampling point of the stroke sequence, H n [i] is the normalized stroke sequence, H[i] is the stroke sequence before normalization, H max represents the maximum value in the current stroke sequence, min represents the minimum value in the current stroke sequence.

[0126] The step 3 of the present application comprises a stroke feature extraction module, a dictionary learning module, and a classifier parameter output module.

[0127] According to the input of the data preprocessing of the air stroke data set completed in step 2, the input stroke data contains eight dimensions, which are the normalized trajectory center distance coding sequence C[n] after preprocessing of the stroke data, three-axis acceleration data A n =[a x ,a y ,a z ] T , quaternion data Q n =[q1,q2,q3,q4] T ; wherein the collected stroke data needs to be divided into training set and test set after data preprocessing, and the training set and test set are divided according to the ratio of a:b, wherein a>b; a,b∈N * wherein N * is a natural number;

[0128] Wherein the PCA feature extraction is used in the stroke feature extraction module, and this layer completes the feature extraction in the time dimension of the stroke data.

[0129] Suppose there are c kinds of strokes, and each kind of stroke has p in the training sample. After the above preprocessing, each stroke feature matrix has a size of N = n x 8. Each stroke feature matrix is expanded into an N x 1 dimensional column vector. Let M = c x p be the number of training set samples. The total training set scatter matrix is:

[0130]

[0131] where x i represents the i-th stroke sample in a certain class of samples in the c class samples, and u represents the average vector of all stroke features in the training sample, which is calculated as follows:

[0132]

[0133] The training sample matrix X is constructed, and the average value u of the vector sample matrix is calculated.

[0134] The covariance matrix C = E[(x-u)(x-u) T ] is calculated, where E represents the expectation, and the eigenvalues and eigenvectors are calculated. The eigenvalues are arranged in descending order, and the U i corresponding to the first k largest eigenvalues that satisfy the information amount are selected as the transformation matrix W. The training sample is transformed y = W T X to obtain the reduced dimension training sample matrix y, where W T represents the dimension reduction matrix.

[0135] The dictionary learning module uses an improved label consistent dictionary learning algorithm to learn a dictionary with label features. The label consistency constraint of the "discriminative sparse coding error" improved by the cosine similarity and the "optimal" classification performance standard are added to the objective function, and the K-SVD algorithm is used for optimization.

[0136] The label consistent dictionary learning algorithm is generally as follows:

[0137]

[0138] The improved label consistent dictionary learning algorithm model is as follows:

[0139]

[0140] In the formula, the parameters a and b represent the weight of the discriminative sparse coding error, Q represents the discriminative sparse coding of the input sample, D represents the size of the dictionary, X is the sparse coefficient, Y is the total number of training samples, W represents the classifier weight parameter, A represents the linear transformation matrix, and T is the sparse threshold.

[0141] On the basis of the formula, the initialization of discriminative sparse coding Q value is changed by using cosine similarity, and the dimension reduction matrix is trained in the dictionary learning process to better extract the discriminative information in the sample, and the specific algorithm is:

[0142]

[0143] Where L is the dimension reduction matrix, used to reduce the dimension of the sample, Y is the training sample, D is the dictionary, and X is the sparse coefficient, Q represents the discriminative sparse coding, A represents the linear transformation matrix, W is the classifier parameter, and T is the sparse threshold; Parameters a, b, g represent the weight of each item, control the relative contribution coefficient, and are used to adjust the proportion of the three items in the model;

[0144] Set the dimension of the sample to be reduced and initialize the necessary variables, first use the PCA algorithm to get the initialized dimension reduction matrix L, set M = LY, at this time the sample is M, the total number of its categories is C, and then set the size of the dictionary, then use the KSVD algorithm to train the class dictionary of each class, and then get the class dictionary of each class to initialize the dictionary D (0) , then use the l1 or l2 norm method to get the sparse coefficient X of the sample according to the dictionary;

[0145] Use multivariate ridge regression model to get the initialized transformation matrix A (0) , as follows:

[0146]

[0147] The following solution is obtained:

[0148] A (0) = QX T (XX T + λ1I) -1

[0149] Similarly, for the classifier parameter W (0) , the following solution can be obtained by using multivariate ridge regression model again:

[0150] W (0) = HX T (XX T + λ2I) -1

[0151] Where λ1, λ2 are positive numbers, and I is the unit matrix;

[0152] After initializing the parameters, the initialization of discriminative sparse coding Q value is changed by using cosine similarity based on the label consistent dictionary learning algorithm, and the dimension reduction matrix is trained in the dictionary learning process to better extract the discriminative information in the sample, and the specific algorithm is:

[0153]

[0154] Keeping the initialization dictionary D (0) , classifier parameters W (0) , dimension reduction matrix L and other parameters unchanged, let M = LY, then the above formula changes to:

[0155]

[0156] When updating the dictionary and sparse coefficients, the above formula changes except the dictionary D and sparse coefficients X, which can be converted to:

[0157]

[0158] Using the initialization parameters to obtain the updated dictionary model parameters D new and X new , specifically:

[0159]

[0160] The above formula is introduced into the algorithm formula and converted to a solving formula:

[0161]

[0162] From the updated dictionary, the dictionary D that meets the classification requirements, the change matrix A and the classifier parameters W are obtained, specifically:

[0163]

[0164] Where d1 is the first column of the dictionary D, d n is the nth column; a1 is the first column of the transformation matrix A, a n is the nth column of the transformation matrix A; w1 is the first column of the classifier parameter W, w n is the nth column of the classifier parameter W;

[0165] Wherein the classifier parameter output module, the label l corresponding to the maximum element in the classifier parameter matrix is the output recognition label:

[0166]

[0167] Where W is the classifier parameter matrix, X j is the sparse coefficient.

[0168] The specific operation method of the step 4 for training the model for the input training set and adjusting the model parameters through the test set is as follows:

[0169] The air gesture recognition model is tested by taking the gesture sequence corresponding to the test set gesture as input, the multi-channel air gesture recognition model is trained, the gesture recognition accuracy is tested, the hyperparameters are adjusted to make the training set complete multiple iterations on the air gesture recognition model, and the air gesture recognition model is optimized on the test set.

[0170] The specific adjustable parameters include the sampling point number k in the step 2 data preprocessing, the dimension reduction matrix W in the step 3 T , the training set and test set ratio a:b, the dictionary atom Z, the iteration number R and the assignment parameters alpha, beta and gamma, and the multiple optimization parameters make the air gesture recognition model optimal on the test set.

[0171] The step 5 is specifically to use the optimized recognition model obtained by the step 4 to complete experimental evaluation of the multi-channel air gesture recognition model based on sparse representation, and output the optimal recognition rate of the model for different gestures.

[0172] In order to make the purpose, technical scheme and advantages of the present application more obvious and easy to understand, the following will be described in detail by means of the accompanying drawings:

[0173] The hardware devices used by the method include a host and an electronic pen, wherein the host is composed of a main controller and six receivers, and the six receivers are responsible for receiving ultrasonic signals and infrared signals emitted by the electronic pen, so as to calculate three-dimensional ultrasonic positioning coordinates representing gesture actions. The electronic pen integrates a key module, an ultrasonic driving module, an inertial sensor module, a microprocessor module and a data transmission module, and can complete extraction of three-axis acceleration data and quaternions representing gesture actions. The ultrasonic coordinate data, three-axis acceleration data and quaternion multi-channel data representing gesture actions provided by the present application are provided by the device, and the host and the electronic pen are shown in the schematic diagram as shown in Figure 2 、 Figure 3 .

[0174] Figure 4 The schematic diagram of the multi-channel data gesture collection is shown in the figure. The experimenter holds the electronic pen with nine-axis inertial sensors and performs the specified gesture action, presses the start button to start recording the gesture data and transmits it to the upper computer, presses the end button to complete the single gesture data sample collection, and repeats the above steps to complete the collection of the entire gesture data set.

[0175] Figure 5Figure for influence of dictionary scale and sparsity parameter on gesture recognition in the improved label consistent multi-channel gesture dictionary learning method. The collected gesture data set is continuously experimented and parameterized by learning method to seek the best dictionary scale and sparsity parameter to improve the recognition rate of air gesture. In the specific range of the figure, the increase of the dictionary scale can significantly improve the recognition rate. However, in the case of limited training data, too large dictionary scale may lead to overfitting, excessive sensitivity to training data and reduced generalization ability, so the relative optimal dictionary scale is selected as 500.

[0176] Figure 6 Figure for influence of dimension reduction amount on gesture recognition in the improved label consistent multi-channel gesture dictionary learning method. The figure shows that the original data of air gesture is mapped to a low-dimensional space by the PCA linear dimension reduction method, and the selected principal component amount influences the gesture recognition. It can be seen from the figure that if the dimension reduction amount is too large, such as 30 in the figure, important features in the data will be lost, the model cannot effectively represent the key features of the input data, the learning and generalization ability of the model is affected, and the recognition accuracy is reduced. In the range of the illustrated dimension reduction amount greater than 50, the influence of the dimension reduction amount on improving the recognition rate is not large, but if the dimension reduction amount is too small, i.e. too many original dimensions are retained, the redundant information of the data may not be effectively removed. It will increase the computational burden, so the optimal dimension reduction amount is 50.

[0177] To further verify the effectiveness of the learning method of the application in gesture recognition, under the conditions of 500 dictionary atoms, sparsity value of 15, dimension reduction sample value of 50 and iteration parameter setting of 50, and total number of sample training of 3600, the recognition rate of different algorithms for 360 sample gestures is calculated. Table 1 is the recognition accuracy of the improved label consistent multi-channel air gesture dictionary learning method (referred to as improved LCKSVD in the table) for gestures under the same parameters.

[0178] Table 1 Comparison of gesture recognition rates of different algorithms under the same conditions

[0179]

[0180] It can be seen from Table 1 that the improved LCKSVD method proposed in the application has an accuracy of 93.80%, which is higher than that of other methods. Therefore, it can be shown that the method has better classification effect for multi-channel air gesture recognition technology.

[0181] Finally, it should be noted that the above examples are only used to illustrate the technical solutions of the application, but not to limit them.

Claims

1. A multi-channel air gesture dictionary learning method based on improved label consistency, characterized by: The following steps are involved: Step 1. Multiple types of gesture data samples are collected by having multiple users hold the electronic pen and perform predetermined gestures, thereby obtaining multi-channel data representing the multiple types of gestures. Step 2. Preprocess the multi-channel data corresponding to multiple types of gestures as follows: Step 2.1: Screening of gesture data samples. Since the collected gesture data has data point redundancy and data contamination problems, as well as insufficient gesture feature information due to too few sampling points, it is necessary to remove duplicate points and eliminate gesture samples with too short gesture length sequences. First, read all the gesture sample sequences in a loop, and assume that the gesture sequence without data preprocessing is ,in is the sequence length, The sampling points are Next, judge Sampling points Hedi Sampling points Are they equal? ​​If so, delete data points, then the gesture sequence after removing duplicate points is ,in Represents the stroke sequence after deduplication. ; Finally, the relationship between the length k of the gesture sequence after removing duplicate points and the sampling point threshold is determined. If k is greater than the sampling point threshold, the gesture sequence sample is retained; if k is less than the sampling point threshold, the gesture sequence sample is deleted. Step 2.2: Resampling the gesture data. The collected gesture data has issues with inconsistent gesture sample sequence lengths and information redundancy due to excessive lengths. This is also necessary for subsequent dictionary learning and sparse representation. Therefore, the gestures are resampled using linear interpolation. First, determine the fixed length of the stroke sequence after sampling, that is, the number of sampling points n. Since the number of sampling points in the original stroke sequence varies and is less than k according to step 2.1, the number of sampling points after resampling is n, which is calculated as follows: ； in Represents the current position sequence after target resampling, Represents the coordinate value corresponding to the resampled sequence, It is the original gesture data sequence close to the current sampling point The horizontal coordinates of two adjacent sampling points are Is the original gesture data sequence The corresponding coordinate values; Step 2.3: Smoothing the gesture data. To remove noise and other interference and smooth the tip burrs of the data, a moving average filter is used to smooth the gesture data. The calculation method is as follows: ； in is the gesture sequence after processing in steps 2.1 and 2.1 before filtering, is the data sequence after moving mean filtering, and the above formula is used to perform mean filtering on the three-dimensional ultrasonic coordinate data and the three-axis acceleration data in the gesture data, respectively. N is the number of sampling points in the moving mean filter; Step 2.4: Ultrasonic feature extraction, obtained after step 2.3 The data sequence contains ultrasonic data, acceleration and quaternion data, and the ultrasonic coordinate sequence is , ;in is the three-axis ultrasonic coordinate of the three-dimensional gesture at any point, and n is the sequence length; Calculate the three-dimensional coordinates of the trajectory center point : ； Calculate the three-dimensional ultrasonic coordinate sequence of gestures in sequence Each set of ultrasound coordinates To the center of the trajectory The Euclidean distance of : ； Get the trajectory center distance encoding sequence , ; To calculate the total length of the trajectory, L, the sum of the linear distances between two adjacent points in the gesture sequence is taken as the total length of the gesture trajectory: ； The trajectory center code is normalized using the full length of the trajectory to obtain the normalized trajectory center distance code sequence : ； Step 2.5: Normalize the gesture data. Since the amplitude deviations between different gesture data measured by the gesture data acquisition device are usually different, in order to normalize the amplitude of the gesture motion signal, the data processed in the above steps are processed and scaled to the range , which is calculated as follows: ； in Indicates the first sampling points, is the normalized stroke sequence, is the stroke sequence before normalization, Represents the maximum value in the current stroke sequence, Represents the minimum value in the current stroke sequence; Step 3. Build a multi-channel mid-air gesture dictionary learning model based on improved label consistency, as follows: The improved label consistent multi-channel air gesture dictionary learning model includes a gesture feature extraction module, a dictionary learning module, and a classifier parameter output module. The input is an air gesture dataset that has completed data preprocessing in step 2. The input gesture data contains eight dimensions, which are the normalized trajectory center distance encoding sequence after gesture data preprocessing. , three-axis acceleration data , quaternion data ; After data preprocessing, the collected gesture data needs to be divided into training set and test set, according to The training set and test set are divided into two parts according to the ratio of ; in is a natural number; The gesture feature extraction module uses PCA feature extraction to complete feature extraction in the time dimension of gesture data; It is assumed that There are 2 types of strokes in each training sample. After the above preprocessing, the size of each gesture feature matrix is , expand each gesture feature matrix into dimensional column vector, let is the number of training set samples, then the overall distribution matrix of the training set is: ； in represent The first sample in a class of samples A sample of gestures, Represents the average vector of all gesture features in the training sample, which is calculated as follows: ； Construct the training sample matrix X and calculate the average value of the vector sample matrix ; Compute the covariance matrix: , where E represents the expectation, calculate the eigenvalue and eigenvector; arrange the eigenvalues ​​from large to small, and select the front that can meet the information volume The largest eigenvalue corresponds to As a change matrix , change the training samples Get the training sample matrix after dimensionality reduction ,in represents the dimension reduction matrix; The dictionary learning module uses an improved label consistency dictionary learning algorithm to learn a dictionary with label features, adds the label consistency constraint of "discriminative sparse coding error" improved by cosine similarity and the "optimal" classification performance criterion to the objective function, and uses the K-SVD algorithm for optimization; The improved label consistent dictionary learning algorithm model is: ； Parameters in the formula 、 Represents the weight of discriminating sparse coding error, represents the discriminative sparse coding of input samples, Indicates the size of the dictionary and is the sparse coefficient, is the total number of training samples, represents the classifier weight parameter, represents the linear transformation matrix, is the sparse threshold; Based on the above formula, the cosine similarity is used to change the discriminative sparse coding The value is initialized, and the dimension reduction matrix is ​​trained simultaneously during the dictionary learning process to better extract the discriminative information in the sample. The specific algorithm is: ； in Is a dimensionality reduction matrix used to reduce the dimensionality of samples, parameters For training samples, For dictionaries and is the sparse coefficient, stands for Discriminative Sparse Coding, represents the linear transformation matrix, are the classifier parameters, is the sparse threshold; parameter 、 、 Indicates the weight of each item, controls the relative contribution coefficient, and is used to adjust the proportion of the three items in the model; Set the dimension of the sample to be reduced and initialize the necessary variables. First, use the PCA algorithm on the sample to obtain the initialized dimensionality reduction matrix ,set up , the sample is , the total number of categories is , then set the size of the dictionary, and then use the KSVD algorithm to train the class dictionary of each class, and then use them to form the initialization dictionary after obtaining the class dictionary of each class , and then use the dictionary or Norm method to obtain the sparse coefficient of the sample ; Use the multivariate ridge regression model to get the initial transformation matrix ,as follows: ； The following solution is obtained: ； Similarly, for the classifier parameters , again using the multivariate ridge regression model we can obtain the following solution: ； in , is a positive number, is the identity matrix; After initializing the parameters, the discriminative sparse coding is changed using cosine similarity based on the dictionary learning algorithm based on label consistency. The value is initialized, and the dimension reduction matrix is ​​trained simultaneously during the dictionary learning process to better extract the discriminative information in the sample. The specific algorithm is: ； Keep the dictionary initialized , classifier parameters , dimensionality reduction matrix The parameters remain unchanged, let , then the above formula changes to: ； When updating the dictionary and sparse coefficients, the above formula is divided by the dictionary and the sparse coefficient Except for the change, other parameters remain unchanged and can be converted into: ； Use the initialization parameters to obtain the updated dictionary model parameters and , specifically: , ； Import the above formula into the algorithm formula and convert it into a solution formula: ； Get the dictionary that meets the classification requirements from the updated dictionary , change matrix and classifier parameters , specifically: ， , ； in For dictionary The first column of For the List; is the transformation matrix First column, is the transformation matrix No. List; is the classifier parameter The first column of is the classifier parameter No. List; The classifier parameter output module, the label corresponding to the largest element in the classifier parameter matrix This is the output identification label: ； in is the classifier parameter matrix, is the sparse coefficient; Step 4. Input the training set to train the model and adjust the model parameters through the test set; Step 5. Use the optimized recognition model to complete the experimental evaluation of the improved label-consistent multi-channel aerial gesture dictionary learning model, and output the model's optimal recognition rate for different gestures.

2. The multi-channel air gesture dictionary learning method based on improved label consistency according to claim 1, characterized in that: The step 1 comprises: The experimenter holds an electronic pen equipped with a nine-axis inertial sensor and performs prescribed gestures. He presses the start button to start recording gesture data and transmits it to the host computer. He presses the end button to complete the collection of a single gesture data sample. The above steps are repeated to complete the collection of the entire gesture data set.

3. The method for learning a dictionary of mid-air gestures based on improved label consistency multi-channel according to claim 1, characterized in that: In step 4, the specific method of training the model based on the input training set and adjusting the model parameters based on the test set is as follows: The in-flight gesture recognition model is tested using the gesture sequences corresponding to the test set as input. The trained multi-channel in-flight gesture recognition model is used to test the gesture recognition accuracy. The in-flight gesture recognition model is then trained to perform optimally on the test set by adjusting hyperparameters to allow the training set to complete multiple iterations. The specific adjustable parameters include the number of sampling points in the data preprocessing in step 2 , the dimension reduction matrix in step 3 is , the ratio of training set and test set , dictionary atom , the number of iterations R and the assignment parameters 、 and , and optimized the parameters multiple times to make the air gesture recognition model perform best on the test set.

4. The multi-channel air gesture dictionary learning method for improving label consistency according to claim 1, characterized in that: The step 5 specifically uses the optimized recognition model obtained by optimizing the parameters in the step 4 to complete the experimental evaluation based on the improved label consistent multi-channel air gesture dictionary learning model, and outputs the optimal recognition rate of the model for different gestures.

Images