A cross-user identification method based on feature fusion and screening

Abstract

The application discloses a cross-user identification method based on feature fusion and screening, belongs to the technical field of machine learning, and aims at a cross-user identification problem based on physiological and motion signals. By fusing statistical features and deep features, the representation capability of a model is improved. Meanwhile, on the basis of Gini impurity, a cross-user decision tree node division criterion is proposed. In the process of constructing a classification model, features related to an identification task and unrelated to users are selected as much as possible, the trade-off between a classification task and domain generalization is realized, and the generalization performance of the model is improved.

Description

Technical Field

[0001] This invention relates to the field of behavior recognition, and in particular to a cross-user recognition method based on feature fusion and filtering. Background Technology

[0002] With the rapid development of the Internet of Things (IoT) industry and the widespread adoption of wearable devices, physiological and motion signals such as electrocardiograms (ECG), electromyography (EMG), and inertial signals, as direct reflections of human state and behavior, are widely used in human-computer interaction, environmental assistance, and health monitoring, including behavior recognition, gesture recognition, emotion recognition, and anomaly detection. Unlike image or video information, physiological and motion signals are more susceptible to inter-individual differences, limiting the ability of existing models trained according to the principle of empirical risk minimization to generalize to new users and exhibiting significant user dependence in practical applications. Although this problem can be addressed by training models with stronger generalization capabilities using a large amount of user data, data collection and labeling are quite tedious and time-consuming. Therefore, improving the generalization performance of models across user scenarios is a meaningful and challenging task.

[0003] Current research on cross-user recognition based on physiological and motion signals is limited. Existing solutions can be broadly categorized into domain adaptation and domain generalization. Domain adaptation aims to transfer knowledge learned by a neural network model from the source domain to the target domain. During training, labeled data from the source subjects and unlabeled data from the target subjects are used, and feature distributions of the source and target domains are aligned through distance metrics or adversarial methods to obtain a personalized recognition model. However, because physiological and motion signals are more susceptible to individual differences, general domain adaptation methods may not achieve satisfactory results when inter-individual differences are significant. Furthermore, the requirement for unlabeled target user data during offline training can be inconvenient for users in practical applications. Domain generalization, on the other hand, uses strategies to extract domain-independent features from the source domain data as much as possible, improving the model's generalization ability. Some studies utilize user labels from the source domain data to train neural network models adversarially, extracting user-invariant domain features. Other studies use generative adversarial networks (GANs) to generate more training data to obtain better generalization models. However, under the constraint of dataset size, end-to-end trained neural network models are difficult to escape the influence of source domain data distribution. When there are large individual differences, domain generalization methods may not achieve ideal generalization results.

[0004] Fundamentally, the key to building a cross-user recognition model with good generalization ability lies in extracting and filtering features that are only relevant to the recognition task and not to the user. Compared to deep features that rely on training data for automatic extraction, statistical features are designed by experts based on domain knowledge, reflecting the overall trend of the sample and reducing distributional differences between different users to some extent, thus providing better generalization potential. However, statistical features require manual design, have limited information representation capabilities, and may struggle to cope with the challenges of large datasets and complex recognition tasks. Summary of the Invention

[0005] To address the shortcomings of the existing technologies, this invention proposes a cross-user identification method based on feature fusion and filtering, which can improve the generalization performance of the model in cross-user scenarios.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] A cross-user identification method based on feature fusion and filtering includes the following steps:

[0008] Step 1: In the offline model building phase, acquire physiological and motion data X of the source domain user based on wearable sensor devices. s And label the corresponding category Y s and user tag G s ;

[0009] Step 2: For source domain user data X s Category tag Y s User tag G s Preprocessing and sliding window segmentation are performed to construct the training sample set.

[0010] Step 3: Construct a neural network model based on one-dimensional convolution and adversarial learning, using the training sample set. Training is performed on the above to obtain the deep feature extractor E. θ ;

[0011] Step 4: Extract source domain data X separately s Statistical eigenvector F h and depth feature vector F d And perform feature fusion to obtain the fused feature vector F. s ;

[0012] Step 5: Based on the fused feature vector F s Category tag Y s and user tag G s Training a cross-user random forest classifier C RF ;

[0013] Step 6: In the online application phase of the model, acquire the physiological and motion data X of the target user based on wearable sensor devices. t The target user's fused feature vector F is obtained through steps 2 and 4. t And use the cross-user random forest classifier C obtained in step 5 RF Perform identification.

[0014] Furthermore, in step 3, a neural network model based on one-dimensional convolution and adversarial learning is constructed on the training sample set. The deep feature extractor E is obtained by training on it. θ The steps are as follows:

[0015] Step 3-1: Construct a neural network model based on one-dimensional convolution;

[0016] The neural network model includes a deep feature extractor E. θ Task tag classifier And User Distinguisher D ω The depth feature extractor E θ The network includes e-layer sub-networks, each of which comprises: a one-dimensional convolutional layer, a pooling layer, an activation function layer, and a batch normalization layer; the task label classifier. And User Distinguisher D ω Each layer consists of a random deactivation layer, a fully connected layer, an activation function layer, and an output layer. The deep feature extractor E... θ Extracting depth features F from the input samples d After being flattened, it was used as a task label classifier. And User Distinguisher D ω Input;

[0017] Step 3-2: Train the deep feature extractor E using an adversarial learning strategy. θ .

[0018] Preferably, in step 3-2, the training process includes the following steps:

[0019] Step 3-2-1: Standardize the training sample set data for each sensor channel using the following formula:

[0020]

[0021] Where vectors μ and σ represent the mean and standard deviation of the sensor channels in the training sample set data.

[0022] Step 3-2-2: In the training sample set In the process, all samples from two users are randomly selected to form a training subset U1∪U2;

[0023] Step 3-2-3: Input the samples in U1∪U2 into the neural network model in batches, and calculate the task label classifier for each batch. Losses User Authentication Device D ω Loss L d (θ,ω; x,g) and deep feature extractor E θ Loss L d (θ;x,y,g), and the network parameters are updated through backpropagation, and the calculation formula is as follows:

[0024]

[0025]

[0026]

[0027] Where θ, ω, E, respectively, is a deep feature extractor. θ Task tag classifier User Authentication Device D ω The network parameters are given, where μ is the weight coefficient, and the deep feature extractor E is trained using an adversarial learning strategy. θ Extract features that are irrelevant to the user as much as possible.

[0028] Step 3-2-4: Iterate through the above training process multiple times until the model converges.

[0029] Furthermore, in step 4, the source domain data X is extracted. s Statistical characteristics F h and depth features F d And perform feature fusion, the steps are as follows:

[0030] Step 4-1: Prepare the source domain training sample set Input the deep feature extractor E trained in step 3 θ Calculate the deep feature vector F d ;

[0031] Step 4-2: Calculate the source domain training sample set The time-frequency domain statistical characteristics, including but not limited to: maximum value, minimum value, arithmetic mean, harmonic mean, standard deviation, correlation coefficient, peak-to-peak amplitude, average signal power, skewness, and divergence, are used to obtain the statistical feature vector F. h ;

[0032] Step 4-3: Concatenate the deep features and statistical features of each sample to obtain the fused feature vector F. s Construct a feature-based training sample set.

[0033] Furthermore, in step 5, based on the fused feature vector F s Category tag Y s and user tag G s Training a cross-user random forest classifier C RF The steps are as follows:

[0034] Step 5-1: Input source domain feature vector F s Category tag Y s and user tag G s Training the decision tree. Traversing the feature vector F during decision tree node partitioning. s And select the optimal feature F according to the given partitioning criteria. best and the corresponding threshold T best The current samples are divided into left or right child nodes, and the division criterion formula is as follows:

[0035]

[0036] in, These represent the current node and the child nodes after partitioning, respectively, within the user tag G. s The impurity of the ginni These represent the current node and its child nodes after partitioning, respectively, in the category label G. y Gini impurity reflects the overall disorder of sample labels within a node; for cross-user models, the classification label G should be minimized as much as possible when splitting nodes. y The impurity of the Gini tag on the screen, while maintaining or increasing the user tag G s The Gini impurity is calculated by adjusting the α parameter to achieve a trade-off between classification tasks and domain generalization. The formulas are as follows:

[0037]

[0038]

[0039]

[0040]

[0041] Where n, n l n r represents the number of samples for the current node, left child node, and right child node, respectively, and m and k represent the number of users and the number of categories in the original domain, respectively. These represent the proportions of the sample belonging to user i in the current node, left child node, and right child node, respectively. These represent the proportions of samples belonging to category j in the current node, left child node, and right child node, respectively.

[0042] Step 5-2: Each time, randomly select n training samples with replacement from the original sample set, and perform k rounds to obtain k training sets. Train k decision trees according to the partitioning criteria described in Step 5-1. Finally, integrate all decision trees to construct a cross-user random forest model C. RF .

[0043] This invention proposes a cross-user recognition framework based on feature fusion and selection. For cross-user recognition problems based on physiological and motion signals, it integrates statistical and deep features to improve the model's representation ability. At the same time, based on Gini impurity, it proposes a cross-user decision tree node partitioning criterion. In the process of constructing the classification model, it selects features that are relevant to the recognition task but irrelevant to the user as much as possible, achieving a trade-off between classification task and domain generalization, thereby improving the model's generalization performance.

[0044] The beneficial effects of this invention are mainly manifested in the following aspects:

[0045] 1. Extract as many user-irrelevant deep features as possible through adversarial learning.

[0046] 2. By combining the advantages of deep features and statistical features, the model's representational ability is enhanced while ensuring generalization, thereby improving recognition performance.

[0047] 3. When splitting decision tree nodes, the proposed splitting criterion can achieve a trade-off between label classification and domain generalization, filter out user-irrelevant features from existing features, and improve the model's cross-user generalization performance. Attached Figure Description

[0048] Figure 1 This is a diagram illustrating the overall architecture of the cross-user identification framework based on feature fusion and filtering according to the present invention.

[0049] Figure 2 This is a diagram of the adversarial learning network structure used in this invention for extracting deep features.

[0050] Figure 3 This is a comparison chart showing the classification accuracy of the proposed classification criteria and the baseline method for a single user.

[0051] Figure 4 This is a comparison chart of the average classification accuracy of the method proposed in this invention under different parameters. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the following describes the invention in further detail with reference to the accompanying drawings, taking cross-user behavior recognition based on motion sensors as an example.

[0053] Reference Figures 1-4A cross-user identification method based on feature fusion and filtering, the overall architecture of which is as follows: Figure 1 As shown, it includes the following steps:

[0054] Step 1: In the offline model building phase, acquire physiological and motion data X of the source domain user based on wearable sensor devices. s And label the corresponding category Y s and user tag G s ,

[0055] Step 2: For source domain user data X s Category tag Y s User tag G s Preprocessing and sliding window segmentation are performed to construct the training sample set.

[0056] During preprocessing, median filtering and low-pass filtering are applied to the signal to remove outliers and high-frequency noise. Then, a sliding window with a length of 1 second is used to segment the signal, with adjacent windows overlapping by 50%. The choice of window length is related to the specific recognition task. It is necessary to choose an appropriate window length so that the sample segments contain sufficient action information and the number of sample points in a single segment is not too large, so as to ensure the real-time performance of the online recognition process.

[0057] Step 3: Construct a neural network model based on one-dimensional convolution and adversarial learning, using the training sample set. Training is performed on the above to obtain the deep feature extractor E. θ ;

[0058] The neural network model includes a deep feature extractor E. θ Task tag classifier And User Distinguisher D ω The depth feature extractor E θ The task label classifier comprises three sub-networks, each including: a one-dimensional convolutional layer, a pooling layer, an activation function layer, and a batch normalization layer; And User Distinguisher D ω Each layer consists of a random deactivation layer, a fully connected layer, an activation function layer, and an output layer. The deep feature extractor E... θ Extracting depth features F from the input samples d After being flattened, it was used as a task label classifier. And User Distinguisher D ω The input, its structure is as follows Figure 2 As shown.

[0059] The deep feature extractor E is trained using an adversarial learning strategy. θ The process is as follows:

[0060] First, the training sample set data is standardized for each sensor channel using the following formula:

[0061]

[0062] Where vectors μ and σ represent the mean and standard deviation of each sensor channel in the training sample set data;

[0063] Then, in the training sample set In the process, all samples from two users are randomly selected to form a training subset U1∪U2. Then, the samples in U1∪U2 are input in batches into a training subset such as... Figure 2 In the neural network model shown, the task label classifier is calculated respectively. Losses User Authentication Device D ω Loss L d (θ,ω; x,g) and deep feature extractor E θ Loss L d (θ;x,y,g), and the network parameters are updated through backpropagation, and the calculation formula is as follows:

[0064]

[0065]

[0066]

[0067] Where θ, ω, E, respectively, is a deep feature extractor. θ Task tag classifier User Authentication Device D ω The network parameters are given by μ, where μ is the weight coefficient. A deep feature extractor E is trained using an adversarial learning strategy. θ Extract features that are irrelevant to the user as much as possible;

[0068] Finally, the above training process is repeated multiple times until the model converges, yielding the deep feature extractor E. θ ,

[0069] Step 4: Extract source domain data X separately s Statistical eigenvector F h and depth feature vector F d And perform feature fusion to obtain the fused feature vector F. s The steps are as follows:

[0070] Step 4-1, source domain training sample set After standardization, input the deep feature extractor E obtained in step 3. θCalculate the deep feature vector F d ;

[0071] Step 4-2, calculate the source domain training sample set The time-frequency domain statistical characteristics are as follows: First, the source domain data is augmented using Discrete Fourier Transform. Then, the amplitude values ​​of each sensor data point (including the x, y, and z axes) are calculated and augmented on the augmented time-frequency domain data. Finally, statistical characteristics, including maximum, minimum, arithmetic mean, harmonic mean, standard deviation, correlation coefficient, peak-to-peak amplitude, average signal power, skewness, and divergence, are calculated on the augmented time-frequency domain data to obtain the statistical feature vector F. h ;

[0072] Step 4-3: Concatenate the deep features and statistical features of each sample to obtain the fused feature vector F. s Construct a feature-based training sample set.

[0073] Step 5: Based on the fused feature vector F s Category tag Y s and user tag G s Training a cross-user random forest classifier C RF The steps are as follows:

[0074] Step 5-1: Input source domain feature vector F s Category tag Y s and user tag G s Training the decision tree. Traversing the feature vector F during decision tree node partitioning. s And select the optimal feature F according to the given partitioning criteria. best and the corresponding threshold T best The current samples are divided into left or right child nodes, and the division criterion formula is as follows:

[0075]

[0076] in, These represent the current node and the child nodes after partitioning, respectively, within the user tag G. s The impurity of the ginni These represent the current node and its child nodes after partitioning, respectively, in the category label G. y Gini impurity reflects the overall disorder of sample labels within a node; for cross-user models, the classification label G should be minimized as much as possible when splitting nodes. y The impurity of the Gini tag on the screen, while maintaining or increasing the user tag G s The Gini impurity is calculated by adjusting the α parameter to achieve a trade-off between classification tasks and domain generalization. The formulas are as follows:

[0077]

[0078]

[0079]

[0080]

[0081] Where n, n l n r represents the number of samples for the current node, left child node, and right child node, respectively, and m and k represent the number of users and the number of categories in the original domain, respectively. These represent the proportions of the sample belonging to user i in the current node, left child node, and right child node, respectively. These represent the proportions of samples belonging to category j in the current node, left child node, and right child node, respectively.

[0082] Step 5-2: Each time, randomly select n training samples with replacement from the original sample set, and perform k rounds to obtain k training sets. Train k decision trees according to the partitioning criteria described in Step 5-1. Finally, integrate all decision trees to construct a cross-user random forest model C. RF ;

[0083] Step 6: In the online application phase of the model, acquire the physiological and motion data X of the target user based on wearable sensor devices. t The target user's fused feature vector F is obtained through steps 2 and 4. t And use the cross-user random forest classifier C obtained in step 5 RF Perform identification.

[0084] Compared to other methods, the cross-user recognition method based on feature fusion and filtering proposed in this embodiment integrates statistical and deep features, improving the model's representational ability. Furthermore, based on Gini impurity, a novel decision tree node partitioning criterion is proposed to select features relevant to the recognition task but irrelevant to the user during the classification model construction process, further enhancing the model's generalization performance. Moreover, the method proposed in this patent can be combined with other deep feature extraction networks to further improve cross-user recognition capabilities.

[0085] This embodiment selects the Daily and Sports Activities Data Set, a representative dataset in the field of human behavior recognition, to verify its effectiveness. The Daily and Sports Activities Data Set is specifically designed for daily and sports activities, recording sensor data for 19 different activities, covering multiple similar activities, such as walking on a treadmill at different incline angles, cycling in different postures, standing in an elevator, and standing outdoors. Five inertial measurement units were placed on the subject's chest, left and right wrists, and left and right legs, respectively, with a data sampling frequency of 25Hz. Each activity was independently performed for 5 minutes by 8 different subjects (4 women and 4 men, aged 20-30 years). During data collection, subjects were required to perform the activity according to their own habits and methods, without restrictions on the activity type, maximizing the reproduction of differences between subjects in a real-world environment. This embodiment uses the Leave-One-Out (LOSO) method to test the generalization performance of the model, i.e., taking all samples from subject i from N subjects as the test set, and the remaining N-1 subjects as the training set, iterating N times, and calculating the average recognition accuracy. Figure 3 This paper shows a comparison of the classification accuracy of the proposed classification criteria and the baseline method for a single user. Figure 4 The impact of different α values ​​and the number of decision trees on the average classification accuracy is shown. It can be seen that compared with the Gini impurity baseline method of random forest (α=0), the improved algorithm achieves a stable improvement. Table 1 presents a comparison of the average generalization performance of the random forest method, the random forest method with deep features, the improved cross-user random forest method, and the cross-user random forest method with deep features.

[0086] Classification methods Accuracy (%) F1 score (%) Random Forest 90.64 89.66 Random forests that incorporate deep features 92.87 92.01 Cross-user Random Forest 93.24 92.87 Cross-user random forest with deep feature integration 93.78 93.32

[0087] Table 1

[0088] The above embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Although the invention has been described in detail with reference to the embodiments, those skilled in the art should understand that various combinations, modifications, or equivalent substitutions of the technical solutions of the invention do not depart from the spirit and scope of the invention and should be covered within the scope of the claims of the invention.

Claims

1. A cross-user identification method based on feature fusion and screening, characterized in that, The method comprises the following steps: Step 1: In the model offline construction phase, physiological and motion data X of source domain users are obtained based on wearable sensor devices s , and the corresponding classification labels Y s and user labels G s are labeled; Step 2: Preprocess and sliding window segmentation on source domain user data X s , classification label Y s , user label G s Construct training sample set Step 3: Constructing a neural network model based on one-dimensional convolution and adversarial learning, training the sample set on the above, obtaining a deep feature extractor E θ , the steps are as follows: Step 3-1: constructing a neural network model based on one-dimensional convolution; The neural network model includes a deep feature extractor E. θ Task tag classifier And User Distinguisher D ω The depth feature extractor E θ The network includes e-layer sub-networks, each of which comprises: a one-dimensional convolutional layer, a pooling layer, an activation function layer, and a batch normalization layer; the task label classifier. And User Distinguisher D ω Each layer consists of a random deactivation layer, a fully connected layer, an activation function layer, and an output layer. The deep feature extractor E... θ Extracting depth features F from the input samples d After being flattened, it was used as a task label classifier. And User Distinguisher D ω Input; Step 3-2: Train the deep feature extractor E with a contrastive learning strategy θ The training process comprises the following steps: Step 3-2-1: standardizing the training sample set data according to each sensor channel, using the formula as follows: Wherein the vector μ and σ are the mean and standard deviation of the sensor channel of the training sample set data; Step 3-2-2: In the training sample set U1∪U2, randomly select all samples of two users to form a training subset Step 3-2-3: input the samples in U1∪U2 into the neural network model in batches, respectively calculate the loss on the task label classifier the loss L ω on the user discriminator D d (θ,ω;x,g) and the loss L θ on the deep feature extractor E d (θ;x,y,g), and update the network parameters by back propagation, whose calculation formula is as follows:​ wherein θ, ω, are respectively the network parameters of the deep feature extractor E θ , the task label classifier and the user discriminator D ω , μ is a weight coefficient, the deep feature extractor E θ is trained by the strategy of adversarial learning to extract the features irrelevant to the user; Step 3-2-4: performing the above training process multiple times until the model converges; Step 4: Extract the source domain data X s the statistical feature vector F h and the deep feature vector F d , and perform feature fusion to obtain the fusion feature vector F s ; Step 5: Based on the fused feature vector F s , the classification label Y s and the user label G s , train a cross-user-random forest classifier C RF ; Step 6: In the online application phase of the model, acquire the physiological and motion data X of the target user based on wearable sensor devices. t The target user's fused feature vector F is obtained through steps 2 and 4. t And use the cross-user random forest classifier C obtained in step 5 RF Perform identification.

2. The cross-user identification method based on feature fusion and screening according to claim 1, characterized in that, In the step 4, the source domain data X is extracted s statistical features F h and deep features F d are extracted, and feature fusion is performed, as follows: Step 4-1: obtaining a source domain training sample set inputting the deep feature extractor E trained in step 3 θ , calculating the deep feature vector F d ; Step 4-2: Compute the source domain training sample set The time-frequency domain statistical features include maximum value, minimum value, arithmetic mean, harmonic mean, standard deviation, correlation coefficient, peak-to-peak amplitude, signal average power, skewness and kurtosis, and a statistical feature vector F is obtained h ; Step 4-3: Concatenate the depth features and the statistical features of each sample to get the fusion feature vector F s , constructing a feature-based training sample set 3. The cross-user identification method based on feature fusion and screening according to claim 1, characterized in that, In the step 5, based on the fusion feature vector F s , the classification label Y s and the user label G s , a cross-user-random forest classifier C RF is trained, with the following steps: Step 5-1: input source domain feature vector F s , classification label Y s and user label G s Train decision tree, traverse feature vector F s when decision tree node is divided, and select optimal feature F best and corresponding threshold T best according to given division criterion, divide current all samples to left child node or right child node, and the division criterion formula is as follows: in, These represent the current node and the child nodes after partitioning, respectively, within the user tag G. s The impurity of the ginni These represent the current node and its child nodes after partitioning, respectively, in the category label G. y Gini impurity reflects the overall disorder of sample labels within a node; for cross-user models, the classification label G should be minimized as much as possible when splitting nodes. y The impurity of the Gini tag on the screen, while maintaining or increasing the user tag G s The Gini impurity is calculated by adjusting the α parameter to achieve a trade-off between classification tasks and domain generalization. The formulas are as follows: where n, n l , n r are the number of samples in the current node, left child node and right child node, respectively, and m, k are the number of users and the number of classes, respectively, are the proportions of the samples belonging to user i in the current node, left child node and right child node, respectively, are the proportions of the samples belonging to class j in the current node, left child node and right child node, respectively. Step 5-2: Randomly sample n training samples from the original sample set with replacement each time, and perform k rounds of sampling to obtain k training sets. Train k decision trees according to the division criteria described in step 5-1, and finally integrate all the decision trees to build a cross-user-random forest model C RF .

Images