Gesture recognition model training method, gesture recognition method and device
By constructing a feature extraction network framework and a federated learning training model, the problems of low accuracy and high data collection cost of passive gesture recognition in cross-domain recognition are solved, and cross-domain gesture recognition with high precision and strong generalization capability is achieved.
Patent Information
- Application Number
- CN202210956575.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-08-10
- Publication Date
- 2025-09-26
- Estimated Expiration
- 2042-08-10
AI Technical Summary
Existing passive gesture recognition technology has low accuracy and insufficient generalization ability in cross-domain recognition, and high data collection costs, making it particularly unsuitable for large-scale applications.
By adopting the federated learning framework, a feature extraction network framework is constructed to train the global model, local model and new domain model, and the model parameters are updated with a small amount of data to achieve cross-domain high-precision gesture recognition.
It achieves cross-domain high-precision gesture recognition, reduces data collection costs, and provides a general model with strong generalization capabilities and personalized local models.
Smart Images
Figure CN115439883B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of passive perception, and specifically to a gesture recognition model training method, a gesture recognition method, and a gesture recognition device. Background Art
[0002] In recent years, with the rapid growth of wireless devices such as laptops, mobile phones, and smart speakers in our daily lives, we are surrounded by radio frequency (RF) signals emitted by these devices. As people move around in these environments, RF signals are interfered with, implicitly capturing information about people. To this end, researchers have been extensively exploring the various possibilities of RF sensing in recent years, including person identification, gesture recognition, and fall detection. As a new type of human-computer interaction, gesture recognition enables user-machine interaction by simply recognizing a few movements, making operation more flexible and convenient, and providing users with a better interactive experience.
[0003] Existing gesture recognition methods can be broadly categorized into two types: active and passive. Active human gesture recognition methods primarily rely on users wearing various sensor devices. For example, they utilize the built-in inertial sensors (including accelerometers and gyroscopes) on mobile devices to identify various activities. Smartwatches and wristbands are also used for activity recognition. While this method achieves fine-grained activity recognition across multiple domains, it requires users to carry additional sensor devices for extended periods, making it less user-friendly and invasive, making it unsuitable for widespread use. Passive recognition methods, which do not require additional sensor devices, are more flexible than active methods and primarily include methods based on visual images and wireless signals. Among them, gesture recognition systems based on visual images use cameras as sensors, collecting user activity data and applying image processing techniques to perform gesture recognition. However, fundamental challenges of gesture recognition systems based on visual images are poor lighting and occlusion. Furthermore, due to the need for cameras, these systems also pose security risks, such as user privacy leaks. Gesture recognition using wireless signals is not affected by lighting and does not present privacy issues. Among them, gesture recognition using Wi-Fi offers advantages such as low cost and ease of deployment. However, Wi-Fi-based gesture recognition methods still have many limitations. First, Wi-Fi signals often carry domain information unrelated to human activities and gestures. Classifiers trained using raw signals in one domain often experience a sharp drop in accuracy when used in another domain. Second, achieving a robust and highly accurate Wi-Fi gesture recognition system requires a labor-intensive and time-consuming process to centrally collect large amounts of data. While collecting this data from every household might be feasible, requiring every employee or visitor in a large organization to collect data is impractical. Consequently, existing passive gesture recognition technologies suffer from shortcomings in cost, robustness, and generalization capabilities. Summary of the Invention
[0004] In view of this, this application proposes a gesture recognition model training method, which can obtain a generalized cross-domain universal gesture recognition model and multiple personalized local models, thereby improving the cross-domain high-precision target gesture recognition rate and reducing the cost of centralized data collection.
[0005] Below is a brief overview of the present application to provide a basic understanding of certain aspects of the present application. It should be understood that this overview is not an exhaustive overview of the present application. It is not intended to identify key or important portions of the present application, nor is it intended to limit the scope of the present application. Its purpose is simply to present certain concepts in a simplified form as a prelude to the more detailed description that will be discussed later.
[0006] According to a first aspect of the present application, a gesture recognition model training method is provided, comprising:
[0007] Build a feature extraction network framework;
[0008] constructing a global model, at least one local model, and at least one new domain model based on a feature extraction network framework;
[0009] Train the local model based on the parameters of the global model and the local gesture training data to update the parameters of the local model;
[0010] Train the new domain model based on the parameters of the global model and the personalized gesture training data to update the parameters of the new domain model;
[0011] updating the parameters of the global model based on the updated parameters of the local model and the updated parameters of the new domain model;
[0012] The process of training local models, training new domain models, and updating parameters of the global model is repeated. When the global model converges, an updated global model, at least one trained local model, and at least one trained new domain model are obtained.
[0013] In one embodiment, the feature extraction network includes a convolutional neural network, a recurrent neural network, and a fully connected neural network connected in sequence.
[0014] In one embodiment, training a local model based on the parameters of the global model and the local gesture training data to update the parameters of the local model includes:
[0015] Download the parameters of the global model and update the parameters of the local model;
[0016] The local model is trained based on the local gesture training data, and the parameters of the local model are updated.
[0017] In one embodiment, training a new domain model based on the parameters of the global model and the personalized gesture training data to update the parameters of the new domain model includes:
[0018] Download the parameters of the global model and update the parameters of the new domain model;
[0019] Train the new domain model based on the personalized gesture training data and update the parameters of the new domain model.
[0020] In one embodiment, the method further includes initializing a global model, comprising:
[0021] The global model is trained based on the initialized training data to obtain the initial parameters of the global model.
[0022] According to a second aspect of the present application, a gesture recognition method is provided, comprising:
[0023] Apply the above method to obtain the updated global model and the trained new domain model;
[0024] Download the updated global model parameters;
[0025] Update the parameters of the trained new domain model according to the parameters of the updated global model;
[0026] Retrain the trained new domain model based on the personalized gesture training data to obtain a second-trained new domain model;
[0027] The gesture data to be recognized is input into the new domain model after secondary training to obtain the gesture recognition result.
[0028] According to a third aspect of the present application, a gesture recognition method is provided, comprising:
[0029] Apply the above method to obtain the updated global model;
[0030] Download the updated global model parameters;
[0031] Update and reconstruct the parameters of the new domain model according to the parameters of the updated global model;
[0032] The reconstructed new domain model is trained based on the personalized gesture training data corresponding to the reconstructed new domain model to obtain the trained reconstructed new domain model;
[0033] The gesture data to be recognized is input into the trained reconstructed new domain model to obtain the gesture recognition result.
[0034] According to a fourth aspect of the present application, a gesture recognition model training device is provided, comprising:
[0035] A network framework building module is configured to build a feature extraction network framework;
[0036] A model building module is configured to build a global model, at least one local model and at least one new domain model based on a feature extraction network framework;
[0037] The model training module is configured to construct a global model, at least one local model, and at least one new domain model based on a feature extraction network framework; train the local model based on the parameters of the global model and local gesture training data to update the parameters of the local model; train the new domain model based on the parameters of the global model and personalized gesture training data to update the parameters of the new domain model; update the parameters of the global model based on the parameters of the updated local model and the parameters of the updated new domain model; repeat the process of training the local model, training the new domain model, and updating the parameters of the global model, and when the global model tends to converge, obtain an updated global model, at least one trained local model, and at least one trained new domain model.
[0038] In one embodiment, the model training module is further configured to:
[0039] Download the parameters of the global model and update the parameters of the local model; train the local model based on the local gesture training data and update the parameters of the local model.
[0040] In one embodiment, the model training module is further configured to:
[0041] Download the parameters of the global model and update the parameters of the new domain model; train the new domain model based on the personalized gesture training data and update the parameters of the new domain model.
[0042] The technical solution of the present application has at least one of the following technical effects: The gesture recognition model training method of the embodiment of the present application uses a small amount of preprocessed Wi-Fi channel state information data to initialize the global model parameters. Each local model then downloads the model parameters from the global model and applies the parameters to itself. The model is then fine-tuned using its own data set before the model parameters are transferred to the global model. This cycle continues to obtain a highly generalized global model and multiple personalized local models. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] The present application may be better understood by referring to the following description in conjunction with the accompanying drawings, which together with the following detailed description are incorporated into and form a part of this specification. In the drawings:
[0044] Figure 1 A schematic diagram of a flow chart of a gesture recognition method according to an embodiment of the present application is shown;
[0045] Figure 2 A flowchart of a gesture recognition model training method according to an embodiment of the present application is shown;
[0046] Figure 3 The following is a structural diagram of a feature extraction network framework according to an embodiment of the present application;
[0047] Figure 4 A schematic diagram showing the relationship between parameter downloading and parameter uploading among the global model, local model, and new domain model according to an embodiment of the present application is shown;
[0048] Figure 5 A structural block diagram of a gesture recognition model training device according to an embodiment of the present application is shown;
[0049] Figure 6 A comparison chart of the recognition performance results with different numbers of training users is shown;
[0050] Figure 7 A comparison chart of the recognition performance results for different environments and different users is shown;
[0051] Figure 8 A comparison of the performance of different classification gesture numbers is shown;
[0052] Figure 9 The figure shows the comparative evaluation results of different models. DETAILED DESCRIPTION
[0053] Exemplary embodiments of the present application are described below with reference to the accompanying drawings. For the sake of clarity and conciseness, not all features of actual embodiments are described in this specification. However, it should be understood that in the process of developing any such actual embodiment, many implementation-specific decisions may be made to achieve the developer's specific goals, and these decisions may vary from one implementation to another.
[0054] It is also necessary to explain here that, in order to avoid obscuring the present application due to unnecessary details, the accompanying drawings only show the device structure closely related to the solution according to the present application, while other details that are not closely related to the present application are omitted.
[0055] It should be understood that the present application is not limited to the described embodiments due to the following description with reference to the accompanying drawings. In this document, where feasible, the embodiments may be combined with each other, features between different embodiments may be replaced or borrowed, and one or more features may be omitted in one embodiment.
[0056] The embodiments of the present application are based on a federated learning framework and perform collaborative machine learning without exchanging original Wi-Fi signals, achieving low-cost and high-precision cross-domain Wi-Fi gesture recognition. This results in a global model with high generalization capability. Furthermore, in new domain recognition, a high-precision personalized model for the new domain can be obtained using only one or two samples.
[0057] Figure 1A flowchart 100 of a gesture recognition method according to an embodiment of the present application is shown. The gesture recognition method according to an embodiment of the present application includes: Step 110, collecting a public Wi-Fi-based gesture dataset. Here, the public Wi-Fi-based gesture dataset can be the Widar 3.0 dataset or the WiAR dataset. The public Wi-Fi-based gesture dataset includes Wi-Fi channel state information data, which can be used to characterize user gesture information. The Widar 3.0 dataset includes thousands of CSI (channel state information) measurements for 22 daily activities, i.e., Wi-Fi channel state information data, involving 17 users, 3 environments, and 6 devices. The 22 activities fall into two categories: the first category is common hand activities such as pushing and pulling, clapping, drawing an O, and drawing a zigzag; the second category is complex and semantic activities (i.e., drawing the numbers 0-9). Wi-Fi data packets are collected at a rate of 1000 packets per second using an off-the-shelf small desktop equipped with an Intel 5300 wireless network card. The WiAR dataset includes CSI measurements of 16 activities performed by 10 volunteers in a conference room measuring 6 by 10 meters and furnished with a small number of desks and chairs. The 16 activities fall into three categories: upper-body activities, lower-body activities, and whole-body activities. Upper-body activities include horizontal arm waving, hand waving, paper throwing, ticking, talking on the phone, making an X, clapping, high-arm waving, drinking water, and high throwing; lower-body activities include front kicking and side kicking; and whole-body activities include squatting, sitting, bending over, and walking. Data is collected using a 20-MHz bandwidth and 30 subcarriers at 5 MHz.
[0058] Step 120 preprocesses the Wi-Fi channel state information data in the public gesture dataset to eliminate the effects of environmental noise and outliers, while also achieving data uniformity in the temporal dimension. Specifically, the Wi-Fi channel state information data in the public gesture dataset contains special outliers. For example, a Hampel filter can be used to remove these outliers. Since gestures are mostly concentrated in low-frequency regions, while environmental noise is primarily concentrated in high-frequency regions, a low-pass filter is used to remove the effects of high-frequency environmental noise. Furthermore, to account for the varying speeds of gestures for different users or the same user, interpolation is used to process the Wi-Fi channel state information data, achieving temporal uniformity.
[0059] Step 130: Train a gesture recognition model based on the preprocessed Wi-Fi channel status information data to obtain a global model, at least one local model, and at least one new domain model. The Wi-Fi channel status information data corresponding to each environment or user in the Widar 3.0 dataset can serve as the local dataset corresponding to that environment or user. For a particular environment or user, if Wi-Fi channel status information data has never been obtained before, the Wi-Fi channel status information data corresponding to that environment or user can serve as new domain data. Otherwise, the Wi-Fi channel status information data corresponding to that environment or user serves as local gesture training data. All local gesture training data forms training data, and a portion of the training data is randomly selected to form initialization training data. The local gesture training data is used to train multiple local models, and the initialization training data is used to initialize the global model. A small amount of sample data (generally no more than two samples) is randomly selected from the new domain data to form personalized gesture training data, which serves as data for training the new domain model.
[0060] In step 140, based on the global model parameters, the gesture data to be recognized is input into the new domain model to obtain a recognition result. Here, the gesture data to be recognized can be data in the new domain data. The new domain model can be a trained new domain model or a newly constructed new domain model.
[0061] Figure 2 A schematic flow chart of a gesture recognition model training method 200 according to an embodiment of the present application is shown. Method 200 begins at step 210, where a feature extraction network framework is constructed. This feature extraction network framework can be used to extract time-domain and frequency-domain features of Wi-Fi channel state information data and may include a sequentially connected convolutional neural network, a recurrent neural network, and a fully connected neural network. Figure 3 A block diagram 300 of a feature extraction network framework according to an embodiment of the present application is shown. A convolutional neural network is used to learn the frequency-domain internal relationships between subcarriers of Wi-Fi channel state information data, a recurrent neural network is used to extract time-domain internal features within the subcarriers of Wi-Fi channel state information data, and a fully connected neural network uses logistic regression to classify gestures.
[0062] A convolutional neural network can be composed of two convolutional layer modules (CNNs). Each convolutional layer module includes a convolutional layer, a batch normalization layer, and a dropout layer. The 2D filter in the convolutional layer is used to learn the interaction between subcarriers; a batch normalization layer is applied after the convolutional layer to reduce internal variable shift; and a dropout layer is then used to reduce overfitting, so that the network has better generalization ability in practical applications. The network can be denoted as F c :Fc =CNN(D o θ c ). Where D o is the pre-processed Wi-Fi channel status information data, θ c It is the set of all parameters output by CNN.
[0063] The recurrent neural network is composed of two bidirectional long short-term memory artificial neural network modules (BiLSTM), which is a special type of recurrent neural network that can learn problems with long-distance time dependencies. Traditional LSTM can extract forward features in time series. On the other hand, activity information depends not only on the future (forward), but also on the past (backward) moments. Therefore, using only the LSTM algorithm will inevitably lose some key features, resulting in a decrease in system performance. To solve this problem, the embodiment of the present application uses BiLSTM, which contains two time streams from the starting point to the end point and from the end point to the starting point. Future features are extracted through the forward state of the subcarrier, and past features are extracted through the backward state. Specifically, the network can be recorded as F r :F r =BiLSTM(F c θ r ). Where F c is the output parameter of CNN, θ r is the set of all parameters output by BiLSTM.
[0064] A fully connected neural network is one in which each node is connected to all nodes in the previous layer. The embodiment of this application is composed of two fully connected layers (Fully Connected Layer), which can integrate the feature representations previously learned through the convolutional neural network and the recurrent neural network. The first fully connected layer can use ReLU as the activation function, and the second fully connected layer can use Softmax as the activation function to achieve classification.
[0065] Then, in step 220, a global model, at least one local model, and at least one new domain model are constructed based on the feature extraction network framework. The global model, local models, and new domain models share the same network framework, using the feature extraction network architecture constructed in step 1. Specifically, one global model, n local models, and m new domain models can be constructed. When constructing the global model, local models, and new domain models, each model generates its own initial model parameters.
[0066] Then, in step 230, the local model is trained based on the parameters of the global model and the local gesture training data to update the parameters of the local model and obtain updated local model parameters; here, each local model uses the local gesture training data Training is performed to update the parameters of the local model, where Represents the local gesture training data corresponding to the i-th local model, and obtains the updated local model parameters.
[0067] Then, in step 240, the new domain model is trained based on the parameters of the global model and the personalized gesture training data to update the parameters of the new domain model; here, the personalized gesture training data set Use a small amount of labeled data from a new field (generally no more than 2 samples), where represents the personalized gesture training data corresponding to the i-th new domain model.
[0068] Then, in step 250, the parameters of the global model are updated based on the updated local model parameters and the updated new domain model parameters. In this step, the local model and the new domain model push their updated local model parameters to the global model, and the global model is updated based on the formula Update the parameters of the global model, where θ i It represents the i-th model parameter received by the global model. The model parameter here can be the updated local model parameter or the updated new domain model parameter.
[0069] Then, in step 260, the process of training the local models, training the new domain model, and updating the parameters of the global model is repeated. When the global model converges, an updated global model, at least one trained local model, and at least one trained new domain model are obtained. Here, the above training steps are repeated with increasing iterations until the global model converges. For example, convergence can be achieved by the loss function used during training approaching zero, while each local model is personalized. Ultimately, a highly generalized global model, multiple local models, and multiple personalized new domain models are obtained.
[0070] The gesture recognition model training method of the embodiment of the present application initializes global model parameters using a small amount of preprocessed Wi-Fi channel state information data. Each local model then downloads the model parameters from the global model and applies them to itself. The model is then fine-tuned using its own dataset before the model parameters are transferred to the global model. This cycle continues, resulting in a highly generalized global model and multiple personalized local models.
[0071] Figure 4A diagram 400 shows the relationship between parameter downloading and parameter uploading among the global model, local model, and new domain model according to an embodiment of the present application. In order to ensure that the global model has initial classification capabilities, the global model can be initialized, which can specifically include: randomly selecting a small amount of sample data (generally no more than 2 samples) from the new domain data to form personalized gesture training data, training the global model, and using classification cross entropy loss during the training process. Optimize the model to obtain the initial parameters of the global model Where V represents the output dimension of the classification, y represents the true distribution, and y c represents the predictive distribution.
[0072] In an embodiment of the present application, training a local model based on the parameters of the global model and the local gesture training data to update the parameters of the local model may include:
[0073] Download the parameters θ of the global model g , update the parameters of the local model. Here, during the first training process, the initial parameters of the global model are downloaded In the subsequent multiple training processes, the updated global model parameters are downloaded; the local model is trained based on the local gesture training data and the local model parameters are updated. Similarly, the local model also uses the classification cross entropy loss during the training process. Optimize implementation.
[0074] In an embodiment of the present application, training a new domain model based on the parameters of the global model and personalized gesture training data to update the parameters of the new domain model may include:
[0075] Download the parameters θ of the global model g , update the parameters of the new domain model. Here, during the first training process, the initial parameters of the global model are downloaded In the subsequent multiple training processes, the updated global model parameters are downloaded; the new domain model is trained based on the personalized gesture training data and the parameters of the new domain model are updated. Similarly, the new domain model also uses the classification cross entropy loss during the training process. Optimize implementation.
[0076] Another embodiment of the present application provides a gesture recognition method, which can be used to recognize new-domain gesture data corresponding to a trained new-domain model, including:
[0077] By applying the above embodiment, an updated global model, a trained global model, and a trained new domain model are obtained. For the recognition of new domain gesture data, it is only necessary to download the parameters of the updated global model and replace the parameters of the trained new domain model with the parameters of the downloaded trained global model. Then, the trained new domain model is retrained based on the personalized gesture training data to obtain a second-trained new domain model, thereby achieving the purpose of fine-tuning the trained new domain model. Finally, the gesture data to be recognized is input into the second-trained new domain model to obtain the gesture recognition result.
[0078] In other embodiments, the gesture recognition method may also be used to recognize gesture data in a reconstructed new domain, including:
[0079] Apply the method in the above embodiment to obtain an updated global model; download the parameters of the updated global model; update the parameters of the reconstructed new domain model according to the parameters of the updated global model, where the reconstructed new domain model can be constructed according to the feature extraction network framework; train the reconstructed new domain model based on the personalized gesture training data corresponding to the reconstructed new domain model to obtain a trained reconstructed new domain model; input the gesture data to be recognized into the trained reconstructed new domain model to obtain a gesture recognition result, where the gesture data to be recognized is the gesture data in the reconstructed new domain.
[0080] The gesture recognition method in the embodiment of the present application is based on a generalized cross-domain universal model and multiple personalized local models obtained through training. It can not only provide a high-precision target gesture recognition rate across domains, but also greatly reduce the cost required for centralized data collection.
[0081] Figure 5 FIG2 shows a structural block diagram of a gesture recognition model training device 500 according to another embodiment of the present application.
[0082] The apparatus 500 includes a network framework construction module 510 configured to construct a feature extraction network framework;
[0083] A model building module 520 is configured to build a global model, at least one local model and at least one new domain model based on a feature extraction network framework;
[0084] The model training module 530 is configured to train a local model based on the parameters of the global model and the local gesture training data to update the parameters of the local model; train a new domain model based on the parameters of the global model and the personalized gesture training data to update the parameters of the new domain model; update the parameters of the global model based on the updated local model parameters and the updated new domain model parameters; repeat the process of training the local model, training the new domain model, and updating the parameters of the global model, and when the global model converges, obtain an updated global model, at least one trained local model, and at least one trained new domain model.
[0085] The gesture recognition model training device of the embodiment of the present application initializes global model parameters using a small amount of preprocessed Wi-Fi channel state information data. Each local model then downloads the model parameters from the global model and applies them to itself. The model is then fine-tuned using its own dataset and the model parameters are then transferred to the global model. This cycle results in a highly generalized global model and multiple personalized local models.
[0086] In other embodiments, the model training module is further configured to: download parameters of the global model and update parameters of the local model; train the local model based on local gesture training data and update parameters of the local model.
[0087] In other embodiments, the model training module is further configured to: download parameters of the global model and update parameters of the new domain model; train the new domain model based on personalized gesture training data and update parameters of the new domain model.
[0088] In other embodiments, the apparatus further includes: an initialization module configured to train the global model based on the initialization training data to obtain initial parameters of the global model.
[0089] Comparison of experimental results:
[0090] The technical effects of the Wi-Fi cross-domain gesture recognition method provided in the embodiment of the present application are evaluated from the following four aspects: (1) recognition performance with different numbers of trained users; (2) recognition performance with different environments and different users; (3) recognition performance with different numbers of classified gestures; and (4) comparative evaluation of different models.
[0091] (1) Recognition performance with different numbers of training users
[0092] Figure 6The system's effectiveness was verified using two datasets, Widar3.0 and WiAR, each containing 9 and 16 gestures, respectively, and tested with two users. The number of users in each dataset was increased from 3 to 6 as training data. As shown in the figure, the Widar3.0 dataset maintains good performance as the number of training users increases. This demonstrates that the small number of training domains in the Widar3.0 dataset can also achieve high-precision predictions for new domains. Experimental results with WiAR show that accuracy improves with the increase in training data, with an increase of nearly 10% when the number of users increases from 3 to 6.
[0093] (2) Recognition performance in different environments and for different users:
[0094] The robustness of the system in different environments using the Widar3.0 dataset. Data from 9 users collected in the classroom was used for training, and data from 2 users collected in the hall and 4 users collected in the office were used for testing. These 15 users have different body shapes. The same activities of the three environments were selected, such as pushing, pulling, sweeping, patting, drawing circles, and drawing zigzags. Specifically, the number of test users in the hall was increased from 1 to 2, and the number of test users in the office was increased from 1 to 4. Figure 7 In the 2018 dataset, even when the number of test users increased from 1 to 4, the average accuracy remained 100%. Furthermore, the system was evaluated using data from the lobby and office (with 6 test users), and the results showed that the average accuracy only dropped by 1.2%. Overall, the system can work effectively across multiple domains as the number of test users grows.
[0095] (3) Recognition performance of gesture quantity in different categories:
[0096] Figure 8 We trained the system on six training sets with different numbers of users for 4, 8, 12, and 16 gestures, and used the same test set to evaluate the recognition performance. We can see that as the number of gestures increases, the recognition accuracy decreases, but it still maintains a high level of accuracy.
[0097] (4) Comparative evaluation of different models:
[0098] Figure 9 The method of the embodiment of the present application is compared with Widar3.0. In the experiment, for each combination training of 7 users provided by Widar3.0, the data of the remaining people are used for testing. Figure 9 Compared to Widar 3.0, our method improves accuracy by 20%. In addition, our method provides more robust performance, with an accuracy of over 89% across 7 users.
[0099] In general, the gesture recognition technology of this application greatly reduces costs and can achieve satisfactory high-precision cross-domain gesture recognition accuracy.
[0100] The above descriptions are merely examples of various embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any modifications or substitutions that can be readily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included within the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.
Claims
1. A gesture recognition model training method, comprising: Build a feature extraction network framework; constructing a global model, at least one local model, and at least one new domain model based on the feature extraction network framework; Training the local model based on the parameters of the global model and the local gesture training data to update the parameters of the local model; For a certain environment or a certain user, if the Wi-Fi channel state information data of the environment or the user has been obtained before, the Wi-Fi channel state information data corresponding to the environment or the user is the local gesture training data; Training the new domain model based on the parameters of the global model and the personalized gesture training data to update the parameters of the new domain model; For a certain environment or a certain user, if the Wi-Fi channel state information data of the environment or the user has never been obtained before, the Wi-Fi channel state information data corresponding to the environment or the user is the new domain data; a portion of the data is randomly selected from the new domain data, which is the personalized gesture training data; updating the parameters of the global model based on the updated parameters of the local model and the updated parameters of the new domain model; The process of training the local model, training the new domain model, and updating the parameters of the global model is repeated, and when the global model tends to converge, an updated global model, at least one trained local model, and at least one trained new domain model are obtained.
2. The method according to claim 1, wherein The feature extraction network includes a convolutional neural network, a recurrent neural network and a fully connected neural network connected in sequence.
3. The method according to claim 1, wherein The training of the local model based on the parameters of the global model and the local gesture training data to update the parameters of the local model includes: Downloading the parameters of the global model and updating the parameters of the local model; The local model is trained based on the local gesture training data, and parameters of the local model are updated.
4. The method according to claim 1, wherein The training of the new domain model based on the parameters of the global model and the personalized gesture training data to update the parameters of the new domain model includes: Downloading parameters of the global model and updating parameters of the new domain model; The new domain model is trained based on the personalized gesture training data, and parameters of the new domain model are updated.
5. The method according to claim 1, wherein The method further includes initializing the global model, comprising: The global model is trained based on the initialization training data to obtain initial parameters of the global model.
6. A gesture recognition method, comprising: Apply the method of claim 1 to obtain an updated global model and a trained new domain model; Downloading parameters of the updated global model; Updating the parameters of the trained new domain model according to the parameters of the updated global model; Retraining the trained new domain model based on the personalized gesture training data to obtain a second-trained new domain model; The gesture data to be recognized is input into the new domain model after the secondary training to obtain a gesture recognition result.
7. A gesture recognition method, comprising: Applying the method of claim 1 to obtain an updated global model; Downloading parameters of the updated global model; updating and reconstructing parameters of the new domain model according to the parameters of the updated global model; Training the reconstructed new domain model based on the personalized gesture training data corresponding to the reconstructed new domain model to obtain a trained reconstructed new domain model; The gesture data to be recognized is input into the trained reconstructed new domain model to obtain a gesture recognition result.
8. A gesture recognition model training device, comprising: A network framework building module is configured to build a feature extraction network framework; a model building module, configured to build a global model, at least one local model and at least one new domain model based on the feature extraction network framework; A model training module is configured to construct a global model, at least one local model, and at least one new domain model based on the feature extraction network framework; train the local model based on the parameters of the global model and local gesture training data to update the parameters of the local model; train the new domain model based on the parameters of the global model and personalized gesture training data to update the parameters of the new domain model; update the parameters of the global model based on the parameters of the updated local model and the parameters of the updated new domain model; repeat the process of training the local model, training the new domain model, and updating the parameters of the global model, and when the global model tends to converge, obtain an updated global model, at least one trained local model, and at least one trained new domain model; For a certain environment or a certain user, if the Wi-Fi channel state information data of the environment or the user has been obtained before, the Wi-Fi channel state information data corresponding to the environment or the user is the local gesture training data; For a certain environment or a certain user, if the Wi-Fi channel state information data of the environment or the user has never been obtained before, the Wi-Fi channel state information data corresponding to the environment or the user is the new domain data; a portion of the data is randomly selected from the new domain data, which is the personalized gesture training data.
9. The device according to claim 8, wherein The model training module is further configured to: Downloading parameters of the global model and updating parameters of the local model; training the local model based on the local gesture training data and updating parameters of the local model.
10. The device according to claim 8, wherein The model training module is further configured to: Downloading parameters of the global model and updating parameters of the new domain model; training the new domain model based on the personalized gesture training data and updating parameters of the new domain model.