A method for human pose estimation based on self-supervised learning of wi-fi sensor networks

Abstract

The application discloses a method for human posture estimation based on a Wi-Fi sensing network of self-supervised learning, which mainly comprises two parts of acquiring and processing CSI of Wi-Fi signals containing human behaviors and training a deep neural network for perceiving human behaviors. First, Wi-Fi data for perceiving human activities is acquired, and different digital signal processing technologies are used for data preprocessing; second, a data extraction and compression module is used for preliminary self-supervised training of the obtained data; then, a time feature extraction and classification module is added, and the second part of self-supervised training is carried out in combination with the compression module in the previous step; finally, a small amount of accurate sample data with labels is inputted, and model correction work is carried out. The human activity recognition method of the application adopts a self-supervised training method, does not need to consume a large amount of time for data collection and labeling, and uses Wi-Fi signals to perform a sensing recognition task, so that the method can be used without any additional costs such as wearable sensors or special systems.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, and mainly relates to a channel state information (CSI) human activity recognition method and system based on deep learning. BACKGROUND

[0002] Human pose estimation is an important task in computer vision and sensing technology, and is widely used in fields such as motion evaluation, intelligent security, human-computer interaction monitoring, virtual reality, medical rehabilitation, etc. The position information of the limbs can analyze the behavior of people in real time, predict the motion trajectory, and even detect abnormal behavior. This has important value in improving device standardization, improving the quality of human life, and enhancing user experience. For example, in motion analysis, pose estimation can help evaluate the quality of motion; in the medical field, it can assist patients in rehabilitation training and provide accurate motion feedback.

[0003] Camera-based visual pose estimation methods require high-definition images or videos for calculation, and in complex indication conditions (such as darkness, strong light, night), the accuracy of pose recognition may be significantly reduced. In addition, traditional visual methods cannot accurately obtain the limb position information of the scene when dealing with multiple human body scenes or object scenes, resulting in inaccurate pose estimation results. In sensor-based pose estimation methods, sensors may expose personal privacy when collecting images, especially in public places or personal spaces, which may cause concerns about privacy security. Wearing or installing multiple inertial sensors such as speedometers, gyroscopes, and inertial measurement units not only increases hardware costs, but also may affect user comfort and experience.

[0004] Wi-Fi sensing technology has gradually become an emerging solution for human pose estimation due to its cost-effectiveness and privacy protection advantages. Current technology combines improved deep learning models and CSI in Wi-Fi signals to effectively analyze object motion patterns, but traditional learning models have obvious deficiencies in environmental dependence, and there are problems such as large data set acquisition and labeling workload, and high accuracy requirements for data sets. In summary, an algorithm for human pose estimation that does not require sensors, has low dependence on data sets, and has low network complexity and is easy to deploy on edge devices is one of the key technologies for solving human pose estimation. SUMMARY

[0005] To achieve the above purpose, the technical solution adopted by the application is a Wi-Fi sensing network SCDNet based on self-supervised learning for human pose estimation. The method comprises the following steps:

[0006] Step 1: Based on a wireless network card, collect corresponding Wi-Fi channel state information for different human pose actions to form a data set;

[0007] Step 2: The data set obtained in step 1 is divided into a training set and a test set, the sample ratio of the training set to the test set is 8:2, and then the data set is preprocessed;

[0008] Step 3: The SCDNet model and the segmentation self-supervised training framework are built based on the Pytorch framework. In the first stage of self-supervised training, the EntLoss loss function is used, in the second stage of self-supervised training, the symmetric KL divergence is used as the loss function, and in the last few-sample supervised training stage, the cross-entropy loss function is used. The training stage settings include: batch size, iteration number, optimizer and other hyperparameters. After setting, the offline model training is carried out;

[0009] Step 4: The trained SCDNet network is tested, evaluated, screened and adjusted, and the recognition accuracy is used to measure the inference accuracy of the model, and the calculation method is the percentage of the number of correct predictions to the total number of samples. The optimal model screened is deployed to the terminal for testing.

[0010] Further, the preprocessing of the data set in step 2 includes the following specific steps:

[0011] Step 2.1: Extract the phase and amplitude of the subcarriers in the CSI and preprocess them respectively to obtain the preprocessed subcarrier phase and amplitude signals;

[0012] Step 2.2: The preprocessing in step 2.1 includes amplitude smoothing processing and noise removal processing, phase unwrapping, filtering and linear fitting;

[0013] Further, the SCDNet model based on the Pytorch framework in step 3 includes an Encoder feature extraction module, a RecurrentBlock periodic cycle module and a Classifier classification module. The self-supervised training scheme includes: self-supervised first stage Encoder training feature extraction module, self-supervised second stage training network as a whole, and few-sample supervised training stage for overall optimization.

[0014] Further, the first part of the Encoder feature extraction module in step 3 extracts features from the phase and amplitude signals, respectively, and then adds a Concatenate layer to combine the channel dimensions of the feature maps by feature map splicing. After the Concatenate layer, a convolution layer is added for feature fusion and further feature extraction, and finally the feature map is reshaped to adapt to the subsequent network layer.

[0015] Further, the second part of the Encoder feature extraction module in step 3 includes three DilatedBlock convolutional blocks in parallel; the CatModule feature fusion module is used to perform multiple splicing and fusion operations on the outputs of the DilatedBlock; the GroupedQueryAttention mechanism is used to perform feature extraction on the outputs of the CatModule module based on the attention mechanism.

[0016] Further, the dilated rate of the DilatedBlock used in step 3 is d=1, d=2, d=3, and the kernel size of the 4-layer convolutional layer is (5,1), (3,1), (1,3), (1,5) respectively. The CatModule module includes a Concatenation layer and two convolutional layers, where the convolutional layer stride is adjustable. The outputs of the DilatedBlock with d=1 and d=2 are connected to a CatModule module with stride=1: Cat1; the original data without DilatedBlock convolutional block and the output of the DilatedBlock with d=3 are connected to a CatModule module with stride=1: Cat2; the outputs of Cat1 and Cat2 are connected to a CatModule module with stride=2: Cat3. The feature map size output by Cat3 is reduced by 1 / 2 compared to the original feature map, and the feature map size output by Cat3 is (b*5,c,25,45).

[0017] Further, the Cat3 output in step 3 is connected to the GroupedQueryAttention module through the Encoder_Fc module and a view tensor deformation operation, where the Encoder_Fc module includes a flatten layer and a Linear layer. The Linear layer controls the output length of the network through the compression_rate parameter, and in this patent, compression_rate=0.05. The final feature map size output by the Encoder_Fc module is (b*5,n), and the view tensor deformation operation outputs a feature map size of (b,5,n), which is suitable for the input format of the GroupedQueryAttention module.

[0018] Further, the GroupedQueryAttention module in step 3 is a multi-head attention mechanism-based module, which improves the expression ability of the model to the input through the grouped multi-query mechanism, and is composed of multiple MultiQueryAttention modules, each group is responsible for independent feature extraction, and finally the results of each group are spliced and fused into a unified feature representation through a linear layer.

[0019] The GroupedQueryAttention module is followed by a view operation to reorganize the feature map, and the output of the view operation is the output of the Encoder module, at this time the size of the feature map is (b*5, n).

[0020] Further, the Encoder module in step 3 is connected to the RecurrentBlock module. The RecurrentBlock module is a GRU (Gated Recurrent Unit) based time processing sequence data module, the shape of the input data is (batch_size, sequence_length, input_size), hidden_size = 256, and a sequence of length 5 (9 features per time step) is input into the GRU network, and a hidden state vector of length 256 is output.

[0021] Further, the Classifier module in step 3 is composed of a Flatten layer and three Linear layers. The Flatten layer expands the input tensor (b, 5, 56) into a one-dimensional vector (b, 280), and the Linear layer maps the one-dimensional vector into (b, num_classes) three times, wherein the 1st Linear maps from (b, 280) to (b, 512), the 2nd Linear maps from (b, 512) to (b, 128), and the 3rd Linear maps from (b, 128) to (b, num_classes), and num_classes is set according to the number of classes of the data set.

[0022] Further, the step 4 of testing, evaluating, screening and adjusting the SCDNet includes: according to the actual test effect, fine-tuning the model architecture including the convolution layer setting, the feature extraction compression rate and the attention mechanism parameters; and retraining the network model to make the model achieve the actual most effective effect.

[0023] The Wi-Fi sensor network based on self-supervised learning for human posture estimation method has the following technical effects compared with the prior art:

[0024] This invention relates to a self-supervised learning-based Wi-Fi sensor network method for human pose estimation, innovatively employing an architecture of multi-dimensional feature extraction from CSI signals and segmented self-supervised learning. The multi-dimensional feature extraction divides the CSI signal into phase and amplitude components, offering advantages such as: phase and amplitude carry different information, and separate extraction allows for more comprehensive feature capture; separate processing reduces mutual interference and improves feature clarity. The segmented self-supervised learning offers advantages such as: the feature extraction module utilizes GroupedQueryAttention to enhance feature representation capabilities and reduce reliance on large-scale labeled data; the overall model's self-supervised learning introduces GRU to capture temporal dependencies in the sequence; and the combination of GroupedQueryAttention and GRU enhances the model's ability to capture complex spatiotemporal features. In summary, the proposed SCDNet model has the advantages of shallow layers, few parameters, and no need for high computational power. It achieves lightweight, real-time human pose estimation while maintaining recognition accuracy that meets practical engineering requirements. It features the ability to learn sample features from unlabeled CSI datasets and successfully transfer them to user-defined specific tasks, thus achieving accurate human pose estimation. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating the steps of the self-supervised learning-based Wi-Fi sensor network method for human pose estimation, as described in this invention.

[0026] Figure 2 This is a diagram of the SCDNet model framework for a self-supervised learning-based Wi-Fi sensor network for human pose estimation, as described in this invention.

[0027] Figure 3 This is a training accuracy curve of a self-supervised learning-based Wi-Fi sensor network method for human pose estimation according to the present invention.

[0028] Figure 4 This is a performance comparison table of models for a self-supervised learning-based Wi-Fi sensor network method for human pose estimation, as presented in this invention. Detailed Implementation

[0029] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0030] This embodiment provides a flowchart of a self-supervised learning-based Wi-Fi sensor network method for human pose estimation, as shown below. Figure 1The WiFi data perceived by human activities is first acquired, and the data is preprocessed using digital signal processing techniques to obtain a training set and a test set. The training set data is used to perform offline stage segmentation training on the SCDNet model. The test set is used to perform performance screening on the trained model to obtain the optimal model, which is finally deployed to the terminal for online stage posture detection. The specific implementation steps are as follows:

[0031] Step 1: Collect channel state information (CSI) related to different human postures using specific Wi-Fi network devices. Data collection includes multiple experiments in different environments (such as office, home, outdoor, etc.) to ensure the diversity and representativeness of the data set.

[0032] Step 2: Divide the data set obtained in step 1 into a training set and a test set, with a training set to test set sample ratio of 8:2, and then perform data set preprocessing operations;

[0033] Among them, the specific steps of preprocessing the data set in step 2 include:

[0034] Step 2.1: Extract the phase and amplitude of the subcarriers in CSI and perform preprocessing respectively to obtain preprocessed subcarrier phase and amplitude signals;

[0035] Step 2.2: The preprocessing in step 2.1 includes amplitude smoothing and noise removal, phase unwrapping, filtering, and linear fitting;

[0036] Step 3: Build the SCDNet model and the segmented self-supervised training framework based on the Pytorch framework.

[0037] Among them, building the SCDNet model based on the Pytorch framework includes an Encoder feature extraction module, a RecurrentBlock periodic cycle module, and a Classifier classification module. The self-supervised training scheme includes: self-supervised first stage Encoder training feature extraction module, self-supervised second stage training network as a whole, and few-shot supervised training stage for overall optimization steps.

[0038] Among them, the Encoder feature extraction module: respectively extracts the features of the phase and amplitude signals, and merges the extracted features through the Concatenate layer to form a joint feature map. Multiple DilatedBlock (dilated convolution block) parallel processing is adopted to enhance the diversity of feature extraction. The specific convolution kernel size and dilation rate are as described in

[0017] . GroupedQueryAttention module: the output of the Cat3 module is input into the GroupedQueryAttention module to enhance the feature expression ability through the multi-head attention mechanism. In the specific implementation, the multi-query mechanism of grouping can improve the fine-grained attention to the input features. Among them, the RecurrentBlock module adopts GRU (Gated Recurrent Unit) to process time series data and capture the dynamic features of human posture changes over time.

[0039] Among them, the Classifier module: the hidden state vector output by the GRU is flattened through the Flatten layer and mapped to the classification result through multiple fully connected layers.

[0040] Among them, the model training is divided into three stages: the first stage of self-supervised learning: independently train the Encoder feature extraction module using the EntLoss loss function. The second stage of self-supervised learning: train the whole network using the symmetric KL divergence as the loss function to enhance the feature learning ability of the whole model. The third stage of few-shot supervised training: fine-tune on the small sample dataset using the cross-entropy loss function to ensure the effectiveness of the model on a small number of labeled samples.

[0041] Step 4: Test and evaluate the trained SCDNet network, use recognition accuracy to measure the inference accuracy of the model, the calculation method is the percentage of the number of correct predictions and the total number of samples. The selected optimal model is deployed to the terminal for testing. After the model training is completed, the test set is used to evaluate the model:

[0042] Step 5: After the optimal model is verified, it is deployed to the edge computing device or terminal to realize real-time human posture detection. The system can work adaptively in different environments and provide accurate human posture estimation to meet the actual application requirements.

[0043] This embodiment demonstrates the application of Wi-Fi sensor network based on self-supervised learning in human posture estimation, combining multi-dimensional feature extraction of CSI signals and segmented self-supervised learning strategy, which significantly improves the accuracy and robustness of the model. This method has the characteristics of privacy protection and cost-effectiveness, shallow hierarchy and fewer parameters, and is suitable for real-time operation in resource-constrained environments.

[0044] While the application has been described in detail by reference to preferred embodiments thereof, it should be recognized that the description set forth herein is by way of example and that modifications of the procedures described can be employed without departing from the scope of the application. Accordingly, the scope of the application should be determined by the appended claims and equivalents thereof.

Claims

1. A method for human pose estimation using a self-supervised learning based Wi-Fi sensor network, characterized in that, The method comprises the following steps: Step 1: based on a wireless network card, collecting corresponding Wi-Fi channel state information for different human body posture actions to form a data set; Step 2: dividing the data set obtained in step 1 into a training set and a test set, the sample ratio of the training set to the test set being 8:2, and then performing a data set preprocessing operation; Step 3: building an SCDNet model and a segmented self-supervised training framework based on a Pytorch framework; in the first stage of self-supervised training, using an EntLoss loss function, in the second stage of self-supervised training, using symmetric KL divergence as a loss function, and in the last few-sample supervised training stage, using a cross-entropy loss function, the training stage settings including batch size, iteration number, optimizer, learning rate and loss function; after the settings are completed, the model training in the offline stage is performed; Step 4: testing and evaluating and adjusting the trained SCDNet network, using recognition accuracy to measure the inference accuracy of the model, the calculation method being the percentage of the number of correct predictions to the total number of samples, and deploying the selected optimal model to the terminal for testing.

2. The method for human pose estimation based on self-supervised learning of a Wi-Fi sensor network according to claim 1, characterized in that: The preprocessing of the data set in step 2 comprises the following steps: 2.1) extracting the phase and amplitude of the subcarriers in the CSI and respectively preprocessing them to obtain preprocessed subcarrier phase and amplitude signals; 2.2) the preprocessing in step 2.1) comprises amplitude smoothing processing and noise removal processing, phase unwrapping, filtering and linear fitting.

3. The method for human pose estimation based on self-supervised learning of a Wi-Fi sensor network according to claim 1, characterized in that: The SCDNet model based on the Pytorch framework in step 3 comprises an Encoder feature extraction module, a RecurrentBlock periodic cycle module and a Classifier classification module; the self-supervised training scheme comprises the following steps: training the feature extraction module alone in the first stage of self-supervised training, training the whole network in the second stage of self-supervised training, and optimizing the whole network in the few-sample supervised training stage.

4. The method for human pose estimation based on self-supervised learning of a Wi-Fi sensor network according to claim 1, characterized in that: The testing, evaluating, screening and adjusting of the SCDNet in step 4 comprise the following steps: according to the actual test effect, fine-tuning the model architecture including the convolutional layer setting, the feature extraction compression rate and the attention mechanism parameters; and retraining the network model to make the model achieve the actual most effective effect.

5. The method for human pose estimation based on self-supervised learning of a Wi-Fi sensor network according to claim 3, characterized in that: The first part of the feature extraction network respectively extracts features from the phase and amplitude signals, after which a Concatenate layer is added, a feature combination in the channel dimension is performed by using a feature map splicing method, a convolutional layer is added after the Concatenate layer for feature fusion and further feature extraction, and finally the feature map is reshaped to adapt to the subsequent network layer.

6. The method for human pose estimation based on self-supervised learning of a Wi-Fi sensor network according to claim 3, characterized in that: The second part of the feature extraction network comprises three DilatedBlock convolutional blocks in parallel. The CatModule feature fusion module is used for multiple splicing and fusion operations on the output of the DilatedBlock; the GroupedQueryAttention mechanism is used for feature extraction based on the attention mechanism on the output of the CatModule module; the RecurrentBlock module and the Classifier module are used for time series processing and action classification identification on the output of the GroupedQueryAttention.

7. The method for human pose estimation using self-supervised learning based Wi-Fi sensor network according to claim 6, wherein: The dilated rate of the DilatedBlock is d=1, d=2, d=3, and the convolution kernel size of the 4-layer convolution layer is (5,1), (3,1), (1,3), (1,5) respectively; the CatModule module includes a Concatenation layer and two convolution layers, wherein the convolution layer stride is adjustable, wherein the output of the DilatedBlock with d=1 and d=2 is connected to a CatModule module with a stride of 1: Cat1; the original data not passing through the DilatedBlock convolution block and the output of the DilatedBlock convolution block with d=3 are connected to a CatModule module with a stride of 1: Cat2; the outputs of Cat1 and Cat2 are connected to a CatModule module with a stride of 2: Cat3, and the feature map size output by Cat3 is reduced by 1 / 2 compared with the original feature map.

8. The method for human pose estimation using self-supervised learning based Wi-Fi sensor network according to claim 6, wherein: The Cat3 output is connected to the GroupedQueryAttention module through an Encoder_Fc module and a view tensor deformation operation, wherein the Encoder_Fc module includes a flatten layer and a Linear layer; the Linear layer controls the network output length through the compression_rate parameter, wherein compression_rate=0.05, and the final feature map size output by the Encoder_Fc module is (b*5,n); the view tensor deformation operation outputs a feature map with a size of (b,5,n), which is suitable for the input format of the GroupedQueryAttention module.

9. The method for human pose estimation using self-supervised learning based Wi-Fi sensor network according to claim 6, wherein: The GroupedQueryAttention module improves the expression ability of the model to the input through a multi-query mechanism, and is composed of multiple MultiQueryAttention modules, each of which is responsible for independent feature extraction; finally, the results of each group are spliced and fused into a unified feature representation through a linear layer; a view operation is connected after the GroupedQueryAttention module to reorganize the feature map, and the output of the Encoder module is the output of the GroupedQueryAttention module, and the feature map size is (b*5,n) at this time.

10. The method for human pose estimation using self-supervised learning based Wi-Fi sensor network according to claim 6, wherein: The RecurrentBlock module connected after the Encoder module is a module based on gated recurrent unit (GRU) for processing time series data, the shape of the input data is (batch_size, sequence_length, input_size), hidden_size = 256, a sequence with length 5 is input into the GRU network, and a hidden state vector with length 256 is output, wherein there are 9 features at each time step.

11. The method for human pose estimation using self-supervised learning based Wi-Fi sensor network according to claim 6, wherein: The Classifier module connected after the RecurrentBlock module is composed of a Flatten layer and three Linear layers; the Flatten layer expands the input tensor (b, 5, 56) into a one-dimensional vector (b, 280), and the Linear layers map the one-dimensional vector into (b, num_classes) for three times, wherein the 1st Linear maps from (b, 280) to (b, 512), the 2nd Linear maps from (b, 512) to (b, 128), and the 3rd Linear maps from (b, 128) to (b, num_classes), and num_classes is set according to the category of the data set.

Images