Human body posture estimation method based on self-supervised learning Wi-Fi sensor network

Abstract

The invention discloses a self-supervised learning-based method for estimating a human body posture by using a Wi-Fi sensing network. The method mainly comprises two parts of acquiring and processing CSI of a Wi-Fi signal containing human behaviors and training a deep neural network for sensing the human behaviors. The method comprises the following steps: firstly, acquiring Wi-Fi data for sensing human activities, and preprocessing the data by using different digital signal processing technologies; secondly, performing preliminary self-supervised training on the obtained data by using a data extraction and compression module; then adding a time feature extraction and classification module, and combining the compression module in the previous step to carry out self-supervised training of a second part; and finally, inputting a small amount of accurate sample data with labels, and carrying out model correction work. According to the human activity recognition method, a self-supervised training method is adopted, a large amount of time does not need to be consumed for collecting and labeling data, in addition, a Wi-Fi signal is used for executing a perception recognition task, and the method can be used without any additional cost of a wearable sensor or a special system or the like.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and mainly relates to a channel state information (CSI) human activity recognition method and system based on deep learning. Background Art

[0002] Human posture estimation is an important task in computer vision and sensing technology, and is widely used in sports, intelligent security, human-computer interaction monitoring, virtual reality, medical rehabilitation and other fields. The position information of limbs can analyze human behavior in real time, predict movement trajectories, and even detect abnormal behavior. This is of great value in improving equipment standardization, improving the quality of human life, and enhancing user experience. For example, in motion analysis, posture estimation can help evaluate the quality of movement; in the medical field, it can assist patients in rehabilitation training and provide accurate movement feedback.

[0003] Camera-based visual posture estimation methods need to rely on high-definition images or videos for calculation. Under complex indication conditions (such as darkness, strong light, and nighttime), the accuracy of posture recognition may drop significantly. In addition, when dealing with multiple human scenes or object scenes, traditional visual methods cannot accurately obtain the limb position information of the scene, resulting in inaccurate posture estimation results. In sensor-based posture estimation methods, sensors may expose personal privacy when collecting images, especially in public places or personal spaces, which can easily cause concerns about privacy and security. Wearing or installing multiple inertial sensors, such as speedometers, gyroscopes, inertial measurement units, etc., not only increases hardware costs, but may also affect user comfort and experience.

[0004] Wi-Fi sensing technology has gradually become an emerging solution for human posture estimation due to its cost-effectiveness and privacy protection advantages. Current technology combines improved deep learning models with CSI in Wi-Fi signals to effectively analyze the motion patterns of objects, but traditional learning models have obvious deficiencies in environmental dependence, and there are problems such as large workloads in data set acquisition and annotation, and high requirements for data set accuracy. In summary, a human posture estimation algorithm that does not require sensors, has low dependence on data sets, has low network complexity, and is easy to deploy in edge devices is one of the key technologies to solve human posture estimation. Summary of the invention

[0005] To achieve the above purpose, the technical solution adopted by the present invention is a Wi-Fi sensor network SCDNet based on self-supervised learning for human posture estimation. The method comprises the following steps:

[0006] Step 1: Based on the wireless network card, the corresponding Wi-Fi channel status information is collected for different human postures and movements to form a data set;

[0007] Step 2: Divide the data set obtained in step 1 into a training set and a test set, with a sample ratio of 8:2 between the training set and the test set, and then perform data set preprocessing operations;

[0008] Step 3: Build the SCDNet model and the segmented self-supervised training framework based on the Pytorch framework. The EntLoss loss function is used in the first stage of self-supervised training, the symmetric KL divergence is used as the loss function in the second stage of self-supervised training, and the cross entropy loss function is used in the final few-sample supervised training stage. The settings of the training stage include: batch size, number of iterations, optimizer and other hyperparameters. After the settings are completed, the model training is carried out in the offline stage;

[0009] Step 4: Test, evaluate, screen and adjust the trained SCDNet network, and use the recognition accuracy to measure the reasoning accuracy of the model, which is calculated as the percentage of the number of correct predictions to the total number of samples. Deploy the selected optimal model to the terminal for testing.

[0010] Furthermore, the specific steps of implementing the preprocessing of the data set in step 2 include:

[0011] Step 2.1: Extract the phase and amplitude of the subcarrier in the CSI and preprocess them respectively to obtain the preprocessed subcarrier phase and amplitude signals;

[0012] Step 2.2: The preprocessing described in step 2.1 includes: amplitude smoothing and noise removal, phase unwrapping, filtering and linear fitting;

[0013] Furthermore, the SCDNet model built based on the Pytorch framework in step 3 includes three parts: Encoder feature extraction module, RecurrentBlock cycle module and Classifier classification module. The self-supervised training scheme adopts the following steps: the Encoder trains the feature extraction module separately in the first stage of self-supervision, the network is trained as a whole in the second stage of self-supervision, and the overall optimization is performed in the few-sample supervised training stage.

[0014] Furthermore, the first part of the Encoder feature extraction module in step 3 extracts features of the phase and amplitude signals respectively, and then adds a Concatenate layer, uses the feature map splicing method to combine the features of the channel dimension, and adds a convolution layer after the Concatenate layer to perform feature fusion and further feature extraction. Finally, the feature map is reshaped to be suitable for subsequent network layers.

[0015] Furthermore, the second part of the Encoder feature extraction module in step 3 includes a feature extractor that uses three DilatedBlock convolution blocks in parallel; a CatModule feature fusion module is used to perform multiple splicing and fusion operations on the above DilatedBlock output; and a GroupedQueryAttention mechanism is used to perform attention-based feature extraction on the above CatModule module output. Furthermore, the convolution expansion rates of the DilatedBlock used in step 3 are d=1, d=2, and d=3, respectively, and the convolution kernel sizes used in the four convolutional layers are (5,1), (3,1), (1,3), and (1,5), respectively. The CatModule module used includes a Concatenation layer and two convolutional layers, wherein the stride of the convolutional layer is adjustable. The outputs of the DilatedBlocks of d=1 and d=2 are connected to a CatModule module with stride=1: Cat1; the original data that has not passed through the DilatedBlock convolution block and the output of the DilatedBlock convolution block of d=3 are connected to a CatModule module with stride=1: Cat2; the outputs of Cat1 and Cat2 are connected to a CatModule module with stride=2: Cat3. The size of the feature map output by Cat3 is reduced by 1 / 2 compared to the original feature map, and the size of the feature map output by Cat3 is (b*5,c,25,45).

[0016] Furthermore, the Cat3 output in step 3 is connected to the GroupedQueryAttention module through the Encoder_Fc module and a view tensor deformation operation, wherein the Encoder_Fc module includes a flatten layer and a Linear layer. The Linear layer controls the network output length through the compression_rate parameter. In this patent, compression_rate=0.05 is set. Finally, the Encoder_Fc module outputs a feature map of size (b*5,n), and the view tensor deformation operation outputs a feature map of size (b,5,n), which is adapted to the input format of the GroupedQueryAttention module.

[0017] Furthermore, the GroupedQueryAttention module in step 3 is a module based on a multi-head attention mechanism, which improves the model's ability to express input through a grouped multi-query mechanism. It consists of multiple MultiQueryAttention modules, each group is responsible for independent feature extraction, and finally the results of each group are spliced ​​and fused into a unified feature representation through a linear layer. A view operation is connected after the GroupedQueryAttention module to reintegrate the feature map, and its output is the output of the Encoder module. At this time, the feature map size is: (b*5,n).

[0018] Furthermore, the RecurrentBlock module is connected after the Encoder module in step 3. The RecurrentBlock module is a module for processing time sequence data based on GRU (Gated Recurrent Unit). The shape of the input data is (batch_size, sequence_length, input_size), hidden_size = 256, and a sequence of length 5 (9 features per time step) is input into the GRU network, and a hidden state vector of length 256 is output.

[0019] Furthermore, the Classifier module in step 3 is composed of a Flatten layer and three Linear layers. The Flatten layer flattens the input tensor (b, 5, 56) into a one-dimensional vector (b, 280), and the Linear layer maps the one-dimensional vector into (b, num_classes) three times, where the 1st Linear is from (b, 280) to (b, 512), the 2nd Linear is from (b, 512) to (b, 128), and the 3rd Linear is from (b, 128) to (b, num_classes), where num_classes is set according to the type of dataset.

[0020] Furthermore, the test, evaluation, screening and adjustment of SCDNet in step 4 include: fine-tuning the model architecture including convolutional layer settings, feature extraction compression rate and attention mechanism parameters according to actual test results; and retraining the network model to make the model achieve the most practical effect.

[0021] The method for estimating human posture using a Wi-Fi sensor network based on self-supervised learning described in the present invention has the following technical effects compared with the prior art:

[0022] The present invention relates to a method for human posture estimation using a Wi-Fi sensor network based on self-supervised learning, and innovatively adopts a CSI signal multi-dimensional feature extraction and segmented self-supervised learning architecture. The multi-dimensional feature extraction divides the CSI signal into two parts, phase and amplitude, and its advantages are: phase and amplitude carry different information, and separate extraction can capture features more comprehensively; separate processing reduces mutual interference and improves feature clarity. The advantages of segmented self-supervised learning are: the self-supervised learning of the feature extraction module uses GroupedQueryAttention to enhance the feature expression ability and reduce the dependence on large-scale label data; the self-supervised learning of the overall model introduces GRU to capture the time dependency information in the sequence; the combination of GroupedQueryAttention and GRU enhances the model's ability to capture complex spatiotemporal features. In short, the SCDNet model proposed in the present invention has the advantages of shallow layers, small number of parameters, and no need for high computing power. While realizing lightweight and real-time human posture estimation, the recognition accuracy meets the actual engineering needs, and has the characteristics of learning sample features from unlabeled CSI data sets and successfully migrating to user-defined specific tasks to achieve accurate human posture estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flowchart of the steps of the method for human posture estimation based on Wi-Fi sensor network for self-supervised learning of the present invention.

[0024] Figure 2 This is a SCDNet model framework diagram of a Wi-Fi sensor network for human posture estimation based on self-supervised learning in the present invention

[0025] Figure 3 This is a training accuracy curve diagram of a method for estimating human posture using a Wi-Fi sensor network based on self-supervised learning in the present invention.

[0026] Figure 4 This is a model performance comparison table of a Wi-Fi sensor network based on self-supervised learning for human posture estimation method of the present invention. DETAILED DESCRIPTION

[0027] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiment steps.

[0028] The present embodiment provides a flow chart of a method for using a Wi-Fi sensor network for human posture estimation based on self-supervised learning, as shown in FIG. Figure 1As shown. First, WiFi data of human activity perception is obtained, and digital signal processing technology is used to pre-process the data to obtain training sets and test sets; the training set data is used to perform segmented training on the SCDNet model in the offline stage; the test set is used to perform performance screening on the trained model to obtain the optimal model, which is finally deployed to the terminal for posture detection in the online stage. The specific implementation steps are as follows:

[0029] Step 1: Use specific Wi-Fi network equipment to collect channel state information (CSI) related to different human postures. Data collection includes multiple experiments in different environments (such as office, home, outdoor, etc.) to ensure the diversity and representativeness of the data set.

[0030] Step 2: Divide the data set obtained in step 1 into a training set and a test set, with a sample ratio of 8:2 between the training set and the test set, and then perform data set preprocessing operations;

[0031] Among them, the specific steps for preprocessing the data set in step 2 include:

[0032] Step 2.1: Extract the phase and amplitude of the subcarrier in the CSI and preprocess them respectively to obtain the preprocessed subcarrier phase and amplitude signals;

[0033] Step 2.2: The preprocessing described in step 2.1 includes: amplitude smoothing and noise removal, phase unwrapping, filtering and linear fitting;

[0034] Step 3: Build the SCDNet model and segmented self-supervised training framework based on the Pytorch framework.

[0035] Among them, the SCDNet model built based on the Pytorch framework includes three parts: Encoder feature extraction module, RecurrentBlock cycle module and Classifier classification module. The self-supervised training scheme adopts the following steps: the Encoder trains the feature extraction module separately in the first stage of self-supervision, the network is trained as a whole in the second stage of self-supervision, and the overall optimization is performed in the few-sample supervised training stage.

[0036] Among them, the Encoder feature extraction module: extracts features from the phase and amplitude signals respectively, and the extracted features are merged through the Concatenate layer to form a joint feature map. Multiple DilatedBlocks (expanded convolution blocks) are used for parallel processing to enhance the diversity of feature extraction. The specific convolution kernel size and dilation rate used are as described in

[0017] . GroupedQueryAttention module: the output of the Cat3 module is passed into the GroupedQueryAttention module, and the feature expression capability is enhanced through the multi-head attention mechanism. In the specific implementation, the grouped multi-query mechanism can enhance the fine-grained attention to the input features.

[0038] Among them, the RecurrentBlock module uses GRU (gated recurrent unit) to process time series data and capture the dynamic characteristics of human posture changing over time.

[0037] Among them, the Classifier module: flattens the hidden state vector output by GRU through the Flatten layer, and maps it to the classification result through multiple layers of fully connected layers.

[0038] The model training is divided into three stages: Self-supervised learning stage 1: Independently train the Encoder feature extraction module, using the EntLoss loss function. Self-supervised learning stage 2: Train the network as a whole, using the symmetric KL divergence as the loss function to enhance the feature learning ability of the overall model. Few-sample supervised training stage: Fine-tune on a small sample data set, using the cross entropy loss function to ensure the effectiveness of the model on a small number of labeled samples.

[0039] Step 4: Test, evaluate, screen and adjust the trained SCDNet network, and use the recognition accuracy to measure the inference accuracy of the model, which is calculated as the percentage of the number of correct predictions to the total number of samples. Deploy the selected optimal model to the terminal for testing. After the model training is completed, use the test set to evaluate the model:

[0040] Step 5: After the optimal model is verified, it is deployed to the edge computing device or terminal to achieve real-time human posture detection. The system can work adaptively in different environments and provide accurate human posture estimation to meet practical application needs.

[0041] This example demonstrates the application of Wi-Fi sensor networks based on self-supervised learning in human posture estimation, combining multi-dimensional feature extraction of CSI signals and segmented self-supervised learning strategies to significantly improve the accuracy and robustness of the model. This method has the characteristics of privacy protection and cost-effectiveness, shallow layers, and fewer parameters, and is suitable for real-time operation in resource-constrained environments.

[0042] Although the content of the present invention has been described in detail through the above preferred embodiments, it should be appreciated that the above description should not be considered as a limitation of the present invention. After reading the above content, it will be apparent to those skilled in the art that various modifications and substitutions of the present invention will occur. Therefore, the protection scope of the present invention should be limited by the appended claims.

Claims

1. A method for human posture estimation using a Wi-Fi sensor network based on self-supervised learning, characterized in that: The following steps are involved: Step 1: Based on the wireless network card, the corresponding Wi-Fi channel status information is collected for different human postures and movements to form a data set; Step 2: Divide the data set obtained in step 1 into a training set and a test set, with a sample ratio of 8:2 between the training set and the test set, and then perform data set preprocessing operations; Step 3: Build the SCDNet model and the segmented self-supervised training framework based on the Pytorch framework. The EntLoss loss function is used in the first stage of self-supervised training, the symmetric KL divergence is used as the loss function in the second stage of self-supervised training, and the cross entropy loss function is used in the final few-sample supervised training stage. The settings of the training stage include: batch size, number of iterations, optimizer and other hyperparameters. After the settings are completed, the model training is carried out in the offline stage; Step 4: Test, evaluate, screen and adjust the trained SCDNet network, and use the recognition accuracy to measure the reasoning accuracy of the model, which is calculated as the percentage of the number of correct predictions to the total number of samples. Deploy the selected optimal model to the terminal for testing.

2. The method for human posture estimation based on Wi-Fi sensor network based on self-supervised learning according to claim 1, characterized in that: The specific steps of implementing the preprocessing of the data set in step 2 include: 2.1) Extract the phase and amplitude of the subcarrier in the CSI and preprocess them respectively to obtain the preprocessed subcarrier phase and amplitude signals; 2.2) The preprocessing described in step 2.1) includes: amplitude smoothing and noise removal, phase unwrapping, filtering and linear fitting.

3. The method for human posture estimation based on Wi-Fi sensor network based on self-supervised learning according to claim 1, characterized in that: In step 3, the SCDNet model based on the Pytorch framework includes three parts: Encoder feature extraction module, RecurrentBlock cycle module and Classifier classification module. The self-supervised training scheme adopts the following steps: the feature extraction module is trained separately in the first stage of self-supervision, the network is trained as a whole in the second stage of self-supervision, and the overall optimization is performed in the few-sample supervised training stage.

4. The method for human posture estimation based on Wi-Fi sensor network based on self-supervised learning according to claim 1, characterized in that: The test, evaluation, screening and adjustment of SCDNet in step 4 include: fine-tuning the model architecture including convolutional layer settings, feature extraction compression rate and attention mechanism parameters according to actual test results; and retraining the network model to make the model achieve the most practical effect.

5. The method for human posture estimation based on Wi-Fi sensor network based on self-supervised learning according to claim 3, characterized in that: The first part of the feature extraction network extracts features from the phase and amplitude signals respectively. After that, a concatenate layer is added, and the feature map splicing method is used to combine the features of the channel dimension. After the concatenate layer, a convolution layer is added for feature fusion and further feature extraction. Finally, the feature map is reshaped to be suitable for subsequent network layers.

6. The method for human posture estimation based on Wi-Fi sensor network based on self-supervised learning according to claim 3, characterized in that: The second part of the feature extraction network includes a feature extractor that uses three DilatedBlock convolution blocks in parallel; a CatModule feature fusion module that performs multiple splicing and fusion operations on the above DilatedBlock output; a GroupedQueryAttention mechanism that performs attention-based feature extraction on the above CatModule module output; and a RecurrentBlock module and a Classifier module that perform time series processing and action classification recognition on the output of the above GroupedQueryAttention.

7. The method for human posture estimation based on Wi-Fi sensor network based on self-supervised learning according to claim 6, characterized in that: The convolution expansion rates of the DilatedBlock used are d=1, d=2, and d=3, respectively. The convolution kernel sizes used in the four convolutional layers are (5,1), (3,1), (1,3), and (1,5), respectively. The CatModule module used includes a Concatenation layer and two convolutional layers, where the stride of the convolutional layer is adjustable. The outputs of the DilatedBlocks of d=1 and d=2 are connected to a CatModule module with stride=1: Cat1; the original data that has not passed through the DilatedBlock convolution block and the output of the DilatedBlock convolution block of d=3 are connected to a CatModule module with stride=1: Cat2; The outputs of Cat1 and Cat2 are connected to a CatModule module with stride = 2: Cat3. The size of the feature map output by Cat3 is reduced by 1 / 2 compared to the original feature map.

8. The method for human posture estimation based on Wi-Fi sensor network based on self-supervised learning according to claim 6, characterized in that: The Cat3 output is connected to the GroupedQueryAttention module through the Encoder_Fc module and a view tensor deformation operation. The Encoder_Fc module includes a flatten layer and a Linear layer. The Linear layer controls the network output length through the compression_rate parameter. In this patent, compression_rate=0.05 is set. The final Encoder_Fc module outputs a feature map size of (b*5,n), and the view tensor deformation operation outputs a feature map size of (b,5,n), which is adapted to the input format of the GroupedQueryAttention module.

9. The method for human posture estimation based on Wi-Fi sensor network based on self-supervised learning according to claim 6, characterized in that: The GroupedQueryAttention module improves the model's ability to express input through a grouped multi-query mechanism. It consists of multiple MultiQueryAttention modules, each of which is responsible for independent feature extraction. The results of each group are finally spliced ​​and fused into a unified feature representation through a linear layer. The GroupedQueryAttention module is then connected to a view operation to reintegrate the feature map, and its output is the output of the Encoder module. At this time, the feature map size is: (b*5,n).

10. The method for human posture estimation based on Wi-Fi sensor network based on self-supervised learning according to claim 6, characterized in that: The RecurrentBlock module connected after the Encoder module is a module for time processing sequence data based on the gated recurrent unit (GRU). The shape of the input data is (batch_size, sequence_length, input_size), hidden_size = 256, and a sequence of length 5 (9 features per time step) is input into the GRU network, and a hidden state vector of length 256 is output.

11. The method for human posture estimation based on Wi-Fi sensor network based on self-supervised learning according to claim 6, characterized in that: The Classifier module connected after the RecurrentBlock module consists of a Flatten layer and three Linear layers. The Flatten layer flattens the input tensor (b, 5, 56) into a one-dimensional vector (b, 280), and the Linear layer maps the one-dimensional vector into (b, num_classes) three times, where the 1st Linear is from (b, 280) to (b, 512), the 2nd Linear is from (b, 512) to (b, 128), and the 3rd Linear is from (b, 128) to (b, num_classes). num_classes is set according to the type of dataset.

Images