WiFi human body behavior identification method based on channel time sequence double-flow network
By using a channel-time dual-stream network, combining CNN and Bi-LSTM to extract channel and temporal features, and introducing a subcarrier attention mechanism, the problem of underutilization of sub-channel relationships in existing technologies is solved, thereby improving the accuracy and generalization ability of human behavior recognition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NORTHEASTERN UNIV CHINA
- Filing Date
- 2025-12-12
- Publication Date
- 2026-05-01
AI Technical Summary
Existing research focuses more on the time-related relationships of subcarriers and less on the relationships between subchannels, failing to fully capture the correlation between subcarriers and resulting in insufficient accuracy in human behavior recognition.
A channel-based temporal dual-stream network approach is adopted, which extracts channel and temporal features through convolutional neural networks (CNN) and bidirectional long short-term memory networks (Bi-LSTM), and introduces a subcarrier attention mechanism to capture the correlation between subcarriers. The behavior recognition is then performed by combining feature concatenation and a classifier module.
It improves the accuracy of human behavior recognition, reduces overfitting to specific temporal structures, and enhances the model's generalization ability, especially in the recognition of easily confused actions.
Smart Images
Figure CN121959232A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of wireless sensing and intelligent behavior recognition technology, and in particular to a WiFi human behavior recognition method based on a channel-time dual-stream network. Background Technology
[0002] In recent years, with the widespread adoption of commercial Wi-Fi, Channel State Information (CSI) of Wi-Fi signals has been extensively developed and applied, making Wi-Fi sensing technology a research hotspot. CSI can reflect human movement in a non-invasive manner because human movement affects the propagation path of Wi-Fi signals, causing significant changes in CSI. Compared to traditional sensing methods (such as those based on wearable devices, video images, or millimeter-wave technology), Wi-Fi sensing offers significant advantages such as wide coverage, low cost, strong applicability, good privacy protection, and immunity to weather conditions.
[0003] Recently, some researchers have begun to explore the use of dual-stream networks to extract features from both the channel stream and the temporal stream. Supported by multiple-input multiple-output (MIMO) and multi-carrier techniques, information can be simultaneously acquired from both the temporal and channel domains of the channel signaling system (CSI). Among these, channel features at the subcarrier dimension show good recognition performance in distinguishing behaviors with similar actions but different postures (such as "sitting down" and "lying down"). This is because different channel frequencies exhibit varying performance when penetrating the human body, resulting in different sensitivities to human activity. However, existing research focuses primarily on the temporal relationship between subcarriers, paying less attention to the relationships between individual subchannels, and has not yet fully captured the correlation between subcarriers. Summary of the Invention
[0004] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a WiFi human behavior recognition method based on channel-time dual-stream network. This method can not only focus on the timing information of subcarriers laterally, but also achieve vertical learning of information between subcarriers, effectively capturing the correlation between subcarriers.
[0005] To achieve the above objectives, the main technical solutions adopted by the present invention include: A WiFi human behavior recognition method based on a channel-time dual-stream network includes the following steps: Step S1: Data acquisition. Within the sensing area formed by multiple WiFi signal transceiver pairs, collect the Channel Status Information (CSI) data of human movements under each transceiver pair, record the corresponding movement category labels, construct a multi-view CSI dataset, and divide it into training and test sets. Step S2, data preprocessing, including removing outliers from the signal, removing noise, feature dimensionality reduction, and compression of redundant information; Step S3: Extract channel temporal features, including, Step S301: Disrupt the subcarrier data in the time dimension of the channel stream data; for time series data, retain the original time sequence relationship and construct the channel stream data; Step S302: The channel flow network adopts a convolutional neural network (CNN). The preprocessed channel flow data is input into the CNN for training. During each training session, the time order of the training CSI data is shuffled. An attention mechanism that can capture the correlation of each subcarrier is introduced into the CNN model for improvement. The improved CNN model is used for channel flow feature extraction. Step S303: Temporal flow feature extraction. The temporal flow network uses a bidirectional long short-term memory network (Bi-LSTM), with the original CSI time series as input. The shape of the input sequence is... Where T is the time length of the CSI sequence, each This represents the CSI feature vector at a certain time step, where K is the number of subcarriers and N is the number of antennas. After Bi-LSTM encoding, the hidden state vector at the end of the sequence is extracted. Step S304: Feature aggregation. The channel aggregation feature matrix output by the channel flow network and the time aggregation feature matrix output by the time flow network are concatenated along the feature dimension by feature concatenation to construct a unified spatiotemporal fusion representation. Step S4: After the fused feature vector is processed by layer normalization, it is input into the classifier module. The classifier module is used to integrate and analyze the feature information extracted by each network and output the probability distribution of the corresponding activity category. The activity category corresponding to the largest output probability value is selected as the recognition result of the model.
[0006] Furthermore, in step S1, the standard format of the CSI dataset is [sampling time, number of antennas, number of subcarriers].
[0007] Furthermore, in step S2, a Hampel filter is used to remove outliers from the original signal.
[0008] Furthermore, in step S2, after removing outliers, the signal is denoised using Discrete Wavelet Transform (DWT).
[0009] Furthermore, in step S2, the denoised signal is subjected to principal component analysis (PCA) for feature dimensionality reduction and redundant information compression.
[0010] Furthermore, in step S302, the improvement by introducing an attention mechanism capable of capturing the correlation of each subcarrier into the convolutional neural network (CNN) model includes, for the features in the CNN network... Perform a one-dimensional pooling operation in the time dimension to squeeze the material: ; in, To The compressed feature obtained after average pooling in the time dimension, where T represents the time series length of the entire CSI data segment, and i represents the i-th time step; Perform activation operations to generate attention weight vectors. : ; in, express function, express The function, where W1 is the weight matrix of the first fully connected layer in the activation module, and W2 is the weight matrix of the second fully connected layer in the activation module. and ,in and The vectors obtained represent the number of feature units along the subcarrier dimension and the dimensionality reduction factor, respectively. For the middle and The weights between them; Multiply the attention weights by the original features point by point: ; in, The output features after applying attention weights, This represents the attention weight for the s-th subcarrier of the corresponding channel c.
[0011] Furthermore, in step S302, the CSI channel flow feature extraction model adopts a network based on the ResNet18 architecture. The model front end performs preliminary feature reconstruction on the input data through two convolutional layers, and embeds two subcarrier attention modules in the network structure. The first attention module is set at the front end of the convolution after data reshaping, and the second attention module is set after the first convolutional layer and before the pooling layer.
[0012] Further, in step S303, the input feature dimension of the Bi-LSTM network is... K represents the number of subcarriers, N represents the number of antennas, and the hidden state dimension is 64.
[0013] Further, in step S304, the channel aggregation feature matrix output by the channel flow network is denoted as Z. C The time-aggregated feature matrix output by the time-stream network is denoted as Z. T , will Z C With Z TBy performing a concatenation operation along the feature dimension, the final channel-time joint feature vector Z is expressed as: Z = Concat(Z C Z T ), where Concat(•) represents the feature concatenation operation.
[0014] Furthermore, in step S4, the classifier module consists of a fully connected layer and a Softmax normalization function, the Softmax function being as follows:
[0015] in, Indicates the first The output value of each neuron This indicates the total number of activity categories. For the model to the first The predicted probability of a class; Using the cross-entropy loss function as the optimization objective, let the true label after one-hot encoding be... The loss function for a single sample is expressed as follows: ; in, Indicates the first The true label of the class, To predict the probability of this class, the model uses the Adam optimizer for parameter learning, calculates the gradient, and updates the weights using the backpropagation algorithm.
[0016] The beneficial effects of this invention are as follows: This invention generates channel stream data by shuffling CSI data in the time dimension, causing the channel stream data to no longer retain its original temporal information. This forces the model to focus on learning channel information in the subcarrier dimension, avoiding the model's dependence on temporal features. Simultaneously, shuffling the temporal order of CSI data during each model training effectively increases sample diversity, improves the model's generalization ability, and reduces overfitting to specific temporal structures. Secondly, this invention designs a one-dimensional attention mechanism at the subcarrier level, which can dynamically measure and adjust the contribution of each subcarrier feature to the final output, enabling the model to automatically focus on subcarriers more relevant to behavior discrimination. Attached Figure Description
[0017] Figure 1 This is a schematic diagram of the WiFi human behavior recognition method based on a channel-time dual-stream network according to the present invention. Figure 2 The confusion matrix represents the experimental results of the CNN baseline model on existing public datasets; Figure 3 This is the confusion matrix of the experimental results of the model of this invention on existing public datasets. Detailed Implementation
[0018] To better explain and facilitate understanding of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.
[0019] This invention provides a WiFi human behavior recognition method based on a channel-time dual-stream network, such as... Figure 1 As shown, it includes the following steps: Step S1, Data Acquisition Within a sensing area comprised of multiple WiFi signal transceiver pairs, subjects sequentially perform preset actions. The system simultaneously collects Channel State Information (CSI) data from each transceiver pair and records the corresponding action category labels, constructing a multi-view CSI dataset. After collection, the CSI dataset is divided into training and testing sets for subsequent model training and evaluation. The final CSI dataset is formatted as [sampling time, number of antennas, number of subcarriers].
[0020] Step S2: Data Preprocessing First, to improve the quality of the acquired WiFi CSI data, this invention employs a Hampel filter to preprocess the original signal, effectively removing outliers. WiFi CSI signals are often affected by various factors such as environmental noise, electromagnetic interference, and changes in the internal state of the device, leading to abnormal fluctuations such as spikes. The Hampel filter identifies and replaces outliers by comparing each data point with its neighborhood. Outliers, of which Represents the sample mean. Indicates standard deviation, As a constant factor, this method can effectively suppress the interference of outliers on subsequent analysis while preserving the main characteristics of the signal.
[0021] However, while Hampel filtering can remove most outliers, it cannot completely eliminate noise in the signal. Therefore, Discrete Wavelet Transform (DWT) is further used for denoising. Wavelet transform has significant advantages in time-frequency localization and multi-resolution analysis, effectively preserving key signal features while eliminating high-frequency noise. This transform results in a smoother CSI signal compared to the Hampel-filtered signal, effectively retaining key feature information.
[0022] Subsequently, principal component analysis (PCA) was performed on the CSI signals of the three antennas after wavelet denoising to further achieve feature dimensionality reduction and redundant information compression. PCA is a classic linear dimensionality reduction method that linearly transforms multidimensional correlated variables into a set of uncorrelated principal components, preserving the original information to the greatest extent. Considering the heterogeneity of the signals acquired by each antenna, PCA analysis was performed independently on its CSI data, and finally, a weighted average was calculated for the PCA results of each antenna to retain as much useful information as possible. The specific process is as follows: Let the original dataset be... First, a centering process is performed to eliminate the bias, and then the covariance matrix is constructed: ; Where X and Y are two variables, x i ,y i For the first The values of each sample, , Let E(•) be the mean of the two variables, E(•) be the expected value, and n be the sample size.
[0023] Then, eigenvalue decomposition was performed on the covariance matrix, and the first eigenvalues were selected. The eigenvectors corresponding to the largest eigenvalues form the eigenspace. Projecting the original data into this space achieves dimensionality compression, which maximizes information retention while significantly reducing redundant features, thus providing a more compact and discriminative data representation for subsequent data modeling.
[0024] Step S3: Extract channel temporal features, including, Step S301: Disrupt the subcarrier data in the time dimension of the channel stream data; for time series data, retain the original time sequence relationship and construct the channel stream data.
[0025] For channel stream data, to enable the model to learn the interrelationships between CSI subcarriers across different samples, rather than focusing solely on the temporal dependence of a single subcarrier, this invention shuffles the subcarrier data along the time dimension. This breaks the temporal sequence dependency, allowing the model to better focus on the changes and connections between subcarriers. For time series data, since the original temporal sequence needs to be preserved, no order modification is made. Subsequently, the channel feature extractor and the temporal feature extractor perform feature learning on the two types of data, respectively extracting subcarrier features and temporal features from the CSI data.
[0026] Step S302: The channel flow network adopts a convolutional neural network (CNN). The preprocessed channel flow data is input into the CNN for training. During each training, the time order of the training CSI data is shuffled. An attention mechanism that can capture the correlation of each subcarrier is introduced into the CNN model for improvement. The improved CNN model is used for channel flow feature extraction.
[0027] Since the subcarriers lack temporal sequence, the channel stream network uses a convolutional neural network (CNN) instead of a temporal model for feature extraction. During training, the preprocessed channel stream data is input into the CNN for training. To further improve the model's generalization ability, this invention not only generates channel stream data by shuffling the temporal sequence but also modifies the model's learning method. Specifically, during each training iteration, the model re-shuffles the temporal order of the training CSI data, effectively increasing the diversity of the training data. By continuously introducing samples with different sequences during the training phase, the model can learn on multiple different data arrangements, thereby effectively avoiding overfitting to a specific temporal structure.
[0028] Furthermore, to enable the model to focus on the more critical subcarriers between different actions, this invention designs an attention mechanism. This mechanism can capture the correlation of each subcarrier. Specifically, for a certain feature in a CNN network... First, a one-dimensional pooling operation is performed in the time dimension to squeeze the material.
[0029] In the channel and subcarrier index The input feature sequence at point has a time dimension of length of , Input features In time index The value at that location, The time dimension length (e.g., number of sampling points or number of frames). To The compressed features obtained after average pooling in the time dimension This represents the number of feature units in the subcarrier dimension.
[0030] Then, an activation operation is performed to generate the attention weight vector. :
[0031] in, This represents the attention weight of the s-th subcarrier corresponding to channel c, with a value range of [0,1]. express The function compresses the output to a range. . Represents the ReLU activation function: , and This operation acts as a bottleneck layer, in which... and These represent the number of feature units along the subcarrier dimension and the dimensionality reduction factor, respectively. The resulting vector For the middle and The weights between the attention features are then multiplied point-by-point by the attention weights: ; This represents the output feature after applying attention weights.
[0032] In this invention, the CSI channel flow feature extraction model employs a ResNet18-based network to achieve accurate recognition of human behavior. The model's front end first performs preliminary feature reconstruction on the input data through two convolutional layers, and embeds two subcarrier attention modules into the network structure. The first attention module is placed before the convolutional layer after data reshaping, while the second attention module is placed after the first convolutional layer and before the pooling layer. The backbone network includes four residual layers, each containing two residual blocks, and finally outputs a 64-dimensional feature vector after global pooling and fully connected layers.
[0033] Step S303: Temporal Stream Feature Extraction For temporal flow networks, CSI data is treated as a time-dependent sequential signal. While LSTM excels at modeling temporal relationships, traditional LSTMs only utilize historical information to influence the current hidden state, ignoring the potential contribution of future context. In fact, future information is equally important when recognizing dynamic behaviors. For example, both "punching" and "clapping" begin with a "wrist raise," but their meanings are entirely different. To fully capture the bidirectional temporal dependencies, the model employs a bidirectional long short-term memory network (Bi-LSTM), which can simultaneously integrate past and future contextual information, thereby enhancing the discriminative power of features.
[0034] In this invention, the behavior recognition model directly uses the original CSI time series as input, and the shape of the input sequence is as follows: T is the length of the time dimension, where each Indicates the first The CSI feature matrix is obtained at each time step. To utilize both preceding and following temporal information simultaneously, this invention employs a single-layer Bi-LSTM structure for temporal feature extraction. The input feature dimension of this Bi-LSTM network is... K represents the number of subcarriers (the number of subcarriers on each antenna), N represents the number of antennas, and the hidden state dimension is 64. After Bi-LSTM encoding, the hidden state vector at the end of the sequence is extracted. This vector is further mapped to a category space through a fully connected layer (fc) for the final classification and discrimination of behavior recognition.
[0035] Step S304, Feature Aggregation After completing channel feature extraction and temporal feature modeling, in order to fully integrate behavioral feature information from different dimensions, this invention fuses channel flow features and temporal flow features.
[0036] Specifically, the channel aggregation feature matrix output by the channel flow network is denoted as... The time-aggregated feature matrix output by the time-stream network is denoted as This invention uses a feature splicing method to... and A concatenation operation is performed along the feature dimension to construct a unified spatiotemporal fusion representation. Specifically, the final channel-temporal joint feature vector... Represented as: ; in, The feature concatenation operation combines feature representations from different sub-networks into a single overall feature. Through this feature aggregation mechanism, the model can simultaneously utilize information from both the channel and temporal dimensions to obtain a more discriminative joint feature representation, thereby improving the overall accuracy of behavior recognition.
[0037] Step S4: Behavior recognition, classification, and model training fuse feature vectors After layer normalization, the data is input to the classifier module. This module integrates and analyzes the feature information extracted by the aforementioned networks and outputs the probability distribution of the corresponding activity category. Finally, the activity category corresponding to the highest output probability value is selected as the model's recognition result.
[0038] In this embodiment, the classifier module consists of a fully connected layer and a Softmax normalization function. The goal of this module is to classify and recognize predefined human activities. The Softmax function maps the model output to probability values for each category, and its mathematical expression is as follows: ; in, Indicates the first The output value of each neuron This represents the total number of activity categories (i.e., the number of output nodes). For the model to the first The predicted probability of a class. The classifier outputs the probability of each activity pattern, and the activity pattern with the highest probability is taken as the result of the model's recognition.
[0039] After completing the model structure construction, this invention trains the entire neural network in a supervised manner to optimize model parameters and improve the accuracy of behavior recognition. Specifically, during the training phase, preprocessed CSI data is collected and fed into the model for forward propagation. The model outputs the prediction value for each activity category corresponding to each sample, denoted as [missing information]. ,in This represents the total number of behavior categories.
[0040] To measure the difference between the predicted results and the true labels, this invention uses the cross-entropy loss function as the optimization objective. Let the true labels after one-hot encoding be... The loss function for a single sample is expressed as follows: ; in, Indicates the first The actual label of the class (1 for positive classes, 0 for others). This represents the model's predicted probability for that class. This loss function effectively reflects the distance between the predicted probability distribution and the true distribution, thus guiding the model parameters towards a better outcome. The model uses the Adam optimizer for parameter learning, calculating gradients and updating weights through backpropagation.
[0041] This invention was experimentally evaluated on the publicly available dataset UT-HAR. The UT-HAR dataset contains seven categories of human activity, collected using an Intel 5300 wireless network card. Three pairs of antennas were used during the collection process, with each pair recording 30 subcarriers. All data was collected in the same indoor environment. Figure 2-3 As shown, Figure 2 This is the confusion matrix of the results from the CNN baseline model. Figure 3 This is the confusion matrix of the model proposed in this invention. Experimental results show that this invention significantly improves accuracy and reduces confusion rates in easily confused actions such as "sit down," "stand up," and "fall."
[0042] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Any modifications, alterations, substitutions, and variations made by those skilled in the art to the above embodiments are within the scope of the present invention.
Claims
1. A WiFi human behavior recognition method based on a channel-time dual-stream network, characterized in that, Includes the following steps: Step S1: Data acquisition. Within the sensing area formed by multiple WiFi signal transceiver pairs, collect the Channel Status Information (CSI) data of human movements under each transceiver pair, record the corresponding movement category labels, construct a multi-view CSI dataset, and divide it into training and test sets. Step S2, data preprocessing, including removing outliers from the signal, removing noise, feature dimensionality reduction, and compression of redundant information; Step S3: Extract channel temporal features, including, Step S301: Disrupt the subcarrier data in the time dimension of the channel stream data; for time series data, retain the original time sequence relationship and construct the channel stream data; Step S302: The channel flow network adopts a convolutional neural network (CNN). The preprocessed channel flow data is input into the CNN for training. During each training session, the time order of the training CSI data is shuffled. An attention mechanism that can capture the correlation of each subcarrier is introduced into the CNN model for improvement. The improved CNN model is used for channel flow feature extraction. Step S303: Temporal flow feature extraction. The temporal flow network uses a bidirectional long short-term memory network (Bi-LSTM), with the original CSI time series as input. The shape of the input sequence is... Where T is the time length of the CSI sequence, each This represents the CSI feature vector at a certain time step, where K is the number of subcarriers and N is the number of antennas. After Bi-LSTM encoding, the hidden state vector at the end of the sequence is extracted. Step S304: Feature aggregation. The channel aggregation feature matrix output by the channel flow network and the time aggregation feature matrix output by the time flow network are concatenated along the feature dimension by feature concatenation to construct a unified spatiotemporal fusion representation. Step S4: After the fused feature vector is processed by layer normalization, it is input into the classifier module. The classifier module is used to integrate and analyze the feature information extracted by each network and output the probability distribution of the corresponding activity category. The activity category corresponding to the largest output probability value is selected as the recognition result of the model.
2. The WiFi human behavior recognition method based on a channel-time dual-stream network according to claim 1, characterized in that: In step S1, the standard format of the CSI dataset is [sampling time, number of antennas, number of subcarriers].
3. The WiFi human behavior recognition method based on a channel-time dual-stream network according to claim 1, characterized in that: In step S2, a Hampel filter is used to remove outliers from the original signal.
4. The WiFi human behavior recognition method based on a channel-time dual-stream network according to claim 3, characterized in that: In step S2, after removing outliers, the signal is denoised using Discrete Wavelet Transform (DWT).
5. The WiFi human behavior recognition method based on a channel-time dual-stream network according to claim 4, characterized in that: In step S2, the denoised signal is subjected to principal component analysis (PCA) for feature dimensionality reduction and redundant information compression.
6. The WiFi human behavior recognition method based on a channel-time dual-stream network according to claim 1, characterized in that: In step S302, the improvement by introducing an attention mechanism capable of capturing the correlation of each subcarrier into the convolutional neural network (CNN) model includes, for the features in the CNN network... Perform a one-dimensional pooling operation in the time dimension to squeeze the material: ； in, To The compressed feature obtained after average pooling in the time dimension, where T represents the time series length of the entire CSI data segment, and i represents the i-th time step; Perform activation operations to generate attention weight vectors. : ； in, express function, express The function, where W1 is the weight matrix of the first fully connected layer in the activation module, and W2 is the weight matrix of the second fully connected layer in the activation module. and ,in and The vectors obtained represent the number of feature units along the subcarrier dimension and the dimensionality reduction factor, respectively. For the middle and Weights between them; Multiply the attention weights by the original features point by point: ； in, The output features after applying attention weights, This represents the attention weight for the s-th subcarrier of the corresponding channel c.
7. A WiFi human behavior recognition method based on a channel-time dual-stream network according to claim 1 or 6, characterized in that: In step S302, the CSI channel flow feature extraction model adopts a network based on the ResNet18 architecture. The model front end performs preliminary feature reconstruction on the input data through two convolutional layers, and embeds two subcarrier attention modules in the network structure. The first attention module is set at the front end of the convolution after data reshaping, and the second attention module is set after the first convolutional layer and before the pooling layer.
8. The WiFi human behavior recognition method based on a channel-time dual-stream network according to claim 1, characterized in that: In step S303, the input feature dimension of the Bi-LSTM network is... K represents the number of subcarriers, N represents the number of antennas, and the hidden state dimension is 64.
9. A WiFi human behavior recognition method based on a channel-time dual-stream network according to claim 1, characterized in that: In step S304, the channel aggregation feature matrix output by the channel flow network is denoted as Z. C The time-aggregated feature matrix output by the time-stream network is denoted as Z. T , will Z C With Z T By performing a concatenation operation along the feature dimension, the final channel-time joint feature vector Z is expressed as: Z = Concat(Z C Z T ), where Concat(•) represents the feature concatenation operation.
10. A WiFi human behavior recognition method based on a channel-time dual-stream network according to claim 1, characterized in that: In step S4, the classifier module consists of a fully connected layer and a Softmax normalization function, as follows: ； in, Indicates the first The output value of each neuron This indicates the total number of activity categories. For the model to the first The predicted probability of a class; Using the cross-entropy loss function as the optimization objective, let the true label after one-hot encoding be... The loss function for a single sample is expressed as follows: ； in, Indicates the first The true label of the class, To predict the probability of this class, the model uses the Adam optimizer for parameter learning, calculates the gradient, and updates the weights using the backpropagation algorithm.