Indoor human behavior identification method based on CSI information

Abstract

The invention relates to an indoor human behavior recognition method based on CSI information, and belongs to the technical field of behavior recognition. The method comprises the following steps: acquiring CSI signals of different indoor human behaviors through a wireless router and a wireless network card; performing iterative variance calculation on the CSI signals received by each antenna of the wireless network card, and removing the antenna which is most insensitive to activity; carrying out noise reduction on the processed CSI signal by adopting discrete wavelet transform, and carrying out dimensionality reduction on the CSI signal by adopting a principal component analysis method; constructing a human behavior recognition model based on BiLSTM and a self-attention mechanism, training the model through the training set, and guiding the model to perform parameter optimization by using the verification set until the training is completed; and collecting CSI signals of human activities, and inputting the collected CSI signals into the trained human behavior recognition model to realize recognition of human behaviors. According to the method, the capturing capability of the model on key action features is improved, and the recognition precision and robustness of the model are enhanced.

Description

Technical Field

[0001] The present invention belongs to the technical field of behavior recognition and relates to an indoor human behavior recognition method based on CSI information. Background Art

[0002] Human activity recognition (HAR) aims to identify various human behaviors, such as walking, sitting, falling, etc. Its research has many applications in different fields, so it has academic and commercial value, such as smart home, security monitoring, rescue system, etc. Due to its importance, many research works have been devoted to HAR in recent years, mainly relying on computer vision, sensors, etc. However, these methods require specific environments and special processing during deployment and debugging. For example, the observation range of the camera is limited, there are certain requirements for lighting, and line-of-sight conditions are required (no smoke or other obstacles). The dependence on specific equipment and environment leads to high cost and low efficiency, which hinders their application in daily life scenarios. Compared with the above methods, WiFi-based HAR has recently received widespread attention because of its advantages such as wide coverage, ubiquity, non-intrusiveness, and protection of user privacy and identity. There are two commonly used WiFi signal data: received signal strength indication (RSSI) and channel state information (CSI). In HAR, the channel state information has a finer granularity and contains rich environmental information, so CSI is usually collected for analysis.

[0003] Traditional machine learning algorithms use different feature extraction techniques to manually extract features from CSI signals to classify or identify human behaviors, such as support vector machines, K-nearest neighbors, etc. However, manual feature extraction is very time-consuming, and the extracted features are not comprehensive enough. Current deep learning algorithms mainly focus on convolutional neural networks, sparse autoencoders, and long short-term memory (LSTM), which can automatically learn features from input signals. The LSTM-based architecture is specifically used to extract temporal features. However, traditional LSTM can only process continuous channel state information signals in one direction, ignoring future context information, and when the sequence is long, the model tends to ignore distant information, and important information is also easily forgotten; in addition, CSI signals contain carriers of all antennas, so the computational complexity and amount of data processing are also very large. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a method for indoor human behavior recognition based on CSI information, by combining the antenna selection algorithm and the BiLSTM-based attention model, using the antenna selection algorithm to eliminate the antenna that is least sensitive to the activity, and then using the BiLSTM-based attention model for feature extraction to improve the accuracy of behavior recognition.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A method for indoor human behavior recognition based on CSI information, the method comprising:

[0007] S1, collect CSI signals of different indoor human behaviors through wireless routers and wireless network cards;

[0008] S2, performing iterative variance calculation on the CSI signals received by each antenna of the wireless network card, and eliminating the antenna that is least sensitive to the activity;

[0009] S3, using discrete wavelet transform to reduce noise on the processed CSI signal, using principal component analysis to reduce the dimension of the CSI signal, and constructing a training set and a validation set based on the preprocessed data;

[0010] S4. Build a human behavior recognition model based on BiLSTM and self-attention mechanism, where BiLSTM is used to extract time-related feature information in CSI signals, and the attention mechanism is used to highlight the information of key time steps and suppress redundant features;

[0011] S5. Train the model using the training set, and use the validation set to guide the model to perform parameter optimization until the training is completed;

[0012] S6. Collect CSI signals of human activities and input them into the trained human behavior recognition model to realize the recognition of human behavior.

[0013] Furthermore, step S1 is specifically to build a simulated indoor environment, in which the transmitting device is a wireless router and the receiving device is a computer equipped with a wireless network card; the collected human behaviors include standing, walking, jogging, changing clothes, cleaning, falling, sitting, squatting, bending over, and lying down, and the humans whose behaviors are collected include children, adult men, and the elderly.

[0014] Furthermore, step S2 is specifically to express the data packet received by the i-th antenna of the wireless network card as:

[0015]

[0016] In the formula, represents the signal strength of the nth data packet received by the ith antenna; N represents the number of data packets received by the antenna in time series, that is, the sequence length;

[0017] A sliding window of size Wz is set. The sliding window moves at a fixed step length on the signal sequence received by the i-th antenna. The variance of the signal in the sliding window is calculated each time it moves. After the movement is completed, a variance sequence is obtained.

[0018] Since the signal in the inactive range tends to be stable, the variance in the sliding window is small, while the signal in the active range is unstable, the variance in the sliding window will become larger. The antennas sensitive to the target action have overlapping human behavior signal windows with signal fluctuations during the action implementation phase, while the antennas insensitive to the action have no overlapping signal fluctuation windows. The antennas that are least sensitive to the activity are identified and eliminated based on whether they have overlapping signal fluctuation windows.

[0019] Furthermore, in step S4, the constructed human behavior recognition model includes a BiLSTM, a self-attention module, a flattening layer, a fully connected layer and a Softmax layer connected in sequence, wherein the BiLSTM and the self-attention module are connected based on a residual network structure.

[0020] Furthermore, in step S6, performing human behavior recognition based on the trained human behavior recognition model includes:

[0021] S61, inputting the collected CSI signals of human activities into the model, and extracting time-related feature information in the CSI signals through BiLSTM;

[0022] S62, weighted summation of the output of each time step of BiLSTM is performed through the self-attention module to highlight the information of key time steps and suppress redundant features;

[0023] S63, concatenate the output of the BiLSTM and the output of the self-attention module, retain the input information and fuse the attention enhancement features;

[0024] S64. The multi-dimensional feature tensor is converted into a one-dimensional vector through the flattening layer, and nonlinear mapping is performed through the fully connected layer to learn the relationship between features and categories. Finally, the Softmax layer converts the output of the fully connected layer into a probability distribution for classification decision.

[0025] Further, in step S64, performing nonlinear mapping through a fully connected layer to learn the relationship between features and categories includes using a fully connected layer to map the flattened one-dimensional vector to a preset target action to obtain an unnormalized score H fc :

[0026] H fc =Wfc ·H flatten +b fc

[0027] Where W fc represents the weight matrix, b fc represents the bias vector, H flatten A one-dimensional vector representing the output of the flatten layer;

[0028] The unnormalized scores are converted into probability distributions through the Softmax layer:

[0029]

[0030] In the formula, P(y m |X) represents the probability that the input X belongs to the mth type of behavior, y m represents the mth type of behavior, C represents the number of behavior categories, and H fc,m Represents the unnormalized score of the mth category behavior; find the category with the highest probability in the Softmax output:

[0031]

[0032] Indicates the final category to which the input X is classified.

[0033] The beneficial effects of the present invention are:

[0034] (1) This invention builds a simulated indoor environment and collects data on 10 typical daily behaviors (such as drinking water, falling, etc.) of different groups of people, focusing on enhancing the model's learning ability and recognition effect on indoor behavioral activities. By constructing a highly targeted data set, the present invention significantly improves the accuracy of the model in capturing indoor behavioral characteristics and effectively reduces the impact of scene mismatch on performance. Even in complex and diverse indoor environments, the model can still accurately identify behavioral activities, demonstrating strong scene adaptability and practical application value.

[0035] (2) The present invention uses an insensitive antenna elimination method to effectively eliminate the antennas that are least sensitive to motion in CSI signal reception, significantly reducing the system's operational complexity and amount of calculation. By optimizing the antenna selection process, the present invention significantly improves the model's ability to capture key motion features and effectively enhances the model's response efficiency and stability in processing complex scenarios. Even in the case of multipath interference and weak signals, the model can still maintain a high recognition accuracy and demonstrate strong robustness.

[0036] (3) The present invention effectively extracts continuous time-related features and enhances feature selection by combining BiLSTM with the attention mechanism. The hidden state of the BiLSTM output is weighted and summed through the attention module, thereby highlighting the information of the key time step and suppressing redundant features. The residual network is introduced to perform feature layer addition, effectively retaining the input information and integrating the attention-enhanced features. This makes the model have stronger generalization ability, can perform well in different time series feature extraction, and has strong generalization.

[0037] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0039] Figure 1 A schematic diagram of a flow chart of a method for indoor human behavior recognition based on CSI information provided by an embodiment of the present invention;

[0040] Figure 2 Schematic diagram of the principle of eliminating insensitive antennas;

[0041] Figure 3 (a) is a schematic diagram of the BiLSTM attention network structure;

[0042] Figure 3 (b) is a schematic diagram of the LSTM network structure;

[0043] Figure 4 The BiLSTM and attention modules are connected by a residual network structure;

[0044] Figure 5 is the confusion matrix obtained on the indoor dataset. DETAILED DESCRIPTION

[0045] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0046] An embodiment of the present invention provides a method for indoor human behavior recognition based on CSI information, such as Figure 1 As shown, the method specifically comprises the following steps:

[0047] 1. Data Collection

[0048] A simulated indoor environment was built, with a wireless router as the transmitting device and a computer equipped with a wireless network card as the receiving device; different behaviors of volunteers were collected, including: standing, walking, jogging, changing clothes, cleaning, falling, sitting, squatting, bending over, lying down, etc. The volunteers included children, adult men and the elderly. The number of samples for each action was not less than 800, the total number of samples in the data set was not less than 8,000, and each sample contained 300 data packets.

[0049] 2. Eliminate insensitive antennas

[0050] Antennas that are insensitive to activity are unlikely to reflect real human activity, and the presence of insensitive antennas will increase the system's computational complexity and amount of calculation. In a local range, variance can represent the degree of discreteness of instantaneous activity. If the signal in the inactive range is stable, the variance in the sliding window will be small, while if the signal in the active range is unstable, the variance in the sliding window will become larger. Based on this, if Figure 2 As shown, a sliding window is introduced to calculate the variance in the local range to measure the stability and instability of the signal, thereby distinguishing human behavior signals from environmental interference signals and further eliminating insensitive antennas.

[0051] Specifically, the data packet received by the i-th antenna of the receiving wireless network card is expressed as:

[0052]

[0053] Where N represents the number of data packets received by the antenna in time series, that is, the sequence length.

[0054] First, calculate the variance of the original signal with a sliding window size Wz of 3, with a step size of 2 for each move, to form a variance sequence:

[0055]

[0056] in, represents the signal strength of the pth data packet in the sliding window Wz of the ith antenna, Represents the mean center of the samples within the sliding window. represents the jth variance of the ith antenna, A i Represents the variance sequence, the length of the variance sequence k = (N-Wz) / M+1, and M is the step length of each move.

[0057] After multiple variance signal calculations, since the signal in the inactive range is relatively stable, the variance will become smaller and smaller and approach 0, that is, the boundary of the human behavior signal and the boundary of the environmental interference are clear. Therefore, the stability measurement based on the variance theory is: the signal in the inactive range tends to be stable, then the variance in the sliding window is smaller, and the signal in the active range is unstable, then the variance in the sliding window will become larger. The antenna that is sensitive to the target action has an overlapping human behavior signal window with signal fluctuations during the action implementation stage, while the antenna that is insensitive to the action has no overlapping signal fluctuation window, thereby realizing the identification and removal of the antenna that is least sensitive to the activity.

[0058] 3. Use discrete wavelet transform to reduce the noise of the signal and PCA principal component analysis to reduce the dimension, and build training set and validation set based on the preprocessed CSI signal

[0059] The nonlinear wavelet transform soft threshold method is used to reduce the noise of the data after the insensitive antenna is removed, and the signal is decomposed into components of different scales and frequencies. The wavelet transform formula is as follows:

[0060] W(S)={c l,N ,d l,N}

[0061] Where W(S) represents the coefficient set after the processed signal S is subjected to wavelet transform; l represents the scale of the wavelet transform, which determines the frequency range when the signal is decomposed; N represents the packet number, which controls the wavelet function ψ l,N (n) or scaling function φ l,N (n) translation.

[0062] c l,N is the approximation coefficient, which is the global trend information of the signal (low-frequency part):

[0063]

[0064] Among them, S n is the nth data packet received by the antenna; φ l,N (n) is a scaling function, the specific form of which depends on the selected wavelet basis and is used to extract low-frequency components.

[0065] d l,N is the detail coefficient, which contains local high-frequency detail information:

[0066]

[0067] Among them, ψ l,N (n) is a wavelet function obtained by translating and scaling the mother wavelet and used to extract high-frequency components. The soft threshold function is obtained after denoising the detail coefficients and compressing the data, expressed as:

[0068] d′ l,N =sign(d l,N )·max(|d l,N |-γ,0)

[0069] Among them, sign(·) is the sign function, γ is the soft threshold parameter, σ is the standard deviation of the noise, and M is the number of sampling points of the signal.

[0070] The soft threshold function achieves denoising by retaining larger wavelet coefficients and setting smaller wavelet coefficients to zero, and then reconstructing the signal using the processed coefficients. The reconstructed signal is:

[0071]

[0072] Then use PCA principal component analysis to remove the correlation between subcarriers and reduce the feature dimension. Follow the steps below:

[0073] First, the CSI matrix H = [H1, H2, ..., H N ] T Centering, subtract the mean column vector H from each column of H to obtain the centered matrix H m ;

[0074] Then, calculate the central matrix H m The covariance matrix H c , use the singular value decomposition (SVD) method to decompose the covariance matrix to obtain the eigenvalue λ and eigenvector Q:

[0075]

[0076] H c =QAQ T

[0077] Among them, A is a diagonal matrix; select the eigenvectors corresponding to the first p largest eigenvalues ​​in Q to obtain the transposed matrix Q p T , the dimension is p*m, and then with H of dimension m*n m Multiply them together to form a new matrix Hw , mapping high-dimensional features to low-dimensional feature space;

[0078] Finally, H and H w Multiply them together to ensure that only the majority of the variance is taken into account, and those variances close to 0 are excluded, thus completing the dimensionality reduction operation of the data, as follows:

[0079] H w =Q p T H m

[0080] X=H·H w

[0081] Where X is the CSI signal after dimensionality reduction.

[0082] 4. BiLSTM is used to extract the time-related feature information in the CSI signal after dimensionality reduction. Then the self-attention model takes the high-dimensional feature matrix of the time series learned from BiLSTM as input. The residual network structure is used to connect the BiLSTM and self-attention modules, such as Figure 4 As shown, the feature matrix is ​​added up every other layer as the output.

[0083] Traditional feature extraction methods, such as CNN, can extract local features, but their dependence on time is weak. LSTM cannot fully capture the global dependence of the previous and next information in the time series, so BiLSTM is used for feature extraction. BiLSTM can effectively filter out interference information and retain key features through bidirectional processing and gating mechanism. When the CSI signal fluctuates violently during human activities, it can still stably extract effective information, and can simultaneously focus on the global information and local dynamic changes of the time series, significantly improving the accuracy of extracting human activity features.

[0084] The reduced CSI signal is fed into BiLSTM, where the LSTM structure is as follows: Figure 3 As shown in (b), the input signal is expressed as:

[0085] X={x1,x2,…,x N}

[0086] Where N represents the sequence length, x t Indicates the current t-th data packet, automatic feature extraction is performed in both the forward and reverse directions. The hidden states of the forward and reverse directions are h Rt and h Lt , as shown below:

[0087] h Rt =LSTM forward (x t ,hRt(t-1) )

[0088] h Lt =LSTM backward (x t ,h Lt(t+1) )

[0089] The output at each time step is the concatenation of the forward and backward hidden states:

[0090]

[0091] The final output hidden state matrix B = {h1,h2,…,h N}, there are three dimensions, namely batch dimension, time step dimension and feature channel dimension.

[0092] The purpose of introducing the attention mechanism into CSI signal feature extraction is to solve the problem of dilution of key information caused by BiLSTM paying equal attention to the features of all time steps. The attention mechanism dynamically assigns weights to the time series features extracted by BiLSTM, thereby highlighting the most critical time step information for action recognition and generating an enhanced feature matrix H with higher discriminability. attention Moreover, the attention mechanism reduces the computational complexity of the system by suppressing non-effective feature information.

[0093] Specifically, the attention model uses the features learned from BiLSTM as input. The attention-enhanced features H are calculated according to the weight matrix attention , the specific calculation formula is as follows:

[0094]

[0095] r t =tanh(W T h t +b)

[0096] Among them, W T and b represent the weight vector and bias respectively, which are initialized by Xavier, that is, the weights are randomly initialized to normal distribution. t is a nonlinear transformation of the feature importance, α t It is right t The normalized attention distribution of determines the contribution weight of each time step.

[0097] Then B and H attention Concatenate and generate the final feature matrix:

[0098] H final =[B;H attention ]

[0099] 5. Flatten the high-dimensional feature matrix into a one-dimensional vector. The fully connected layer maps the flattened high-dimensional feature vector to the target action. Finally, the Softmax layer converts the output of the fully connected layer into a probability distribution for classification decision-making. The training set is used to train the human behavior recognition model, and the validation set is used to guide the model parameter optimization process until the training is completed.

[0100] The flattening layer, fully connected layer, and Softmax layer are introduced into the model as a whole, mainly to efficiently convert the extracted high-dimensional features into the final classification results. The combination of these three layers can solve the problems of information compression and dimension conversion in the process from feature extraction to classification mapping. The multi-dimensional feature tensor is converted into a one-dimensional vector through the flattening layer, the fully connected layer performs nonlinear mapping to learn the relationship between features and categories, and the Softmax layer converts the unnormalized scores into probability distributions. This overall module can not only better retain key feature information, but also reduce computational complexity by reducing dimensionality.

[0101] Among them, the flattening layer is used to flatten the multi-dimensional input tensor into a one-dimensional vector as follows:

[0102] H flatten =Flatten(H final )

[0103] Then use the fully connected layer to map the flattened high-dimensional feature vector to the ten target actions to obtain the unnormalized score H fc :

[0104] H fc =W fc ·H flatten +b fc :

[0105] Among them, W fc is the weight matrix, b fc is the bias vector, both of which are automatically learned during the training process.

[0106] Then the Softmax layer converts the output of the fully connected layer into a probability distribution for classification decision:

[0107]

[0108] Among them, P(y m |X) represents the probability that the input X belongs to the mth type of behavior, y m represents the mth type of behavior, C is the number of behavior categories, H fc,m is the unnormalized score of the mth class, Represents all categories of H fc Perform exponential summation.

[0109] Finally, for each sample, find the category with the highest probability in the Softmax output:

[0110]

[0111] Indicates the final category into which the sample is classified.

[0112] The confusion diagram of the final recognition result is as follows Figure 5 As shown, the recognition accuracy can reach 96%.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A method for indoor human behavior recognition based on CSI information, characterized in that: The method includes: Collect CSI signals of different indoor human behaviors through wireless routers and wireless network cards; Iteratively calculate the variance of the CSI signals received by each antenna of the wireless network card to eliminate the antenna that is least sensitive to activity; The processed CSI signal is denoised using discrete wavelet transform, the CSI signal is dimensionally reduced using principal component analysis, and a training set and a validation set are constructed based on the preprocessed data. Construct a human behavior recognition model based on BiLSTM and self-attention mechanism, where BiLSTM is used to extract time-related feature information in CSI signals, and the attention mechanism is used to highlight the information of key time steps and suppress redundant features; The model is trained using the training set, and the validation set is used to guide the model to optimize parameters until the training is completed; The CSI signals of human activities are collected and input into the trained human behavior recognition model to realize the recognition of human behavior.

2. The method according to claim 1, characterized in that The method of collecting CSI signals of different indoor human behaviors through a wireless router and a wireless network card includes building a simulated indoor environment, wherein the transmitting device is a wireless router and the receiving device is a computer equipped with a wireless network card; the collected human behaviors include standing, walking, jogging, changing clothes, cleaning, falling, sitting, squatting, bending over, and lying down, and the humans whose behaviors are collected include children, adult men, and the elderly.

3. The method according to claim 1, characterized in that The iterative variance calculation of the CSI signals received by each antenna of the wireless network card is performed, and the antenna least sensitive to the activity is eliminated. The data packet received by the i-th antenna of the wireless network card is expressed as: In the formula, represents the signal strength of the nth data packet received by the ith antenna; N represents the number of data packets received by the antenna in time series, that is, the sequence length; A sliding window of size Wz is set. The sliding window moves at a fixed step length on the signal sequence received by the i-th antenna. The variance of the signal in the sliding window is calculated each time it moves. After the movement is completed, a variance sequence is obtained. Since the signal in the inactive range tends to be stable, the variance in the sliding window is small, while the signal in the active range is unstable, the variance in the sliding window will become larger. The antennas sensitive to the target action have overlapping human behavior signal windows with signal fluctuations during the action implementation phase, while the antennas insensitive to the action have no overlapping signal fluctuation windows. The antennas that are least sensitive to the activity are identified and eliminated based on whether they have overlapping signal fluctuation windows.

4. The method according to claim 1, characterized in that: The constructed human behavior recognition model includes a BiLSTM, a self-attention module, a flattening layer, a fully connected layer and a Softmax layer connected in sequence, wherein the BiLSTM and the self-attention module are connected based on a residual network structure.

5. The method according to claim 1 or 4, characterized in that: Performing human behavior recognition based on the trained human behavior recognition model includes inputting the collected CSI signal of human activities into the model, and first extracting time-related feature information in the CSI signal through BiLSTM; Then, the output of each time step of BiLSTM is weighted and summed through the self-attention module to highlight the information of key time steps and suppress redundant features. Then, the output of BiLSTM and the output of the self-attention module are concatenated to retain the input information and fuse the attention-enhanced features. The multi-dimensional feature tensor is then converted into a one-dimensional vector through a flattening layer, and nonlinear mapping is performed through a fully connected layer to learn the relationship between features and categories; finally, the Softmax layer converts the output of the fully connected layer into a probability distribution for classification decisions.

6. The method according to claim 5, characterized in that The nonlinear mapping through the fully connected layer to learn the relationship between the feature and the category includes using the fully connected layer to map the flattened one-dimensional vector to a preset target action to obtain an unnormalized score H fc : H fc =W fc ·H flatten +b fc Where W fc represents the weight matrix, b fc represents the bias vector, H flatten A one-dimensional vector representing the output of the flatten layer; The unnormalized scores are converted into probability distributions through the Softmax layer: In the formula, P(y m |X) represents the probability that the input X belongs to the mth type of behavior, C represents the number of behavior categories, and H fc,m Represents the unnormalized score of the mth category behavior; find the category with the highest probability in the Softmax output: Indicates the final category to which the input X is classified.

Images