Lightweight user behavior recognition method based on multi-level feature extraction algorithm

Abstract

The invention discloses a lightweight user behavior recognition method based on a multi-level feature extraction algorithm. The lightweight user behavior recognition method comprises the following steps: collecting CSI (Channel State Information) original data; extracting a first time domain data set; obtaining a second time domain data set and a first frequency domain data set through a sample feature enhancement strategy; carrying out deep feature extraction, obtaining time domain sample-level features and frequency domain sample-level features, carrying out feature fusion, obtaining first fusion features, inputting the first fusion features into a bidirectional Mamba network, obtaining second fusion features, inputting the second fusion features into a multi-head attention mechanism, obtaining third fusion features, carrying out global average pooling, inputting the third fusion features into a classifier, and carrying out classification; outputting probability distribution of each action category; dynamically identifying and shielding low-contribution features in a model training process; through the sample feature enhancement strategy, the multi-level depth feature representation extraction strategy and the model redundancy feature lightweight strategy, the user behavior recognition precision is effectively improved, and the model complexity and the calculation cost are reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of human behavior recognition, and in particular to a lightweight user behavior recognition method based on a multi-level feature extraction algorithm. BACKGROUND

[0002] User behavior recognition is an important part of artificial intelligence in the fields of smart home, human-computer interaction, security monitoring and intelligent medical treatment. In the traditional sensing technology stage, researchers rely on cameras or wearable sensors to capture human motion features. Cameras can recognize user behavior through visual images, which has high accuracy and intuitiveness, but is easily affected by light, angle and occlusion, and has the risk of privacy leakage. Wearable sensors can directly capture human motion data and recognize fine-grained behavior features, but their use depends on the user wearing the device, which has the problems of high cost, inconvenience and easy to forget. Therefore, for the behavior recognition in people's daily life, there is an urgent need for a detection method without wearing, without light influence and low cost.

[0003] With the development of wireless technology and the wide popularity of WiFi devices, WiFi has become an ideal device in current wireless sensing due to its passive sensing, low cost, flexibility and ease of use. Channel state information (CSI) in WiFi signals can reflect the multipath effect and phase change of electromagnetic waves in the propagation process, and can indirectly represent the human motion state. Wi-Fi devices use multiple input multiple output (MIMO) and orthogonal frequency division multiplexing (OFDM) technology, so that CSI can capture high-resolution spatial and temporal features between different carriers and antennas. Compared with traditional sensing methods, CSI sensing does not require additional devices, does not invade privacy, can penetrate through obstacles, and has the advantages of low cost, non-contact and wide coverage.

[0004] However, the existing CSI-based behavior recognition model still has some deficiencies: first, the deep learning model has a large number of parameters and high computational complexity, which is not conducive to real-time deployment on edge devices; second, the traditional feature extraction method cannot fully exploit the complementary characteristics of CSI signals between time and frequency domains, and cannot effectively model the hierarchical association information between subcarriers, antenna pairs and overall samples, resulting in limited feature expression ability. SUMMARY

[0005] The purpose of the present application is to provide a lightweight user behavior recognition method based on a multi-level feature extraction algorithm, which solves the problems of the above-mentioned CSI-based behavior recognition model with large parameters and high computational complexity, which is not conducive to real-time deployment of edge devices, and the feature expression ability of the features extracted by the traditional feature extraction method is limited, and the user behavior recognition accuracy is low.

[0006] To achieve the above purpose, the present application provides a lightweight user behavior recognition method based on a multi-level feature extraction algorithm, comprising the following steps: Step 100, collecting CSI data of multiple users in multiple action categories to obtain an original data set; Step 200, extracting a first time domain data set from the original data set; Step 300, processing the first time domain data set through a sample feature enhancement strategy to obtain a second time domain data set and a first frequency domain data set; Step 400, using a deep feature extraction network in a multi-level deep feature representation extraction strategy to extract deep features from the second time domain data set and the first frequency domain data set respectively to obtain time domain sample level features and frequency domain sample level features; Step 500, using an adaptive weighted fusion method to fuse the time domain sample level features and the frequency domain sample level features to obtain first fusion features; Step 600, inputting the first fusion features into an encoder to obtain third fusion features; Step 700, inputting the third fusion features into a classifier after global average pooling to output the probability distribution of each action category; Using a total loss function for backpropagation, the deep feature extraction network, the encoder, the classifier and the trainable parameters are optimized; After training convergence, through a model redundant feature lightweight strategy, low contribution features are dynamically identified and shielded during the training process; After the model training is completed, the network recognition accuracy is calculated using a macro average calculation method, and the recognition accuracy is compared with the set threshold; if the recognition accuracy is greater than the set threshold, it is determined that the performance meets the sensing requirement, and the final user behavior recognition model is obtained, otherwise the performance does not meet the sensing requirement, and when the performance does not meet the sensing requirement, the number of iterations of the loss function is increased to rejudge until the performance meets the sensing requirement.

[0007] Further, the calculation formula of the total loss function is as follows: ; ; Wherein represents the loss function, is the total loss function, a label representing a sample corresponding to the jth action, a probability predicted as the jth action, and respectively, F is the number of subcarriers, S is the number of antennas, is the number of action categories, a sequence mask representing the ith subcarrier, a sequence mask representing the jth antenna.

[0008] Further, the method for obtaining the second time domain data set and the first frequency domain data set by performing sample feature enhancement strategy on the first time domain data set comprises: Hampel filtering is performed on the first time domain data set to remove outliers that do not conform to the overall trend; Butterworth low-pass filtering is performed on the first time domain data set to remove high-frequency noise and perform smoothing processing; For the first time domain data set, linear interpolation method is used to complete the missing values according to the average trend of the previous and subsequent data; Data regularization processing is performed on the first time domain data set to obtain the second time domain data set; Short-time Fourier transform processing is performed on the obtained second time domain data set to obtain the first frequency domain data set.

[0009] Further, the multi-level deep feature representation extraction strategy is a double-branch input structure, including a time domain branch and a frequency domain branch, the two branches are respectively used for processing time domain and frequency domain signal features, the time domain branch receives the second time domain data set, the frequency domain branch receives the first frequency domain data set, and the two branches perform feature extraction in parallel.

[0010] Further, the time domain branch and the frequency domain branch in the multi-level deep feature representation extraction strategy both adopt a deep-level feature extraction network, and the deep-level feature extraction network comprises three layers of separable convolution networks, specifically: The first layer of the deep-level feature extraction network focuses on subcarrier-level feature extraction, and each subcarrier signal is independently input into the first layer of the deep-level feature extraction network to realize feature extraction, and a subcarrier feature sequence is output

[0011] The second layer of the deep-level feature extraction network is used for aggregating subcarrier features belonging to the same transmit-receive antenna pair, and the subcarrier feature sequence feature extraction, and an antenna pair feature is output

[0012] The third layer of the deep-level feature extraction network is oriented to sample-level feature extraction, and the antenna pair features are spliced into a whole and subjected to feature extraction, and time domain sample-level features and frequency domain sample-level features are output; A nonlinear activation function is used after the deep feature extraction network of each layer.

[0013] Further, the time domain sample level feature and the frequency domain sample level feature are fused by an adaptive weighted fusion method to obtain a first fused feature, and a calculation formula of the first fused feature is as follows: ; Wherein is the first fused feature, and are first and second feature fusion parameters, and are trainable parameters, is a time domain sample level feature, is a frequency domain sample level feature.

[0014] Further, the first fused feature is input into an encoder to obtain a third fused feature, and the method is as follows: the first fused feature is input into a bidirectional Mamba network to obtain a second fused feature, and the second fused feature is obtained through a multi-head attention mechanism to obtain the third fused feature.

[0015] Further, the step of outputting the probability distribution of each action category is as follows: The third fused feature is input into a classifier after global average pooling, the classifier outputs respective scores of M classes of actions, and the probability distribution of the action category is output through Softmax activation.

[0016] Further, through a model redundant feature lightening strategy, the method for dynamically identifying and shielding low contribution features in the training process is as follows: In the training process, the trainable scalar corresponding to the subcarrier calculates the sequence mask of the subcarrier through the Sigmoid function, if the sequence mask of the subcarrier is 0, it is judged as low contribution, and the subcarrier level feature of low contribution is automatically inhibited; the trainable scalar corresponding to the antenna calculates the sequence mask of the antenna through the Sigmoid function, if the sequence mask of the antenna is 0, it is judged as low contribution, and the antenna level feature of low contribution is inhibited.

[0017] Therefore, the application adopts the above-mentioned light weight user behavior recognition method based on a multi-level feature extraction algorithm, and has the following beneficial effects: 1. The sample feature enhancement strategy is used to enhance the sample feature in advance before training, the time domain includes outlier removal, high frequency noise removal, linear interpolation and data regularization, and the frequency domain includes short time Fourier transform; 2. The subcarrier level convolution network, the antenna level convolution network and the sample level convolution network are used to gradually extract multi-level deep feature representations, and the multi-scale feature correlation of CSI is fully mined from local to global; 3. The fused sample features are further refined and modeled using a lightweight bidirectional Mamba module combined with a multi-head attention mechanism to enhance the temporal correlation and global expression ability of the features; 4. A model redundancy feature lightweight strategy is introduced, which assigns trainable masks to the subcarrier-level and antenna-level redundant features to adaptively identify and shield low-contribution features, thereby further enhancing the effectiveness of feature expression and the lightweight performance of the model.

[0018] The technical solutions of the present application will be further described in detail below with the aid of the accompanying drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0019] Fig. 1 The technical framework diagram of the lightweight user behavior recognition method based on the multi-level feature extraction algorithm of the present application; Fig. 2 The flowchart of the model training process of the lightweight user behavior recognition method based on the multi-level feature extraction algorithm of the present application. DETAILED DESCRIPTION

[0020] The detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but only represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative labor are within the scope of protection of the present application.

[0021] Please refer to Figs. 1-2 A lightweight user behavior recognition method based on a multi-level feature extraction algorithm, comprising the following steps: Step 100, collecting CSI data of multiple users of multiple action categories to obtain an original data set ; In an embodiment of the present application, a total of a·b independent propagation routes are used in the environment using a transmitting antenna and b receiving antennas, and the original data set of n users of s action categories is collected in the environment , The data set corresponding label set; Step 200, extracting a first time domain data set from the original data set ; Step 300, processing the first time domain data set by a sample feature enhancement strategy to obtain a second time domain data set and a first frequency domain data set ; The sample feature enhancement strategy for processing the first time domain data set includes outlier removal, high-frequency noise removal, linear interpolation, and data regularization.

[0022] Specifically, in the time domain data, due to device error or environmental interference, there may be value mutations or abnormal outliers. For this reason, first, Hampel filtering is used to remove abnormal values that do not conform to the overall trend. Considering that human behavior signals are mainly distributed in the low frequency band, in order to suppress high frequency noise and highlight human behavior characteristics, further Butterworth low-pass filtering is used to remove high frequency noise and perform smoothing processing. In addition, due to device acquisition defects or abnormal point rejection, the data may appear discontinuous. For missing data, linear interpolation method is used to complete the missing values according to the average trend of the previous and subsequent data. Finally, the dimensions and numerical ranges of different features are different, which is easy to cause weak features to be covered, so the sample is subjected to data regularization processing to balance the influence of each feature. After the above sample feature enhancement strategy processing, the sample features can be enhanced, the problem of insufficient feature extraction capability of the subsequent lightweight module can be compensated, and a second time domain data set with higher quality and stronger continuity is obtained ; Each sample in the second time domain data set is processed by short-time Fourier transform to perform time-frequency analysis.

[0023] ; wherein is a window function, Hamming window is selected, and is defined as: ; wherein N is the window length, and through the transformation, a first frequency domain data set is obtained, which converts one-dimensional time sequence signal into two-dimensional time-frequency image representation; Step 400, the deep feature extraction network in the multi-level deep feature representation extraction strategy is used to respectively extract deep features from the second time domain data set and the first frequency domain data set to obtain time domain sample level features and frequency domain sample level features ; The multi-level deep feature representation extraction strategy is a double-branch input structure, and two branches are used to process time domain and frequency domain signal features respectively. In the structure, the time domain branch receives the second time domain data set , the frequency domain branch receives the first frequency domain data set , and both of them perform feature extraction in parallel. Each branch uses a deep feature extraction network, and the deep feature extraction network includes three separable convolution networks, specifically: The first layer of the deep feature extraction network focuses on subcarrier-level feature extraction, which is used to capture the amplitude difference between different subcarriers. Specifically, each subcarrier signal is input into the first layer of the deep feature extraction network for feature extraction, and the convolution kernel size is 1 6, the output channel number is 64, and the output subcarrier feature sequence is output .

[0024] The second layer of the deep feature extraction network focuses on antenna-level feature modeling to learn the spatial correlation between multiple antennas. In an embodiment of the present application, 30 subcarrier features belonging to the same transmit-receive antenna pair are aggregated, and feature aggregation is performed in the subcarrier dimension. Specifically, a convolution kernel with a size of 64 4, the output channel number is 128, and the subcarrier feature sequence is convolved to extract the local correlation pattern of each antenna pair. After the convolution layer, 6 groups of antenna pair features are output, denoted as . .

[0025] The third layer of the deep feature extraction network is oriented towards sample-level feature extraction, which captures the overall temporal variation and environmental dynamic pattern. In an embodiment of the present application, the 6 groups of antenna pair features are concatenated as the overall input, and a convolution kernel with a size of 128 3, the output channel number is 128, and the global fusion is realized by a convolution layer with a step of 3 to output the time-domain sample-level feature .

[0026] In an embodiment of the present application, a ReLU activation function is used after each separable convolution layer to enhance the non-linear expression ability of the network and accelerate the convergence.

[0027] Similarly, in the frequency domain branch, the same three-layer separable convolution network deep feature extraction network is used to perform layer-by-layer feature extraction on the frequency domain data, and finally the frequency domain sample-level feature .

[0028] In step 500, the time-domain sample-level feature and the frequency domain sample-level feature are fused by an adaptive weighted fusion method to obtain the first fusion feature . The calculation formula of the first fusion feature is as follows: . wherein is the first fusion feature, and respectively, are trainable parameters, used to dynamically adjust the contribution ratio of time domain and frequency domain features in the fusion process, so as to realize adaptive balance at the feature level.

[0029] Step 600, input the first fusion feature into the encoder to obtain the third fusion feature, specifically: input the first fusion feature into the bidirectional Mamba network to obtain the second fusion feature, the first fusion feature is realized by two independent selection state space branches of the forward Mamba and the backward Mamba. ; ; The state equation and the observation equation of the backward branch are: ; ; wherein , respectively represent the first internal representation and the second internal representation at time t, , respectively represent the output of the forward scanning block and the reverse scanning block at time t, A, B and C are linearly changed by the input sample.

[0030] The forward and backward observation outputs are fused through a learnable linear fusion matrix to obtain the second fusion feature, and the calculation formula is: ; , respectively represent the first linear fusion matrix and the second linear fusion matrix, represents the second fusion feature.

[0031] The second fusion feature obtains the third fusion feature through a multi-head attention mechanism , and the specific calculation formula is: ; wherein, , , ∈ respectively are trainable parameter matrices of the query vector Q, the key vector K and the value vector V, is the dimension of the input feature, is the projection dimension; Attention weights are calculated based on the query vector Q and the key vector K, and the value vector V is weighted and summed using the attention weights to generate the third fusion feature , and the calculation formula is:​ ; wherein is a function.

[0032] By means of the bidirectional information interaction mechanism, the parameter size is effectively reduced while maintaining the model expression capability, so as to realize the overall lightweight of the model.

[0033] In step 700, the third fusion feature is input into a classifier after global average pooling, and the probability distribution of each action category is output. The classifier includes two fully connected layers, specifically: the third fusion feature is input into the first layer of the classifier after global average pooling, and the 128-dimensional intermediate feature is output through ReLU activation; the intermediate feature is input into the second layer of the classifier, and the intermediate feature is mapped to dimensional space, and the respective scores of the action categories are output. The probability distribution of the action categories is output through Softmax activation, and the calculation formula is as follows: ; wherein is the probability of being predicted as the i-th action, is the score of the i-th action, is the score of the j-th action, is the number of action categories, is an exponential function.

[0034] By means of the model redundant feature lightweight strategy, low-contribution features are dynamically identified and shielded in the training process, so as to realize the lightweight of the features at the subcarrier level and the antenna level.

[0035] In the embodiment of the application, for the subcarrier level sample, each input subcarrier corresponds to a trainable scalar , and a sequence mask of 0, 1 is obtained through Sigmoid , and the sequence mask is used to automatically shield the low-contribution subcarrier features in the training process, so as to realize adaptive selection at the feature level.

[0036] For the antenna level sample, each input antenna corresponds to a trainable scalar , and a sequence mask of 0, 1 is obtained through Sigmoid , and the sequence mask can adaptively identify and shield the low-contribution antenna features in the training process, so as to further realize the lightweight and feature optimization in the spatial dimension.

[0037] It should be noted that if the sequence mask of the subcarrier is 0, it is judged to be low contribution, and if the sequence mask of the subcarrier is 1, it is judged to be high contribution.

[0038] In the embodiment of the application, the loss function is used for back propagation to optimize the deep feature extraction network, the encoder, the classifier and the trainable parameters, and the calculation formula of the loss function is as follows: ; Among them represents the loss function, represents the label of the sample corresponding to the jth action, represents the probability of predicting the jth action, is the number of action categories.

[0039] Further, L1 sparse regularization is added to the loss function to promote the sparsity of the mask, and the total loss function is obtained, and the calculation formula is as follows: ; is the total loss function, and are the subcarrier coefficients and the antenna sparse coefficients respectively, F is the number of subcarriers, is the number of antennas, represents the sequence mask of the ith subcarrier, represents the sequence mask of the jth antenna.

[0040] After training convergence, the low-contribution subcarrier and antenna mask are 0, which can be directly shielded or pruned to obtain a lightweight model and realize lightweight at the training level.

[0041] After the model training is completed, the network recognition accuracy is calculated by using the macro average calculation method, and the recognition accuracy is compared with the set threshold; if the recognition accuracy is greater than the set threshold, it is determined that the performance meets the sensing requirement, and the final user behavior recognition model is obtained, otherwise the performance does not meet the sensing requirement, and the number of iterations of the loss function is increased to rejudge until the performance meets the sensing requirement.

[0042] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application but not to limit them, although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can still be modified or replaced by equivalents, and these modifications or replacements cannot make the modified technical solutions deviate from the spirit and scope of the technical solutions of the present application.

Claims

1. A lightweight user behavior recognition method based on a multi-level feature extraction algorithm, characterized in that, The method comprises the following steps: Step 100, collecting CSI data of multiple users in multiple action categories to obtain an original data set; Step 200, extracting a first time domain data set from the original data set; Step 300, processing the first time domain data set through a sample feature enhancement strategy to obtain a second time domain data set and a first frequency domain data set; Step 400, performing deep feature extraction on the second time domain data set and the first frequency domain data set respectively by using a deep feature extraction network in a multi-level deep feature representation extraction strategy to obtain time domain sample level features and frequency domain sample level features; Step 500, performing feature fusion on the time domain sample level features and the frequency domain sample level features by using an adaptive weighted fusion method to obtain first fused features; Step 600, inputting the first fused features into an encoder to obtain third fused features; Step 700, inputting the third fused features into a classifier after global average pooling to output a probability distribution of each action category; Performing back propagation by using a total loss function to optimize the deep feature extraction network, the encoder, the classifier and trainable parameters; After training convergence, dynamically identifying and shielding low-contribution features in the training process by using a model redundant feature lightweight strategy; After the model training is completed, calculating a network recognition accuracy by using a macro average calculation method, comparing the recognition accuracy with a set threshold, determining that the performance meets the sensing requirement if the recognition accuracy is greater than the set threshold, obtaining a final user behavior recognition model, otherwise, determining that the performance does not meet the sensing requirement, and increasing the iteration number of the loss function to re-determine until the performance meets the sensing requirement. 2.The lightweight user behavior recognition method based on multi-level feature extraction algorithm according to claim 1, characterized in that, The calculation formula of the total loss function is as follows: ； ； wherein represents a loss function, is a total loss function, represents the label of the sample corresponding to the j-th action, represents the probability of predicting the j-th action, and are subcarrier coefficients and antenna sparse coefficients, respectively, F is the number of subcarriers, and S is the number of antennas, is the number of action categories, represents the sequence mask of the i-th subcarrier, represents the sequence mask of the j-th antenna. 3.The lightweight user behavior recognition method based on multi-level feature extraction algorithm of claim 1, wherein, The method for processing the first time domain data set through the sample feature enhancement strategy to obtain the second time domain data set and the first frequency domain data set comprises: Removing abnormal values that do not conform to the overall trend by using Hampel filtering on the first time domain data set; Removing high-frequency noise by using Butterworth low-pass filtering on the first time domain data set to perform smoothing processing; For missing data, using a linear interpolation method to complete the missing values according to the average trend of the previous and subsequent data; Performing data regularization processing on the first time domain data set to obtain the second time domain data set; Processing the obtained second time domain data set by using short-time Fourier transform to obtain the first frequency domain data set. 4.The lightweight user behavior recognition method based on multi-level feature extraction algorithm of claim 1, wherein, The multi-level deep feature representation extraction strategy is a double-branch input structure comprising a time domain branch and a frequency domain branch, and the two branches are respectively used for processing signal features in the time domain and the frequency domain. The time domain branch receives the second time domain data set, and the frequency domain branch receives the first frequency domain data set. The two branches perform feature extraction in parallel.

5. The lightweight user behavior recognition method based on multi-level feature extraction algorithm according to claim 4, characterized in that, Both the time domain branch and the frequency domain branch in the multi-level deep feature representation extraction strategy adopt a deep feature extraction network. The deep feature extraction network comprises three separable convolution networks, and specifically comprises: The first layer of the deep feature extraction network focuses on subcarrier level feature extraction. Each subcarrier signal is independently input into the first layer of the deep feature extraction network to realize feature extraction, and a subcarrier feature sequence is output. The second layer of the deep feature extraction network is configured to aggregate subcarrier features belonging to the same transmit-receive antenna pair, and output an antenna pair feature sequence feature extraction, output an antenna pair feature The third layer of the deep feature extraction network faces sample-level feature extraction, concatenates antenna pair features into a whole, and extracts features therefrom to output time-domain sample-level features and frequency-domain sample-level features; A nonlinear activation function is used after the deep feature extraction network of each layer.

6. The lightweight user behavior recognition method based on multi-level feature extraction algorithm according to claim 1, characterized in that, The time-domain sample-level features and the frequency-domain sample-level features are fused by an adaptive weighted fusion method to obtain first fused features, and a calculation formula of the first fused features is as follows: ； wherein is a first fused feature, and are first and second feature fusion parameters, respectively, both being trainable parameters, is a time domain sample level feature, is a frequency domain sample level feature.

7. The lightweight user behavior recognition method based on multi-level feature extraction algorithm according to claim 1, characterized in that, The first fused features are input into an encoder to obtain third fused features, the method being that the first fused features are input into a bidirectional Mamba network to obtain second fused features, and the second fused features are used to obtain the third fused features through a multi-head attention mechanism. 8.The light-weight user behavior recognition method based on multi-level feature extraction algorithm of claim 1, wherein, The steps of outputting a probability distribution of each action category are as follows: The third fused features are input into a classifier after global average pooling, the classifier outputs respective scores of M action categories, and a probability distribution of the action category is output through Softmax activation. 9.The light-weight user behavior recognition method based on multi-level feature extraction algorithm of claim 1, wherein, In the training process, the method of dynamically identifying and shielding low-contribution features through a model redundant feature lightweight strategy is as follows: In the training process, a trainable scalar corresponding to a subcarrier is used to calculate a sequence mask of the subcarrier through a Sigmoid function, if the sequence mask of the subcarrier is 0, it is determined that the subcarrier is of low contribution, and the subcarrier-level feature of low contribution is automatically inhibited; A trainable scalar corresponding to an antenna is used to calculate a sequence mask of the antenna through a Sigmoid function, if the sequence mask of the antenna is 0, it is determined that the antenna is of low contribution, and the antenna-level feature of low contribution is inhibited.

Images