A method for gait recognition based on WIFI device

Abstract

The application discloses a gait recognition method based on a WIFI device, and comprises the following steps: collecting CSI of each user when walking in a sensing area of a WIFI propagation link formed by TX and RX included in the WIFI device; performing data preprocessing on the CSI to obtain a corresponding spectrum graph; performing data enhancement processing on marked data in the spectrum graph, performing feature extraction based on the marked data after the enhancement processing and non-marked data in the spectrum graph, learning user gait features independent of non-gait behaviors and paths according to a result of the feature extraction, and performing gait recognition on the user based on the user gait features. The implementation of the application can realize non-contact gait recognition, and has the advantages of low implementation cost and high recognition accuracy.

Description

Technical Field

[0001] This invention relates to the field of WIFI sensing technology in the Internet of Things, and in particular to a method for gait recognition based on WIFI devices. Background Technology

[0002] Human body recognition technology has become a key technology in many applications, such as security management and personalized services in smart spaces. Currently, human body recognition technologies are implemented using various methods, including video-based methods, acoustic signal methods, and wearable devices. Meanwhile, Wi-Fi-based gait recognition technology has also received widespread attention in recent years due to its advantages of universality, non-contact, and non-invasiveness.

[0003] The basic principle of Wi-Fi-based gait recognition technology includes: a person's walking movements disturb Wi-Fi signals, such as channel state information (CSI), and each person's natural gait is unique. These differences in human limb movement patterns and speeds are used as characteristics of each individual's gait, thereby enabling personalized user identification technology. In other words, using Wi-Fi devices for accurate and interference-free human body recognition can provide more opportunities for many personalized services in smart spaces.

[0004] However, current Wi-Fi-based gait recognition technologies face the problem of inaccurate gait recognition in practical deployments. In other words, there is currently no optimal Wi-Fi-based gait recognition technology solution that can meet the application needs of gait recognition in various technical fields. Summary of the Invention

[0005] The purpose of this invention is to provide a method for gait recognition based on WIFI devices, so as to meet the application needs of human gait recognition in various technical fields and solve the problems existing in the prior art.

[0006] The objective of this invention is achieved through the following technical solution:

[0007] A method for gait recognition based on a WIFI device includes:

[0008] In the sensing area where the WIFI propagation link consisting of the transmitter (TX) and receiver (RX) of the WIFI device is located, the channel state information (CSI) of each user is collected when walking in the sensing area.

[0009] The CSI is preprocessed to obtain the corresponding spectrum; the data preprocessing includes removing environmental noise and automatic gain control (AGC) noise from the CSI.

[0010] Data augmentation is performed on the labeled data in the spectrogram. Feature extraction is performed on the augmented labeled data and the unlabeled data in the spectrogram based on a three-layer stacked convolutional neural network (CNN) and a three-layer stacked long short-term memory (LSTM). The augmented labeled data includes expanded data on potential behaviors and paths.

[0011] Based on the results of feature extraction, user gait features independent of non-gait behaviors and paths are learned, and based on the learned user gait features, gait recognition is performed on users walking in the sensing area.

[0012] The process of removing environmental noise and AGC noise from the CSI includes:

[0013] The environmental noise is removed using a principal component analysis method based on subcarriers; and,

[0014] The AGC noise removal process includes: using spatial clustering to cluster AGC-related points according to their sparse distribution density, and retaining only the cluster with the largest number of scattered points to filter out the AGC noise.

[0015] The method also includes:

[0016] The CSI data after preprocessing is subjected to a Fourier Transform (STFT) to obtain the corresponding spectrum, and the spectrum is enhanced to obtain an enhanced spectrum, which is used as the spectrum obtained from the data preprocessing.

[0017] The process of obtaining the enhanced spectrum includes:

[0018] A sliding window is used to slice the time series CSI data, and STFT is performed on each slice;

[0019] The amplitudes of the spectrum maps corresponding to multiple subcarriers are added together to obtain a superimposed spectrum map, and only the spectrum map of the predetermined frequency band range is retained;

[0020] After normalizing each FFT block contained in the spectrum, background noise is eliminated by subtracting the average amplitude of the entire spectrum, and an enhanced spectrum is obtained by processing with a two-dimensional Gaussian filter.

[0021] The process of performing data augmentation on the labeled data in the spectrogram includes: training an adversarial autoencoder (AAE) on the labeled gait data of each user in the labeled data, wherein the training process includes:

[0022] The data of the kth AAE The latent vector z is generated by feeding it into the encoder of AAE#k; where AAE#k refers to the AAE numbered k.

[0023] The latent vector z and the user's label y are sent to the decoder to generate a vector for reconstructing the labeled gait data. In the process of reconstructing labeled gait data, the mean squared error (MSE) is used to define the reconstruction loss, and the corresponding loss function is:

[0024] ;

[0025] in, It is the number of samples;

[0026] Based on the loss function and the vector of the reconstructed marked gait data Data reconstruction is performed to obtain reconstructed gait data related to the user corresponding to the user's label y. This reconstructed gait data includes expanded potential behavior and path data.

[0027] as well as,

[0028] The encoder is used as input data to the Generative Adversarial Network (GAN) generator. The data, along with a vector z' following a prior distribution, is input into the discriminator to standardize the reconstructed data through training. Cross-entropy is used as the loss function of the discriminator, and the loss function of the discriminator is:

[0029] ;

[0030] For the discriminator, when using When used as input, tags The value is 0 when z' is used as input; the label is... =1, It is the predicted value of the i-th sample. It is the true value of the i-th sample.

[0031] The process of extracting feature vectors includes:

[0032] In each layer of the CNN, a convolutional layer with a 2D convolutional kernel is used, a batch normalization layer is used to accelerate the processing, a modified linear unit (ReLU) is inserted to introduce nonlinearity, and a max pooling layer is used to reduce the size of the feature vector.

[0033] as well as,

[0034] LSTM learns the temporal dynamic features extracted by CNN during time series data processing; each LSTM layer has the same number of neurons and uses the Sigmoid activation function to extract data.

[0035] The process of learning user gait features independent of non-gait behaviors and paths based on the results of feature extraction includes:

[0036] The extracted feature vector is learned using three fully connected layers and the ReLU activation function, and then passed through the output layer of the softmax activation function to obtain the gait prediction probability vector. Specifically, cross-entropy is used as the loss function for gait classification, and the gait prediction probability vector is obtained based on the gait classification loss function. The prediction probability vector includes the predicted label for gait classification.

[0037] The feature vector is concatenated with the predicted label for gait classification, the posterior distributions of the source and target domains are aligned, and they are input together into the domain discriminator to predict the domain label. Conditional entropy is used as the loss function for target domain classification, thereby determining the loss function of the domain discriminator. The source domain is the labeled data domain, and the target domain is the unlabeled data domain.

[0038] The overall loss function of the training model is determined based on the loss function of the gait classification and the loss function of the domain discriminator. The training model is then trained using all labeled and unlabeled data based on the overall loss function of the training model, and the parameters are iteratively updated during the training process to obtain a training model that contains user gait features that are independent of non-gait behaviors and paths.

[0039] The loss function for gait classification is:

[0040] , ;

[0041] in, and K represents the number of labeled and unlabeled data points used for training, and K is the total number of users. and Let represent the user labels for the predicted labeled and unlabeled data, respectively; La is the loss function for the labeled data, and Lu is the loss function for the unlabeled data.

[0042] The loss function of the domain discriminator is:

[0043] ;

[0044] ,;

[0045] ;

[0046] in, and These represent the total amount of data used for training and the amount of data in the target domain, where D is the number of domains. Let be the true domain label of the i-th sample. Let be the predicted domain label for the i-th sample. The predicted domain label for the i-th sample when aligning the posterior distributions of the source and target domains. The predicted domain label for the i-th sample when adding classification constraints to the target domain. The predicted label of the target domain for the i-th sample when adding classification constraints to the target domain; Ld is the loss function for all domains, Lo is the loss function for predicting all domains when aligning the posterior distributions of the source and target domains, and Lt is the loss function for predicting all target domains when adding classification constraints to the target domain.

[0047] The overall loss function of the model is:

[0048] ;

[0049] Here, α, β, γ, and λ are hyperparameters, and their values ​​are adjusted based on the training results.

[0050] In this invention, during backpropagation, the loss function of the domain discriminator is negative, while the loss function of the gait classification is positive.

[0051] Compared with the prior art, the present invention provides a gait recognition method based on WIFI devices, which can overcome the limitations of existing WIFI-based gait recognition schemes. In the process of application, it can achieve robustness to user behavior and walking path, thereby achieving high gait recognition accuracy. Application experiments of the present invention in real scenarios show that it can achieve high-precision gait recognition measurement in the gait recognition process. Attached Figure Description

[0052] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 A block diagram illustrating the implementation principle of the method provided in this embodiment of the invention;

[0054] Figure 2 The data preprocessing procedure provided in this embodiment of the invention obtains fine-grained gait spectrograms from CSI;

[0055] Figure 3 This is a schematic diagram of an AEE-based data enhancement architecture provided in an embodiment of the present invention;

[0056] Figure 4A schematic diagram of a domain-adaptive gait recognition deep learning framework provided in an embodiment of the present invention;

[0057] Figure 5 This is a schematic diagram illustrating the prediction accuracy experiment results for three different users using four different methods, provided as an embodiment of the present invention. Detailed Implementation

[0058] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the specific content of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments, which do not constitute a limitation of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0059] First, the following explanations are provided for the terms that may be used in this article:

[0060] The term "and / or" means that either or both can be achieved simultaneously. For example, X and / or Y means that it includes both "X" or "Y" as well as the three cases of "X and Y".

[0061] The terms "comprising," "including," "containing," "having," or other similar semantic descriptions should be interpreted as non-exclusive inclusion. For example, including a technical feature element (such as raw material, component, ingredient, carrier, dosage form, material, size, part, component, mechanism, device, step, process, method, reaction conditions, processing conditions, parameter, algorithm, signal, data, product or article of manufacture, etc.) should be interpreted as including not only the expressly listed technical feature element, but also other technical feature elements that are not expressly listed and are well-known in the art.

[0062] The term "composed of" excludes any technical features not expressly listed. When used in a claim, it closes the claim to exclude all technical features other than those expressly listed, except for associated conventional impurities. If the term appears only in a clause of a claim, it limits the claim to the elements expressly listed in that clause; elements recited in other clauses are not excluded from the overall claim.

[0063] Unless otherwise explicitly specified or limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this document according to the specific circumstances.

[0064] When concentration, temperature, pressure, size, or other parameters are expressed as numerical ranges, such ranges should be understood to specifically disclose all ranges formed by any pairing of upper limits, lower limits, or preferred values ​​within that range, regardless of whether the range is explicitly stated; for example, if the numerical range "2 to 8" is stated, then that range should be interpreted to include ranges such as "2 to 7", "2 to 6", "5 to 7", "3 to 4 and 6 to 7", "3 to 5 and 7", "2 and 5 to 7", etc. Unless otherwise stated, the numerical ranges described herein include both their endpoints and all integers and fractions within that range.

[0065] The terms “center,” “longitudinal,” “lateral,” “length,” “width,” “thickness,” “upper,” “lower,” “front,” “back,” “left,” “right,” “vertical,” “horizontal,” “top,” “bottom,” “inner,” “outer,” “clockwise,” and “counterclockwise” indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience and simplification of description and do not imply that the device or component referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this document.

[0066] In realizing this invention, the major problems faced by Wi-Fi-based human gait recognition technology in practical deployment were first analyzed in order to provide a better Wi-Fi-based human gait recognition technology solution. Specifically, the major problems identified in the analysis mainly include:

[0067] (1) Current WIFI-based human gait recognition systems are based on the assumption that users’ walking patterns are periodic. However, in real-world scenarios, users not only move their legs while walking, but may also engage in various other non-periodic human behaviors (such as turning, bending over, and carrying luggage); moreover, people sometimes do not walk continuously, but may walk and stop intermittently; these non-periodic and discontinuous behaviors make it impossible to fully extract unique gait-related features for scenarios that require long-term recognition, thus making it impossible for WIFI-based human gait recognition systems to accurately recognize human gait.

[0068] (2) Most Wi-Fi-based gait recognition schemes utilize CSI to extract gait-related features. However, since CSI represents fine-grained channel characteristics, it is extremely sensitive to human behavior. This means that after training on data of specific user behaviors, other different human behaviors can alter the propagation characteristics of the Wi-Fi signal. If these effects are not eliminated, the accuracy of gait recognition may be severely reduced.

[0069] (3) In theory, if enough CSI data can be collected from all possible behaviors of each user, a gait recognition model robust to human behavior can be trained; however, considering the diversity and richness of human behavior and the influence of various walking paths on gait patterns, such processing would be extremely labor-intensive. More importantly, many non-gait human behaviors and walking paths may not be taken into account, making the solution impractical in real-world scenarios.

[0070] (4) Combining human behavior recognition solutions with gait recognition systems is also a possible solution, but the implementation of this solution depends on accurately detecting each human behavior and separating it from the hybrid CSI in order to eliminate the influence of human behavior on gait characteristics. Moreover, this solution still requires the collection of data on all possible human behaviors, and signal separation is also a challenging problem.

[0071] (5) If the influence of walking path is eliminated by estimating the user's walking direction, multiple transceivers need to be strictly set up, and the user needs to walk normally and continuously in a specific area. Obviously, these settings are cumbersome for many scenarios and users, hindering the widespread application of gait recognition technology.

[0072] After analyzing the various problems mentioned above, the present invention provides the following specific implementation scheme, namely, a better WIFI-based human gait recognition technology scheme, which can meet the application needs of human gait recognition in various technical fields.

[0073] Specifically, the objective of this invention is to design a novel gait recognition method based on Wi-Fi devices. This method can identify users who exhibit additional discontinuous behaviors and walk along different paths, while ensuring that the objective is achieved by collecting and labeling only a small amount of CSI data. Specifically, this invention includes a complete set of data processing and recognition algorithms, which may include: processing Wi-Fi channel state information, including principal component analysis at the subcarrier level, AGC noise removal, and spectrogram enhancement; and extracting gait features to predict human identity information, i.e., user identity information, when observing rich unknown discontinuous behaviors and walking paths.

[0074] The technical solution of this invention primarily involves obtaining a refined gait pattern spectrogram through meticulous data preprocessing techniques. Simultaneously, it combines domain adaptation and data augmentation techniques to reduce the coupling between gait patterns and discontinuous behaviors and walking paths, generating rich user-labeled data for training the network model. This allows for subsequent gait recognition operations independent of discontinuous behaviors and walking paths. Specifically, within the corresponding deep learning framework based on domain adaptation and data augmentation, domain adaptation techniques are used to reduce the coupling between user gait patterns and discontinuous behaviors and walking paths. This treats different behaviors and paths as different domains, enabling the technical solution provided by this invention to learn domain-independent gait features. Furthermore, data augmentation methods can be used within this framework to effectively process a wide variety of rich, unlabeled user gait data for different behaviors and paths using only a small amount of user-labeled data.

[0075] The specific implementation of the present invention will now be described in detail with reference to the accompanying drawings.

[0076] like Figure 1 As shown, this invention specifically involves placing one Tx and one Rx in physical space to form a WIFI propagation link, thereby constructing a gait sensing area, i.e., a sensing area. Further, the implementation process of this invention may specifically include:

[0077] (1) Collect the CSI of each subject when walking in the sensing area and input it into the data preprocessing module. Perform data preprocessing through the data preprocessing module to obtain the corresponding spectrogram; wherein, the data preprocessing may include removing environmental noise and AGC (automatic gain control) noise from the CSI.

[0078] (2) Perform data augmentation processing on the labeled data in the spectrogram, that is, input a small portion of the data with user IDs (i.e. known behaviors and paths) in the spectrogram into the data augmentation module for data augmentation processing, so as to expand and obtain more potential human behaviors and walking paths;

[0079] (3) Input the enhanced labeled data (source domain data) and unlabeled data (target domain data) into cross-domain gait recognition to learn gait features independent of non-gait human behavior and walking paths;

[0080] Specifically, feature extraction is performed based on the enhanced labeled data and the unlabeled data in the spectrogram. The enhanced labeled data includes expanded potential behavior and path data, that is, more possible user behavior and path data are obtained by expanding a limited amount of labeled data.

[0081] Based on the results of feature extraction, learn user gait features that are independent of non-gait behaviors and paths.

[0082] (4) Based on the user gait features obtained from the learning, gait recognition can be performed on users walking in the sensing area.

[0083] Reference Figure 1 As shown, in the specific implementation process of the present invention, it may include, but is not limited to, the following processing steps: data preprocessing, data augmentation processing, and cross-domain gait recognition processing. The specific implementation process of each processing step will be described in detail below with reference to the accompanying drawings.

[0084] (a) Data preprocessing

[0085] Research has found that the raw CSI collected using a WIFI network card contains a lot of noise information that affects gait recognition, generally manifested as environmental noise and AGC noise, and the corresponding environmental noise follows a Gaussian distribution throughout the space.

[0086] Specifically, in the setup of this invention, a Wi-Fi transceiver is used to continuously identify the user's gait throughout their walking process. When the user moves away from the transceiver or walks in a direction nearly parallel to the propagation path, the gait pattern may be overwhelmed by noise. In this case, traditional denoising methods (such as low-pass and wavelet denoising) perform poorly. Furthermore, the effectiveness of traditional principal component analysis (PCA) denoising in the time dimension depends on the number of principal components chosen. Choosing more or fewer components may result in retaining environmental noise or losing important target signals. When dynamic signals are masked by environmental noise, they lose their effectiveness because the principal components do not clearly correspond to the dynamic signals.

[0087] For the reasons mentioned above, this invention shifts its focus to the subcarrier level, specifically performing principal component analysis at the subcarrier level, and employing a subcarrier-based principal component analysis processing method for the removal of environmental noise; (Refer to...) Figure 2 As shown, only the first principal component is needed to remove environmental noise, and the effect of removing environmental noise is good even if the noise overwhelms the gait pattern signal.

[0088] After removing environmental noise, it is also necessary to remove the influence of AGC noise. The AGC noise removal process can be, but is not limited to, using spatial clustering to cluster AGC-related points based on their sparse distribution density, and retaining only the cluster with the largest number of scattered points to filter out the AGC noise. In other words, to completely preserve the distribution of CSI data, this invention does not use a ratio method, but rather, based on observation, the noise caused by AGC is uncertain in the time dimension, and there are sparse points around the dynamic signal. Therefore, the noise caused by AGC can be filtered out based on the sparse density distribution, i.e., using the DBSCAN spatial clustering method to cluster AGC-related points based on their sparse distribution density, and retaining only the cluster with the largest number of scattered points. The result after filtering the AGC noise is still as follows... Figure 2 As shown.

[0089] Enhancement processing of CSI after removing environmental noise and AGC noise can be performed using the following methods:

[0090] To represent the gait characteristics of different people walking, this invention performs a short-time Fourier transform (STFT) on the CSI processed above to obtain a spectrum reflecting time and frequency information. Specifically, firstly, a sliding window is applied to slice the time series CSI data, each slice can be set to contain 3 seconds of data, and the sliding window can be set to 1 second; then, STFT is performed on each slice.

[0091] To reduce noise in the spectrogram and obtain a refined spectrogram, this invention employs a process that enhances each spectrogram. Specifically, the amplitudes of the spectrograms corresponding to all subcarriers contained in the spectrogram (30 in the experiment) are summed to obtain a superimposed spectrogram, and only the spectrogram within a predetermined frequency range (e.g., 0-80Hz) is retained. Then, each FFT (Fast Fourier Transform) block is normalized, and background noise is eliminated by subtracting the average amplitude of the entire spectrogram (any value less than 0 is set to 0 during the processing). Finally, a two-dimensional Gaussian filter is applied to obtain the enhanced spectrogram, as shown below. Figure 2 As shown.

[0092] (II) Data Augmentation Process

[0093] This invention uses data augmentation in the processing to generate more data about potential behaviors and paths; specifically, through methods such as... Figure 3The adversarial autoencoder (AAE) structure shown performs data augmentation on each user's labeled gait data (i.e., labeled data) using a separate AAE; that is, it uses a small amount of preprocessed gait spectrogram data (i.e., source domain data) accompanying human behavior and different walking paths for data augmentation, and inputs this data into the corresponding AAE#k (i.e., the k-th AAE, or AEE numbered k) for training. The corresponding training process may include:

[0094] First, refer to Figure 3 As shown, the data of the k-th AAE The latent vector z is fed into the encoder of AAE#k and sent to the decoder to generate a vector. To reconstruct labeled gait data; specifically, the reconstruction loss can be defined using mean squared error (MSE), i.e., the corresponding loss function can be:

[0095] ;

[0096] in, It is the number of samples;

[0097] Then, a discriminator is trained to standardize the reconstructed labeled gait data; that is, based on the loss function and the vector of the reconstructed labeled gait data. Data reconstruction is performed to obtain reconstructed gait data related to the user corresponding to the user's label y. This reconstructed gait data includes expanded potential behavior and path data.

[0098] At this point, the encoder in AE is the generator, and its output... It is sent to the discriminator along with a vector z' that follows a prior distribution; for the discriminator, when using When used as input, tags The value is 0 when z' is used as input; the label is... The value is 1; based on this, cross-entropy is used as the loss function for the discriminator:

[0099] ;

[0100] in, It is the predicted value of the i-th sample. It is the true value of the i-th sample;

[0101] However, since data augmentation is performed using a separate AAE for each user, the reconstructed data samples are likely to be located among different users, which in turn affects the classification accuracy of different users. In order to avoid this situation, in addition to the fact that the prior distribution of each AAE is different (the variance of the normal distribution is different), the present invention also inputs the user's label y and latent vector z together into the decoder during the data augmentation process, so as to force the decoder to generate data that is highly relevant to the user.

[0102] (III) Cross-domain gait recognition processing

[0103] In this cross-domain gait recognition process, such as Figure 4 As shown, it may include: a feature extraction process, and a process for learning user gait features independent of non-gait behaviors and paths based on the results of feature extraction.

[0104] In the feature extraction process, all labeled and unlabeled data are input together into the feature extractor to output its feature vector, that is, the feature vector is extracted and obtained;

[0105] Specifically, this invention uses widely adopted convolutional neural networks (CNNs) and long short-term memory (LSTM) deep learning architectures to extract gait features; specifically, it employs a three-layer stacked CNN and a three-layer stacked LSTM to extract features; wherein, in each layer of the CNN, this invention uses a convolutional layer with 2D convolutional kernels, utilizes batch normalization layers to accelerate training, inserts rectified linear units (ReLU) to introduce nonlinearity, and uses max pooling layers to reduce the representativeness of the extracted gait feature size; in addition, LSTM also performs well in time series data processing, it can be used to learn the temporal dynamic features extracted by CNN, and each LSTM layer has the same number of neurons, and uses the sigmoid activation function to extract data;

[0106] Based on the above processing design, given the input data, the corresponding feature vectors can be extracted through the deep learning processes of CNN and LSTM.

[0107] After obtaining the feature vectors, a process can be performed to learn user gait features that are independent of non-gait behaviors and paths based on the results of feature extraction;

[0108] Specifically, in this process, three fully connected layers and the ReLU activation function can be used to learn the representation of the input data (i.e., the extracted feature vector), and then passed through an output layer with a softmax activation function to obtain a gait prediction probability vector, which includes a predicted label for gait classification. Furthermore, to improve the accuracy of gait classification, this invention predicts user labels for both labeled and unlabeled data. and For all data, cross-entropy can be used as the loss function for gait classification:

[0109] , ;

[0110] in, and is the number of labeled and unlabeled data used for training, K is the total number of users, La is the loss function for labeled data, and Lu is the loss function for unlabeled data;

[0111] Domain Adaptive Networks (DANNs) are a special case of transfer learning. This invention utilizes the capabilities of DANNs to eliminate the influence of human behavior and walking paths on gait recognition. Specifically, different walking paths and human behaviors can be defined as different domains, and a domain discriminator is used to identify these different walking paths and human behaviors. The main objective of this invention is to enable the feature extractor to deceive the domain discriminator, thereby generating gait features of the user independent of human behavior and walking paths. The feature vectors are input into the domain discriminator, and the domain labels are predicted through the same process. The domain discriminator can also consist of three fully connected layers with ReLU activation functions and an output layer with a softmax activation function.

[0112] To improve the classification performance of the target domain, two operations can be used to optimize the model.

[0113] First, the feature vectors are concatenated with the predicted labels from gait classification, aligning the posterior distributions of the source and target domains, and then fed together into the domain discriminator to predict the domain labels. .

[0114] Secondly, this invention adds classification constraints to the target domain and uses conditional entropy as the loss function for target domain classification; therefore, the loss function of the domain discriminator consists of three parts:

[0115] ,;

[0116] ,;

[0117] ;

[0118] in and These represent the total amount of data used for training and the amount of data in the target domain, where D is the number of domains. Let be the true domain label of the i-th sample. Let be the predicted domain label for the i-th sample. The predicted domain label for the i-th sample when aligning the posterior distributions of the source and target domains. The predicted domain label for the i-th sample when adding classification constraints to the target domain. Let Ld be the predicted label of the target domain for the i-th sample when adding classification constraints to the target domain, Lo be the loss function for all domains, and Lt be the loss function for predicting all domains when aligning the posterior distributions of the source and target domains.

[0119] Therefore, the overall loss function of the model in this invention can be determined as follows:

[0120] ;

[0121] Here, α, β, γ and λ are hyperparameters. In a specific training model, α, β, γ and λ can all be set to 1, and in practical applications, they may need to be fine-tuned based on the training results.

[0122] In this invention, the training objective of the training model is to minimize the loss of gait classification while maximizing the loss of the domain discriminator. It should be noted that the loss of the backpropagation time-domain discriminator is reversed, that is, the loss function of the time-domain discriminator is negative during backpropagation, while gait classification is directly backpropagated, that is, the loss function of the gait classifier is positive.

[0123] In summary, this invention can use all labeled and unlabeled data to train the corresponding training model and iteratively update the model parameters during the training process.

[0124] In summary, the technical solution provided by this invention can overcome the limitations of existing WIFI-based gait recognition schemes, and it can produce significant technical effects in specific applications, including:

[0125] (1) Based on WIFI devices, the present invention can realize non-contact gait recognition processing, avoiding the device carrying requirements of gait recognition based on wearable devices, which is especially suitable for home-based elderly care, office environments, etc.

[0126] (2) Since commercial WIFI infrastructure is already widely available and its setup cost is relatively low, compared with visual and millimeter-wave radar solutions, the present invention can significantly reduce equipment and installation costs, thereby making large-scale deployment of the gait recognition solution provided by the present invention economically feasible.

[0127] (3) The present invention also achieves robustness against user behavior and walking paths and obtains high gait recognition accuracy by using careful data preprocessing methods and introducing domain adaptation and data augmentation techniques.

[0128] (4) An application experiment of the present invention was conducted in a real-world scenario. The experimental results show that it can accurately predict a total of 15 domain data by collecting and labeling only a small amount of data from 6 source domains. It can also improve the accuracy of user classification. Compared with other existing technologies, the efficiency is improved by about 45%, thus achieving high-precision measurement of gait recognition.

[0129] The invention was applied in real-world scenarios, as shown in the following experiments. Figure 5 As shown, Figure 5 This includes the prediction accuracy for three different users (P1-P3) using four different gait recognition methods. The fourth gait recognition method is the technical solution provided by this invention. Specifically, to evaluate the basic performance of the technical solution provided by this invention, the gait data of three volunteers are first classified, such as... Figure 5 As shown, four gait recognition schemes are employed, namely: CNN module only, AAE+CNN module only, domain adaptation+CNN module only, and the implementation scheme provided by this invention, which includes CNN, AAE, and domain adaptation processing. Figure 5 The experimental results shown demonstrate that the technical solution provided by this invention can significantly improve the accuracy of gait recognition and achieve accurate prediction and recognition for each user.

[0130] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims. The information disclosed in the background section is intended only to enhance the understanding of the overall background technology of the present invention and should not be construed as an admission or implication in any way that such information constitutes prior art known to those skilled in the art.

Claims

1. A method for gait recognition based on a WIFI device, characterized in that, include: In the sensing area where the WIFI propagation link consisting of the transmitter (TX) and receiver (RX) of the WIFI device is located, the channel state information (CSI) of each user is collected when walking in the sensing area. The CSI is preprocessed to obtain the corresponding spectrum; the data preprocessing includes removing environmental noise and automatic gain control (AGC) noise from the CSI. Data augmentation is performed on the labeled data in the spectrogram. Feature extraction is performed on the augmented labeled data, including expanded latent behaviors and paths, and the unlabeled data in the spectrogram. This includes: using convolutional layers with 2D kernels in each layer of the CNN; accelerating the processing with batch normalization layers; inserting ReLU (Revised Linear Units) to introduce nonlinearity; and using max pooling layers to reduce the size of the feature vectors. LSTM learns the temporal dynamic features extracted by the CNN in time series data processing. Each LSTM layer has the same number of neurons, and the Sigmoid activation function is used to extract data. The feature vectors extracted are learned by using three fully connected layers and the ReLU activation function, and then passed through the output layer of the softmax activation function to obtain the gait prediction probability vector. Specifically, cross-entropy is used as the loss function for gait classification, and the prediction probability vector is obtained based on the loss function for gait classification. This prediction probability vector includes the predicted label for gait classification. The feature vector is concatenated with the predicted label for gait classification, the posterior distributions of the source and target domains are aligned, and they are input together into the domain discriminator to predict the domain label. Conditional entropy is used as the loss function for target domain classification, thereby determining the loss function of the domain discriminator. The source domain is the labeled data domain, and the target domain is the unlabeled data domain. The overall loss function of the training model is determined based on the loss function of gait classification and the loss function of the domain discriminator. The training model is trained using all labeled and unlabeled data based on the overall loss function of the training model and the parameters are iteratively updated during the training process to obtain a training model that contains user gait features that are independent of non-gait behaviors and paths. Based on the learned user gait features, gait recognition is performed on users walking in the sensing area.

2. The method according to claim 1, characterized in that, The process of removing environmental noise and AGC noise from the CSI includes: The environmental noise is removed using a principal component analysis method based on subcarriers; and, The AGC noise removal process includes: using spatial clustering to cluster AGC-related points according to their sparse distribution density, and retaining only the cluster with the largest number of scattered points to filter out the AGC noise.

3. The method according to claim 2, characterized in that, The method also includes: The CSI data after preprocessing is subjected to a Fourier Transform (STFT) to obtain the corresponding spectrum, and the spectrum is then enhanced to obtain an enhanced spectrum, which is used as the spectrum obtained from the data preprocessing.

4. The method according to claim 3, characterized in that, The process of obtaining the enhanced spectrum includes: A sliding window is used to slice the time series CSI data, and STFT is performed on each slice; The amplitudes of the spectrum maps corresponding to multiple subcarriers are added together to obtain a superimposed spectrum map, and only the spectrum map of the predetermined frequency band range is retained; After normalizing each FFT block contained in the spectrum, background noise is eliminated by subtracting the average amplitude of the entire spectrum, and an enhanced spectrum is obtained by processing with a two-dimensional Gaussian filter.

5. The method according to any one of claims 1 to 4, characterized in that, The process of performing data augmentation on the labeled data in the spectrogram includes: training the labeled gait data of each user in the labeled data using an adversarial autoencoder (AAE), the training process including: The data of the kth AAE The latent vector z is generated by feeding it into the encoder of AAE#k; where AAE#k refers to the AAE numbered k. The latent vector z and the user's label y are sent to the decoder to generate a vector for reconstructing the labeled gait data. In the process of reconstructing labeled gait data, the mean squared error (MSE) is used to define the reconstruction loss, and the corresponding loss function is: ； in, It is the number of samples; Based on the loss function and the vector of the reconstructed marked gait data Data reconstruction is performed to obtain reconstructed gait data related to the user corresponding to the user's label y. This reconstructed gait data includes expanded potential behavior and path data. as well as, The encoder is used as input data to the Generative Adversarial Network (GAN) generator. The data, along with a vector z' following a prior distribution, is input into the discriminator to standardize the reconstructed data through training. Cross-entropy is used as the loss function of the discriminator, and the loss function of the discriminator is: ； For the discriminator, when using When used as input, tags The value is 0 when z' is used as input; the label is... =1, It is the predicted value of the i-th sample. It is the true value of the i-th sample.

6. The method according to any one of claims 1 to 4, characterized in that, The loss function for gait classification is: , ； in, and K represents the number of labeled and unlabeled data points used for training, and K is the total number of users. and Let represent the user labels for the predicted labeled and unlabeled data, respectively; La is the loss function for the labeled data, and Lu is the loss function for the unlabeled data. The loss function of the domain discriminator is: ； ,； ； in, and These represent the total amount of data used for training and the amount of data in the target domain, where D is the number of domains. Let be the true domain label of the i-th sample. Let be the predicted domain label for the i-th sample. The predicted domain label for the i-th sample when aligning the posterior distributions of the source and target domains. The predicted domain label for the i-th sample when adding classification constraints to the target domain. The predicted label of the target domain for the i-th sample when adding classification constraints to the target domain; Ld is the loss function for all domains, Lo is the loss function for predicting all domains when aligning the posterior distributions of the source and target domains, and Lt is the loss function for predicting all target domains when adding classification constraints to the target domain. The overall loss function of the model is: ； Here, α, β, γ, and λ are hyperparameters, and their values ​​are adjusted based on the training results.

7. The method according to claim 6, characterized in that, During backpropagation, the loss function of the domain discriminator is negative, while the loss function of the gait classification is positive.

Images