Non-contact multi-mode user identity identification method based on WiFi signal

Abstract

The invention discloses a non-contact multi-mode user identity recognition method based on a WiFi signal, and the method comprises the steps: firstly, carrying out the preprocessing of a CSI signal, including complex conjugate product, abnormal value removal through a Hampel filter, median filtering and denoising, and high-frequency disturbance suppression through a Butterworth low-pass filter; then, different indexes such as dynamic time planning, singular value decomposition and information entropy are used for screening subcarrier sequences with large contribution degrees; in the feature construction stage, a complete multi-modal input vector is constructed by extracting statistical features (standard deviation, range and MAD) of amplitude and phase and a Doppler frequency shift spectrogram, finally, a multi-branch user identity recognition model fusing convolution, a self-attention mechanism and a visual converter Transform is constructed, and high-precision identity recognition is achieved.

Description

Technical Field

[0001] This invention belongs to the field of wireless sensing technology, specifically relating to a non-contact multimodal user identification method based on WiFi signals. Background Technology

[0002] With societal development and increased public awareness of safety, identity verification technology has become a hot topic for researchers. Identity verification has wide applications in daily life, such as clocking in at work, logging into accounts, and access control in residential communities. One method involves using WiFi signals to collect and identify a person's respiratory characteristics to complete the identification process.

[0003] However, current identity recognition technologies are mainly divided into computer vision image-based methods and human biometrics-based methods. Both of these methods rely on cameras or specific sensor devices to capture an individual's facial or other physical features to complete identity verification.

[0004] To address the problems of the two types of identity recognition technologies mentioned above, an increasing number of researchers are turning their attention to wireless signals. Compared to cameras and biometrics, wireless signals (such as WiFi) exhibit unique advantages. First, they are non-line-of-sight (NLOS) and contactless, meaning they can work effectively even without direct eye contact, while avoiding potential health or psychological barriers caused by contact. Furthermore, because this method does not rely on cameras, it better protects personal privacy. In 2010, Daniel Halperin's research team modified the firmware of a wireless network card to accurately understand and describe the channel state, successfully extracting Channel State Information (CSI) from Wi-Fi signals and exploring the potential applications of this data in various fields. In 2016, Zhang's team first applied radio signals to the field of identity recognition, analyzing channel state information to extract gait characteristics representative of an individual for identification. Ultimately, they achieved identity recognition in groups of 2 to 6 people, with an average accuracy rate of 93%–77%. In the same year, Feng Hong's team proposed the WFID identity recognition system, which extracts the subcarrier amplitude frequency (SAF) matrix and uses a support vector machine (SVM) to complete identity recognition. The WFID model achieved an accuracy of 90.9% in a task involving the recognition of 9 people. Also in that year, Zeng proposed the WiWho model, which uses stride length and walking analysis to identify walking gait and relies on this information to complete the identity recognition task. The WiWho model achieved an accuracy of 80% in a task involving the recognition of 6 people. In 2018, Akarsh's team built the NeuralWave model, using a deep convolutional neural network and a softmax classifier to achieve identity recognition. This model significantly increased the number of people it could recognize from single digits to 24, and achieved an accuracy of 87.76%. Although the above teams have all completed identity recognition tasks, the following problems still exist: (1) Poor cross-scene adaptability: WiFi signal characteristics vary greatly in different environments. How to make the model have good cross-scene generalization ability is an urgent problem to be solved.

[0005] (2) High algorithm complexity: In order to accurately extract useful biometric features from WiFi signals, complex algorithms and deep learning models need to be designed.

[0006] (3) Insufficient biometric extraction: How to effectively extract rich biometric information from WiFi signals to improve the accuracy of identity recognition remains a research hotspot. Summary of the Invention

[0007] The purpose of this invention is to provide a non-contact multimodal user identification method based on WiFi signals, which solves the problem of the shortcomings of traditional WiFi user identification methods in terms of feature utilization and modeling capabilities.

[0008] The technical solution adopted in this invention is a non-contact multimodal user identification method based on WiFi signals. Addressing the differences in feature spaces across different modalities, a CSA-ViT model is designed. This model is a multi-branch user identification model that integrates convolution, self-attention mechanisms, and a visual transformer (Transformer). A WiFi detection area with multiple antennas, a single transmitter, and a single receiver is constructed to collect CSI data under different users' breathing patterns. The raw CSI data is preprocessed by first performing a complex conjugate product on the CSI signal, then using a Hampel filter to handle outliers, followed by data denoising: a median filter smooths high-frequency random noise, a Butterworth low-pass filter suppresses high-frequency disturbances, and finally, subcarrier selection is performed. In the feature construction stage, statistical features are extracted from the preprocessed CSI, including statistical features of amplitude and phase difference (standard deviation, range, MAD) and the physical feature Doppler spectrum, to construct a complete multimodal input vector. Next, the input vector is fed into different branches according to the feature category: the statistical feature branch uses lightweight convolution combined with a self-attention mechanism to extract the spatial structure and long-range dependencies of statistical features; the physical feature branch uses the Visual Transformer (ViT) to model the Doppler spectrum image and extract time-frequency mode information; finally, the multimodal features are weighted and fused through a cross-attention mechanism to achieve better information integration and user recognition capabilities; the specific operation is as follows: Step 1: Construct a MIMO-OFDM (Multiple-Input Multiple-Output Orthogonal Frequency-Division Multiplexing) WiFi detection area with multiple antennas, single transmitter, and single receiver, and collect channel state information (CSI) under normal breathing conditions of different users in a laboratory environment. Step 2: Preprocess the CSI data collected in Step 1, including complex conjugate product, outlier removal, data denoising, and subcarrier selection. Step 3: Extract the statistical features (standard deviation, range, MAD) of amplitude and phase, as well as the Doppler spectrum, from the preprocessed CSI data in Step 2, and construct and divide the dataset into training set, validation set and test set; Step 4: Input the training set divided in Step 3 into the multi-branch model, extract and fuse features through convolution, self-attention mechanism and visual Transformer, use cross-attention mechanism to perform weighted fusion of multimodal features, complete the training of user recognition, and save the model parameters with the best recognition rate on the validation set to obtain the optimized identity recognition model. Step 5: Input the test set into the user identification model from Step 4 for classification, and finally obtain the prediction results.

[0009] The invention is further characterized in that, Step 1 is as follows: Step 1.1: The hardware platform includes a desktop computer with a modified Intel 5300 NIC driver installed and a standard wireless router with three antennas. The WiFi signal follows the IEEE 802.11n protocol standard, operates in the 2.4 GHz band, has a channel bandwidth of 40 MHz, and a data packet transmission frequency of 100 Hz (i.e., 100 data packets per second). On the software side, a Linux system is used, and the open-source tool CSI-Tool is deployed to achieve real-time capture and recording of CSI data. Step 1.2: According to the Fresnel zone propagation theory, signals are more sensitive to target movement in the area closer to the transmitting and receiving antennas. Therefore, to enhance the system's ability to detect respiratory signals, subjects were required to sit quietly at a position midway between the line of sight of the transmitting and receiving antennas and breathe naturally. To verify the system's detection accuracy, the HKH-11C digital respiratory sensor was introduced as a reference standard. This device is a contact-type strap sensor fixed to the subject's chest and abdomen, capable of recording physical changes in respiratory movements in real time. In all experiments, a horizontal distance of 2 meters was maintained between the transmitting and receiving antennas, and their height was set parallel to the subject's abdomen to maximize the capture of channel state information related to breathing. Ten healthy subjects (5 males and 5 females) were recruited for this experiment. Each subject completed normal breathing in a quiet sitting position. After the respiratory sensor waveform stabilized, CSI data was collected. Each breathing pattern was maintained for 60 seconds, with a total collection time of approximately 5 minutes, yielding multiple sets of different types of respiratory data. As shown in the figure, CSI data was collected in an indoor setting. The specific experimental setting is located on the first floor of the School of Art and Design at the Qujiang Campus of Xi'an University of Technology. The conference room covers an area of ​​40 square meters, is fully enclosed, and is equipped with tables, chairs, monitors, and various books.

[0010] Step 2 specifically includes the following steps: Step 2.1: Using the CSI data of different subjects' breathing collected in Step 1, obtain a file with the .dat extension. Read this file using the read_bf_file function on the Matlab software platform. The CSI data is a... N A two-dimensional matrix of rows; CSI data uses a matrix Represented as:

[0011] The number of antennas at the transmitting end. This refers to the number of antennas at the receiving end. For the first transmitting antenna The first antenna to the receiving antenna CSI data of the root antenna Represented as:

[0012] The number of subcarriers in the CSI signal on the Intel 5300 wireless network card. The value is 30; as shown in step 1.1, both the receiver and transmitter are equipped with three antennas, therefore the received signal contains 9 data streams. The data contained in each data packet is explained as follows:

[0013] Step 2.2: In order to identify the similarity and time delay between signals, complex conjugate is used to ensure the correct phase relationship when calculating the cross-correlation between two signals.

[0014] Suppose we have two discrete complex signals x[n] and y[n], where n represents a discrete time index. For each time point n, the value of the signal is in complex form, i.e. and , where j is the imaginary unit. The conjugate product is defined at each time point n as:

[0015] like Then its conjugate is ,as a result .

[0016] The experimental setup for this experiment consists of three transmitting antennas and three receiving antennas. For each transmitting antenna, there are three receiving antennas receiving data, with three different signals x, y, and z. We perform complex conjugate products on each pair of these signals to enhance the target signal and reduce the impact of environmental noise; that is, we perform complex conjugate products on x and y, y and z, and z and x.

[0017] Step 2.3 involves applying Hampel filtering to the CSI data after the complex conjugate product operation in Step 2.2 to handle outliers. Hampel filtering is a method used in signal processing and data analysis to detect and remove outliers. Based on statistical principles, the Hampel filter identifies outliers by calculating the difference between the median of a data point and the median of its surrounding data points, using the Median Absolute Deviation (MAD) as a scaling estimate. First, for a given data point, a window (usually an odd length to ensure central symmetry) is selected, encompassing that point and several points before and after it. The window size affects the filtering effect; a larger window may smooth out more details, while a smaller window may retain more details but also more noise. In this experiment, a window size of 50 was chosen because it was necessary to extract breathing features; excessive details and noise would make the data more cluttered and affect the experimental results. Next, calculate the median and MAD: Within the selected window, first calculate the median m of all data points, then calculate the absolute deviation of each point from the median, and find the median of these absolute deviations, i.e., the Median Absolute Deviation (MAD), where xi is the data point in the window. MAD is defined as follows:

[0018] Next, a threshold is determined and outliers are identified: data points whose distance from the median exceeds a certain multiple of the median (MAD) are considered outliers. This method sets this multiple, k, to 3, corresponding to 99.3% data coverage under a normal distribution. Specifically, a data point x is considered an outlier if it meets the following conditions:

[0019] Finally, replace outliers: Once outliers are identified, replace them with the median within the window to reduce their impact on subsequent analysis.

[0020] Step 2.4 involves denoising the CSI data after outlier processing in Step 2.3. This method employs a median filter and a Butterworth filter to address the noise in the data. Median filtering effectively reduces noise by replacing the center pixel's value with the median value within a local neighborhood, while preserving edge details. The Butterworth filter is used to extract useful information from the respiratory signal, especially when removing noise and interfering frequency components from physiological signals. Human respiratory signals typically exhibit low-frequency fluctuations; the respiratory rate of a normal adult is generally between 0.16 and 0.6 Hz (approximately 10-20 breaths per minute). However, in actual data acquisition, respiratory signals are often affected by high-frequency electronic noise. Therefore, filters are needed to process the raw signal and extract the signal within the frequency range representing respiratory activity.

[0021] Steps 2.5 and 2.4 involve selecting subcarriers from the denoised CSI data. This experiment uses a 3-transmit, 3-receive setup, with each of the three receiving antennas for the first transmitting antenna having 10 subcarriers, resulting in a total of 30 subcarriers. However, not all subcarriers have the same channel gain or signal-to-noise ratio. Some subcarriers may be in a deep fading state, resulting in poor transmission reliability; while others may have good channel quality. Therefore, to avoid using subcarriers that are severely interfered with or attenuated, and to select subcarriers with good channel quality for data transmission, this method comprehensively considers the contribution rate of Dynamic Time Warping (DTW), Singular Value Decomposition (SVD), and information entropy, selecting the 10 best-performing subcarriers from the 30 subcarriers.

[0022] This method calculates the DTW distance for each pair of the 30 subcarriers and stores the results in a 30x30 matrix. Finally, the average of the 30 DTW distances in the i-th row of the matrix is ​​taken to represent the DTW distance of the i-th subcarrier, and the DTW distances of the 30 subcarriers are normalized.

[0023] Then, singular value decomposition is performed on the matrix storing the information of 30 subcarriers. The diagonal values ​​of the resulting matrix are taken to represent the singular value decomposition contribution rates of the 30 subcarriers, and then normalized.

[0024] Finally, the amplitude and phase of each of the 30 subcarriers are extracted, and their information entropy is calculated separately. The information entropy of the amplitude and phase are then added together and the negative value is taken. Finally, the information entropy of the 30 subcarriers is normalized.

[0025] After calculating the above three indicators, the normalized DTW distance, SVD contribution rate, and information entropy of the 30 subcarriers were obtained. Weighting parameters were then assigned to each of these three indicators, and the results were summed.

[0026]

[0027] Where D is the DTW distance, S is the SVD contribution rate, I is the information entropy, alpha is the weight of the DTW distance (0.4), beta is the weight of the SVD contribution rate (0.4), and gamma is the weight of the information entropy (0.2). Ultimately, 10 subcarriers with the best performance were selected.

[0028] Step 3 is as follows: Step 3.1: Extract the statistical characteristics (standard deviation, range, MAD) of amplitude and phase from the preprocessed CSI data in Step 2: This method extracts the amplitude and phase of the data after outlier handling and noise reduction. Statistical characteristics are then calculated by sliding a window of size 100. Furthermore, the standard deviation (SD), median absolute deviation (MAD), and range are extracted. The standard deviation reflects the dispersion of the dataset, i.e., the distribution of data points relative to the mean. For a sequence of n values ​​{x1, x2, ..., xn}, the sample standard deviation is calculated as follows:

[0029] The median absolute deviation (MAD) measures the degree of statistical dispersion in a dataset. Similarly, for a sequence of n values ​​X = {x1, x2, ..., xn}, the MAD is calculated as follows:

[0030] The range reflects the range or fluctuation of data. Similarly, for a sequence of n values ​​X={x1,x2,...,xn}, the formula for calculating the range is as follows:

[0031] in, The maximum value in X. It is the minimum value in X.

[0032] This experiment calculates the statistical characteristics of each subcarrier in the amplitude and phase data (a 5000*10 matrix, where 5000 represents the number of packets and 10 represents the number of filtered subcarriers). A sliding window with a window size of 100 and a step size of 1 is used to calculate its statistical characteristics (standard deviation, median absolute deviation, and range). The final statistical characteristics of this subcarrier are obtained (a 4900*6 matrix, where 4900 represents the number of packets), with the six columns representing the standard deviation, median absolute deviation, and range of both amplitude and phase information.

[0033] After obtaining the amplitude, phase, and other parameters, they are standardized. Standardization unifies the measurement standard; in practical applications, different features may have completely different units and orders of magnitude. Without standardization, comparisons between these features become meaningless and may lead to model bias towards features with larger values. After standardization, all features are on the same benchmark, facilitating fair comparisons. Step 3.2: Extract the Doppler spectrum of the signal from the preprocessed CSI data in Step 2.

[0034] The Doppler frequency shift spectrum is obtained by short-time Fourier transform. For a discrete signal x[n], the short-time Fourier transform formula is:

[0035] Where m is the m-th time frame, k is the k-th frequency point, M is the frame shift (i.e., the difference in the number of samples between each frame), N is the window length (i.e., the number of samples in each frame), and w[n] is the window function. To facilitate visualization and further analysis, we save the results of the short-time Fourier transform in the form of a heatmap; Step 3.3: Combine the statistical characteristics of amplitude and phase of each CSI data point with the Doppler spectrum into an instance; then, repeat steps 3.1 and 3.2 to construct the complete dataset; finally, divide the dataset according to the ratio of training set:validation set:test set = 7:1.5:1.5.

[0036] Step 4 is as follows: The training set divided in step 3 is used as the input to the user identification model CSA-ViT. The indoor personnel identification model CSA-ViT is built based on self-attention and the Vision Transformer.

[0037] This model is designed to process and analyze three different types of input data: statistical features of amplitude and phase, and physical features (Doppler spectrum). The model processes these inputs through different branches and then fuses the processed features to achieve the final classification task.

[0038] For the statistical features of amplitude and phase, a 3x3 convolution kernel is first used for convolution, then a self-attention mechanism is used to capture long-range dependencies in the sequence, and finally, two vectors of length 500 are obtained through a fully connected layer. For the physical features (Doppler spectrum), the Vision Transformer model is directly applied to process the input data, and then a vector of length 500 is obtained through a fully connected layer. To eliminate the influence of units and unify the measurement standard, the three variables are standardized. Then, cross-attention operations are performed on each of the three vectors pairwise to obtain richer and more comprehensive feature representations. Since the cross-attention mechanism involves dot product operations, the three vectors after cross-attention are standardized again to prevent gradient explosion or gradient vanishing during training. Finally, these three vectors are concatenated to a length of 1500, fully connected, and then fed into a Bidirectional Long Short-Term Memory (BiLSTM) network. The parameters in the hidden layer are extracted and then fully connected again to complete the classification, i.e., to complete the training of user recognition. The model parameters with the best recognition rate on the validation set are saved to obtain the optimized identity recognition model. The training process used a learning rate of 1e-5, a batch size of 1, a cross-entropy loss function, and an adaptive moment estimation (Adam) optimizer.

[0039] Step 5 specifically includes: The test set obtained in step 3.3 is input into the user identification model obtained in step 4 for testing, classification is achieved, and the prediction result is finally obtained.

[0040] The beneficial effects of this invention are as follows: The contactless multimodal user identification method based on WiFi Channel State Information (CSI) proposed in this invention provides an efficient, privacy-preserving, and highly adaptable solution to the problems of privacy issues, line-of-sight limitations, high costs, and insufficient feature extraction in traditional identification methods. This method preprocesses the raw CSI data, including using a Hampel filter to detect and remove outliers, employing a median filter and a Butterworth filter for noise reduction, and mapping the data to a standard range (0 to 255) through linear normalization. Then, it extracts the statistical features of the amplitude and phase of the CSI data (standard deviation, range, median absolute deviation) and the Doppler frequency shift spectrum as physical features. A multi-branch user identification model (CSA-ViT model) is constructed, where the statistical feature branch uses lightweight convolution combined with a self-attention mechanism, the frequency domain branch uses a Vision Transformer to model the DFS image, and a cross-attention mechanism is used to weightedly fuse the multimodal features. Experimental results show that the proposed method achieves a recognition accuracy of 99.07% in indoor scenarios, significantly outperforming traditional methods. This verifies the effectiveness and robustness of the proposed model in multimodal WiFi feature modeling. This method not only overcomes the limitations of traditional approaches but also improves recognition accuracy and generalization ability through multimodal feature fusion, providing new ideas and methods for the development of contactless identity recognition technology. Attached Figure Description

[0041] Figure 1 This is a framework diagram of the user identification model CSA-ViT of this invention; Figure 2(a) Doppler spectrum of user A; Figure 2(b) Doppler spectrum of user B; Figure 2(c) Doppler spectrum of user C; Figure 2(d) Doppler spectrum of user D; Figure 3 This is a structural diagram of the user identification model based on deep learning in this invention; Figure 4 It is a confusion matrix for the identification effect of 10 people; Figure 5 This is a comparison of the number of users and accuracy rates between the model of this invention and the traditional model. Detailed Implementation

[0042] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments, so that the advantages and features of the present invention can be more readily understood by those skilled in the art.

[0043] Example 1 This invention presents a non-contact multimodal user identification method based on WiFi signals. It establishes a MIMO-OFDM WiFi detection area with a single transmitter and receiver, collecting CSI data from different users' breathing patterns. The CSI data is preprocessed: first, outliers are removed using a Hampel filter; then, noise is denoised using a median filter and a Butterworth filter; finally, the data is normalized. The preprocessed CSI data is converted into multimodal feature vectors, including statistical features of amplitude and phase (standard deviation, range, median absolute deviation) and a Doppler frequency shift spectrum, and the dataset is constructed and partitioned. The constructed training set is input into the user identification model CSA-ViT for training. Feature information is extracted through convolution, self-attention mechanisms, and a Vision Transformer, and multimodal feature fusion is performed using a cross-attention mechanism to complete the user identification training. The model parameters with the best recognition rate on the validation set are saved, resulting in an optimized identification model; finally, user identification is achieved.

[0044] Example 2 The specific steps of the contactless multimodal user identification method based on WiFi signals are as follows: Step 1: Construct a MIMO-OFDM (Multiple-Input Multiple-Output Orthogonal Frequency-Division Multiplexing) WiFi detection area with multiple antennas, single transmitter, and single receiver, and collect channel state information (CSI) under normal breathing conditions of different users in a laboratory environment. Step 2: Preprocess the CSI data collected in Step 1, including complex conjugate product, outlier removal, data denoising, and subcarrier selection. Step 3: Extract the statistical features (standard deviation, range, MAD) of amplitude and phase, as well as the Doppler spectrum, from the preprocessed CSI data in Step 2, and construct and divide the dataset into training set, validation set and test set; Step 4: Input the training set divided in Step 3 into the multi-branch model, extract and fuse features through convolution, self-attention mechanism and visual Transformer, use cross-attention mechanism to perform weighted fusion of multimodal features, complete the training of user recognition, and save the model parameters with the best recognition rate on the validation set to obtain the optimized identity recognition model. Step 5: Input the test set into the user identification model from Step 4 for classification, and finally obtain the prediction results.

[0045] Example 3 This invention proposes a contactless multimodal user identification method based on WiFi Channel State Information (CSI). Addressing the privacy concerns, line-of-sight limitations, high costs, and insufficient feature extraction inherent in traditional identification methods, this method offers an efficient, privacy-preserving, and adaptable solution. The method preprocesses the raw CSI data, including using a Hampel filter to detect and remove outliers, employing a median filter and a Butterworth filter for noise reduction, and mapping the data to a standard range (0 to 255) through linear normalization. It then extracts the statistical features of the amplitude and phase of the CSI data (standard deviation, range, median absolute deviation) and the Doppler frequency shift spectrum as physical features. A multi-branch user identification model (CSA-ViT model) is constructed, where the statistical feature branch uses lightweight convolution combined with a self-attention mechanism, the frequency domain branch uses a Vision Transformer to model the DFS image, and a cross-attention mechanism is used to weightedly fuse the multimodal features. Experimental results show that the proposed method achieves a recognition accuracy of 99.07%, significantly outperforming traditional methods, thus validating the effectiveness and robustness of the proposed model in multimodal WiFi feature modeling. This method not only overcomes the limitations of traditional approaches but also improves recognition accuracy and generalization ability through multimodal feature fusion, providing new ideas and methods for the development of contactless identity recognition technology and compensating for the shortcomings of traditional models.

[0046] Example 4 The non-contact multimodal user identification method based on WiFi Channel State Information (CSI) proposed in this invention, such as... Figure 1 As shown, the specific steps include: Step 1 is as follows: Step 1.1: The hardware platform includes a desktop computer with a modified Intel 5300 NIC driver installed and a standard wireless router with three antennas. The WiFi signal follows the IEEE 802.11n protocol standard, operates in the 2.4 GHz band, has a channel bandwidth of 40 MHz, and a data packet transmission frequency of 100 Hz (i.e., 100 data packets per second). On the software side, a Linux system is used, and the open-source tool CSI-Tool is deployed to achieve real-time capture and recording of CSI data. Step 1.2: According to the Fresnel zone propagation theory, signals are more sensitive to target movement in the area closer to the transmitting and receiving antennas. Therefore, to enhance the system's ability to detect respiratory signals, subjects were required to sit quietly at a position midway between the line of sight of the transmitting and receiving antennas and breathe naturally. To verify the system's detection accuracy, the HKH-11C digital respiratory sensor was introduced as a reference standard. This device is a contact-type strap sensor fixed to the subject's chest and abdomen, capable of recording physical changes in respiratory movements in real time. In all experiments, a horizontal distance of 2 meters was maintained between the transmitting and receiving antennas, and their height was set parallel to the subject's abdomen to maximize the capture of channel state information related to breathing. Ten healthy subjects (5 males and 5 females) were recruited for this experiment. Each subject completed normal breathing in a quiet sitting position. After the respiratory sensor waveform stabilized, CSI data was collected. Each breathing pattern was maintained for 60 seconds, with a total collection time of approximately 5 minutes, yielding multiple sets of different types of respiratory data. As shown in the figure, CSI data was collected in an indoor setting. The specific experimental setting is located on the first floor of the School of Art and Design at the Qujiang Campus of Xi'an University of Technology. The conference room covers an area of ​​40 square meters, is fully enclosed, and is equipped with tables, chairs, monitors, and various books.

[0047] Step 2 specifically includes the following steps: Step 2.1: Using the CSI data of different subjects' breathing collected in Step 1, obtain a file with the .dat extension. Read this file using the read_bf_file function on the Matlab software platform. The CSI data is a... N A two-dimensional matrix of rows; CSI data uses a matrix Represented as:

[0048] The number of antennas at the transmitting end. This refers to the number of antennas at the receiving end. For the first transmitting antenna The first antenna to the receiving antenna CSI amplitude data of the root antenna Represented as:

[0049] The number of subcarriers in the CSI signal on the Intel 5300 wireless network card. The value is 30; as shown in step 1.1, both the receiver and transmitter are equipped with three antennas, therefore the received signal contains 9 data streams. The data contained in each data packet is explained as follows:

[0050] Step 2.2: In order to identify the similarity and time delay between signals, complex conjugate is used to ensure the correct phase relationship when calculating the cross-correlation between two signals.

[0051] Suppose we have two discrete complex signals x[n] and y[n], where n represents a discrete time index. For each time point n, the value of the signal is in complex form, i.e. and , where j is the imaginary unit. The conjugate product is defined at each time point n as:

[0052] like Then its conjugate is ,as a result .

[0053] The experimental setup for this experiment consists of three transmitting antennas and three receiving antennas. For each transmitting antenna, there are three receiving antennas receiving data, with three different signals x, y, and z. We perform complex conjugate products on each pair of these signals to enhance the target signal and reduce the impact of environmental noise; that is, we perform complex conjugate products on x and y, y and z, and z and x.

[0054] Step 2.3 involves applying Hampel filtering to the CSI data after the complex conjugate product operation in Step 2.2 to handle outliers. Hampel filtering is a method used in signal processing and data analysis to detect and remove outliers. Based on statistical principles, the Hampel filter identifies outliers by calculating the difference between the median of a data point and the median of its surrounding data points, using the Median Absolute Deviation (MAD) as a scaling estimate. First, for a given data point, a window (usually an odd length to ensure central symmetry) is selected, encompassing that point and several points before and after it. The window size affects the filtering effect; a larger window may smooth out more details, while a smaller window may retain more details but also more noise. In this experiment, a window size of 50 was chosen because it was necessary to extract breathing features; excessive details and noise would make the data more cluttered and affect the experimental results. Next, calculate the median and MAD: Within the selected window, first calculate the median m of all data points, then calculate the absolute deviation of each point from the median, and find the median of these absolute deviations, i.e., the Median Absolute Deviation (MAD), where xi is the data point in the window. MAD is defined as follows:

[0055] Next, a threshold is determined and outliers are identified: data points whose distance from the median exceeds a certain multiple of the median (MAD) are considered outliers. This method sets this multiple, k, to 3, corresponding to 99.3% data coverage under a normal distribution. Specifically, a data point x is considered an outlier if it meets the following conditions:

[0056] Finally, replace outliers: Once outliers are identified, replace them with the median within the window to reduce their impact on subsequent analysis.

[0057] Step 2.4 involves denoising the CSI data after outlier processing in Step 2.3. This method employs a median filter and a Butterworth filter to address the noise in the data. Median filtering effectively reduces noise by replacing the center pixel's value with the median value within a local neighborhood, while preserving edge details. The Butterworth filter is used to extract useful information from the respiratory signal, especially when removing noise and interfering frequency components from physiological signals. Human respiratory signals typically exhibit low-frequency fluctuations; the respiratory rate of a normal adult is generally between 0.16 and 0.6 Hz (approximately 10-20 breaths per minute). However, in actual data acquisition, respiratory signals are often affected by high-frequency electronic noise. Therefore, filters are needed to process the raw signal and extract the signal within the frequency range representing respiratory activity.

[0058] Steps 2.5 and 2.4 involve filtering subcarriers from the denoised CSI data. This experiment uses a 3-transmit, 3-receive setup. Each of the three receiving antennas for the first transmitting antenna has 10 subcarriers, for a total of 30 subcarriers. However, not all subcarriers have the same channel gain or signal-to-noise ratio. Some subcarriers may be in a deep fading state, resulting in poor transmission reliability; while others may be in a state with good channel quality. Therefore, to avoid using subcarriers that are severely interfered with or attenuated, and to select subcarriers with good channel quality for data transmission, this method comprehensively considers the contribution rate of Dynamic Time Warping (DTW), Singular Value Decomposition (SVD), and information entropy, selecting the 10 best-performing subcarriers from the 30 subcarriers.

[0059] This method calculates the DTW distance for each pair of the 30 subcarriers and stores the results in a 30x30 matrix. Finally, the average of the 30 DTW distances in the i-th row of the matrix is ​​taken to represent the DTW distance of the i-th subcarrier, and the DTW distances of the 30 subcarriers are normalized.

[0060] Then, singular value decomposition is performed on the matrix storing the information of 30 subcarriers. The diagonal values ​​of the resulting matrix are taken to represent the singular value decomposition contribution rates of the 30 subcarriers, and then normalized.

[0061] Finally, the amplitude and phase of each of the 30 subcarriers are extracted, and their information entropy is calculated separately. The information entropy of the amplitude and phase are then added together and the negative value is taken. Finally, the information entropy of the 30 subcarriers is normalized.

[0062] After calculating the above three indicators, the normalized DTW distance, SVD contribution rate, and information entropy of the 30 subcarriers were obtained. Weighting parameters were then assigned to each of these three indicators, and the results were summed.

[0063]

[0064] Where D is the DTW distance, S is the SVD contribution rate, I is the information entropy, alpha is the weight of the DTW distance (0.4), beta is the weight of the SVD contribution rate (0.4), and gamma is the weight of the information entropy (0.2). Finally, 10 subcarriers with the best performance were selected. Step 3 is as follows: Step 3.1: Extract the statistical characteristics (standard deviation, range, MAD) of amplitude and phase from the preprocessed CSI data in Step 2: This method extracts the amplitude and phase of the data after outlier handling and noise reduction. Statistical characteristics are then calculated by sliding a window of size 100. Furthermore, the standard deviation (std), median absolute deviation (MAD), and range are extracted. The standard deviation reflects the dispersion of the dataset, i.e., the distribution of data points relative to the mean. For a sequence of n values ​​{x1, x2, ..., xn}, the sample standard deviation is calculated as follows:

[0065] The median absolute deviation (MAD) measures the degree of statistical dispersion in a dataset. Similarly, for a sequence of n values ​​X = {x1, x2, ..., xn}, the MAD is calculated as follows:

[0066] The range reflects the range or fluctuation of data. Similarly, for a sequence of n values ​​X={x1,x2,...,xn}, the formula for calculating the range is as follows:

[0067] in, The maximum value in X. It is the minimum value in X.

[0068] This experiment calculates the statistical characteristics of each subcarrier in the amplitude and phase data (a 5000*10 matrix, where 5000 represents the number of packets and 10 represents the number of filtered subcarriers). A sliding window with a window size of 100 and a step size of 1 is used to calculate its statistical characteristics (standard deviation, median absolute deviation, and range). The final statistical characteristics of this subcarrier are obtained (a 4900*6 matrix, where 4900 represents the number of packets), with the six columns representing the standard deviation, median absolute deviation, and range of both amplitude and phase information.

[0069] After obtaining the amplitude, phase, and other parameters, they are standardized. Standardization unifies the measurement standard; in practical applications, different features may have completely different units and orders of magnitude. Without standardization, comparisons between these features become meaningless and may lead to model bias towards features with larger values. After standardization, all features are on the same benchmark, facilitating fair comparisons. Step 3.2: Extract the Doppler spectrum of the signal from the preprocessed CSI data in Step 2.

[0070] The Doppler frequency shift spectrum is obtained by short-time Fourier transform. For a discrete signal x[n], the short-time Fourier transform formula is:

[0071] Where m is the m-th time frame, k is the k-th frequency point, M is the frame shift (i.e., the difference in the number of samples between each frame), N is the window length (i.e., the number of samples in each frame), and w[n] is the window function. For ease of visualization and further analysis, we save the STFT results in the form of a heatmap; Step 3.3: Combine the statistical characteristics of amplitude and phase of each CSI data point with the Doppler spectrum into an instance; then, repeat steps 3.1 and 3.2 to construct the complete dataset; finally, divide the dataset according to the ratio of training set:validation set:test set = 7:1.5:1.5.

[0072] Step 4 is as follows: The training set divided in step 3 is used as the input to the user identification model CSA-ViT. The indoor personnel identification model CSA-ViT is built based on self-attention and the Vision Transformer.

[0073] This model is designed to process and analyze three different types of input data: statistical features of amplitude and phase, and physical features (Doppler spectrum). The model processes these inputs through different branches and then fuses the processed features to achieve the final classification task.

[0074] For the statistical features of amplitude and phase, a 3x3 convolution kernel is first used for convolution, then a self-attention mechanism is used to capture long-range dependencies in the sequence, and finally, two vectors of length 500 are obtained through a fully connected layer. For the physical features (Doppler spectrum), the Vision Transformer model is directly applied to process the input data, and then a vector of length 500 is obtained through a fully connected layer. To eliminate the influence of units and unify the measurement standard, the three variables are standardized. Then, cross-attention operations are performed on each of the three vectors pairwise to obtain richer and more comprehensive feature representations. Since the cross-attention mechanism involves dot product operations, the three vectors after cross-attention are standardized again to prevent gradient explosion or gradient vanishing during training. Finally, these three vectors are concatenated to a length of 1500 and fed into a Bidirectional Long Short-Term Memory (BiLSTM) network. The parameters in the hidden layer are extracted and then processed through a fully connected layer to complete the classification, i.e., to complete the training of user recognition. The model parameters with the best recognition rate on the validation set are saved to obtain the optimized identity recognition model. The training process used a learning rate of 1e-5, a batch size of 1, a cross-entropy loss function, and an adaptive moment estimation (Adam) optimizer.

[0075] The beneficial effects of this invention are as follows: The contactless multimodal user identification method based on WiFi Channel State Information (CSI) proposed in this invention provides an efficient, privacy-preserving, and highly adaptable solution to the problems of privacy issues, line-of-sight limitations, high costs, and insufficient feature extraction in traditional identification methods. This method preprocesses the raw CSI data, including using a Hampel filter to detect and remove outliers, employing a median filter and a Butterworth filter for noise reduction, and mapping the data to a standard range (0 to 255) through linear normalization. Then, it extracts the statistical features of the amplitude and phase of the CSI data (standard deviation, range, median absolute deviation) and the Doppler frequency shift spectrum as physical features. A multi-branch user identification model (CSA-ViT model) is constructed, where the statistical feature branch uses lightweight convolution combined with a self-attention mechanism, the frequency domain branch uses a Vision Transformer to model the DFS image, and a cross-attention mechanism is used to weight and fuse the multimodal features. Finally, these three vectors are concatenated into a 1500-length vector and fed into a bidirectional long short-term memory network (BiLSTM). The parameters from the hidden layer are extracted and then processed through a fully connected layer to complete the classification, thus detecting the corresponding user identity. Experimental results show that this method achieves a recognition accuracy of 99.07% in indoor scenarios, significantly outperforming traditional methods, validating the effectiveness and robustness of the proposed model in multimodal WiFi feature modeling. This method not only overcomes the limitations of traditional methods but also improves recognition accuracy and generalization ability through multimodal feature fusion, providing new ideas and methods for the development of contactless identity recognition technology.

[0076] like Figure 3 As shown, the user identity recognition model CSA-ViT of the present invention is constructed by fusing convolutional neural networks (CNN), self-attention mechanisms, and visual Transformer (ViT); For the statistical features of amplitude and phase, a 3×3 convolution kernel is first used for convolution, then a self-attention mechanism is used to capture long-range dependencies in the sequence, and finally, two vectors of length 500 are obtained through a fully connected layer. For the physical features (Doppler spectrum), the Vision Transformer model is directly applied to process the input data, and then a vector of length 500 is obtained through a fully connected layer. These three variables are standardized individually. Next, cross-attention operations are performed on each of the three vectors pairwise. The three vectors after cross-attention are standardized again. Finally, these three vectors are concatenated to a length of 1500, fully connected, and then fed into a Bidirectional Long Short-Term Memory (BiLSTM) network. The parameters from the hidden layer are extracted and then processed through another fully connected layer to complete the classification, thus completing the user recognition training. The model parameters with the best recognition rate on the validation set are saved, resulting in the optimized identity recognition model. The training process used a learning rate of 1e-5, a batch size of 1, a cross-entropy loss function, and an adaptive moment estimation (Adam) optimizer.

[0077] Example 5 Figure 4 This is the confusion matrix of the detection results. The vertical columns represent the true labels, and the horizontal columns represent the predicted labels. The correct recognition rate is high for most categories, especially for categories B, C, E, F, G, H, I, and J, where the correct recognition rate reaches 100%. This indicates that the model performs exceptionally well in these categories. There are some misclassifications in category A, with 4% of samples incorrectly predicted as I. In category D, 6% of samples are incorrectly predicted as H. Although the percentage is low, it still needs attention to further improve the model's accuracy. Overall, the model performs very well in most categories, exhibiting high accuracy and stability. However, there are some misclassifications in categories A and D, requiring further optimization and improvement to achieve better classification results.

[0078] Example 6 Figure 5 This paper compares our model with several related identity recognition models (WiFi-ID, WFID, WiWho, and NeuralWave) in terms of the number of people and accuracy. It shows that compared to WiFi-ID, WFID, and WiWho, our model achieves a significant improvement in accuracy in experimental scenarios with a larger number of people. While NeuralWave also handles a larger number of people, its accuracy is not as high. Furthermore, compared to these traditional models, our experiment extracts more feature data and uses a multimodal data feature fusion method to make personnel identification more accurate.

[0079] Table 1 Performance metrics of the user identification model.

[0080]

[0081] The above is only one embodiment of the present invention. For those skilled in the art, any improvements and modifications made to the present invention without departing from the principle of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A non-contact multimodal user identification method based on WiFi signals, characterized in that, To address the differences in feature spaces across different modalities, a multi-branch user identification model based on fused convolution, self-attention mechanism, and visual transformer was designed to collect CSI data under different user breathing patterns. The CSI data is subjected to complex conjugate product, Hampel filter to remove outliers, data denoising, and subcarrier selection. Statistical and physical features are extracted from the preprocessed CSI data to construct a complete multimodal input vector. Then, the input vector is input into different branches according to feature categories, and the multimodal features are weighted and fused through a cross-attention mechanism to achieve identity recognition.

2. The contactless multimodal user identification method based on WiFi signals according to claim 1, characterized in that, The specific operating steps are as follows: Step 1: Set up a WiFi detection area with multiple antennas, a single transmitter, and a single receiver, and collect CSI data of different subjects under normal breathing patterns; Step 2: Preprocess the CSI data from Step 1, including complex conjugate product, Hampel filter to remove outliers, data denoising, and subcarrier selection; Step 3: Extract the statistical features of amplitude and phase, as well as the Doppler spectrum, from the preprocessed CSI data in Step 2 to construct a complete multimodal input vector; the statistical features include standard deviation, range, and median absolute deviation (MAD), and construct and divide the dataset into training set, validation set, and test set; Step 4: Input the training set divided in Step 3 into the multi-branch user identity recognition model, extract and fuse features through convolution, self-attention mechanism and visual Transformer, use cross-attention mechanism to perform weighted fusion of multimodal features, complete the training of identity recognition, and save the model parameters with the best recognition rate on the validation set to obtain the optimized identity recognition model. Step 5: Input the test set into the identity recognition model from Step 4 for classification, and finally obtain the prediction result.

3. The contactless multimodal user identification method based on WiFi signals according to claim 2, characterized in that, Step 1 is as follows: Step 1.1: Construct a multi-antenna single transmitter-single receiver WiFi detection area. The WiFi detection area includes a hardware platform consisting of a desktop computer powered by an Intel 5300 NIC and a wireless router equipped with three antennas; i.e., 3 transmitters and 3 receivers. The WiFi signal follows the IEEE 802.11n protocol standard, operates in the 2.4 GHz band, has a channel bandwidth of 40 MHz, and a data packet transmission frequency of 100 Hz. On the software side, a Linux system is used, and the open-source tool CSI-Tool is deployed to achieve real-time capture and recording of CSI data. Step 1.2: The user sits in a seated position at the midpoint of the line of sight between the transmitting and receiving antennas and breathes naturally. A horizontal distance of 2 meters is maintained between the transmitting and receiving antennas, and their height is set parallel to the user's abdomen to maximize the capture of breathing-related Channel State Information (CSI).

4. The contactless multimodal user identification method based on WiFi signals according to claim 2, characterized in that, Step 2 is as follows: Step 2.1, using the CSI data of different users' breathing collected in Step 1, CSI data is a... N A two-dimensional matrix of rows; CSI data uses a matrix Represented as: The number of antennas at the transmitting end. This refers to the number of antennas at the receiving end. For the first transmitting antenna The first antenna to the receiving antenna CSI data of the root antenna Represented as: The number of subcarriers in the CSI signal on the Intel 5300 wireless network card. The value is 30; as shown in step 1.1, both the receiver and transmitter are equipped with three antennas, therefore the received signal contains 9 data streams. The data contained in each data packet is explained as follows: Step 2.2: In order to identify the similarity and time delay between signals, complex conjugate is used to ensure the correct phase relationship when calculating the cross-correlation between two signals; Given two discrete complex signals x[n] and y[n], where n represents a discrete time index; for each time point n, the value of the signal is in complex form, i.e. and , where j is the imaginary unit; The conjugate product is defined at each time point n as: like Then its conjugate is ,as a result ; Based on the three transmitting antennas and three receiving antennas in step 1, for one transmitting antenna there are three receiving antennas receiving data, and there are three different signals x, y and z; let them be multiplied in pairs by complex conjugate to enhance the target signal and reduce the influence of environmental noise, that is, x and y, y and z, and z and x are multiplied in complex conjugate. Step 2.3 involves applying Hampel filtering to the CSI data after performing the complex conjugate product operation in Step 2.2 to handle outliers, and using the median absolute deviation (MAD) as a scaling estimator; the details are as follows: Within the selected window, first calculate the median of all data points. m Then, calculate the absolute deviation of each point from the median, and find the median of these absolute deviations, i.e., the median absolute deviation (MAD). For data points in the window, MAD is defined as follows: Secondly, determine the threshold and mark outliers: those exceeding the median... k Data points that are multiples of MAD are considered outliers; specifically, if a data point... x A value is considered an outlier if it meets the following conditions: in, k To be a multiple, take 3; For identified outliers, replace them with the median value within the window; Step 2.4 involves performing median filtering to denoise the CSI data after outlier handling in Step 2.3, and... High-frequency disturbances are suppressed by using a Butterworth low-pass filter; Steps 2.5 and 2.4 involve filtering subcarriers from the denoised CSI data.

5. The contactless multimodal user identification method based on WiFi signals according to claim 4, characterized in that, In step 2.5, dynamic time programming, singular value decomposition, and information entropy are used to screen subcarrier sequences with higher contributions, as detailed below: For the experimental equipment with 3 transmit and 3 receive antennas, each receiving antenna has 10 subcarriers, for a total of 30 subcarriers; The DTW distance is calculated pairwise for each of the 30 subcarriers and stored in a 30x30 matrix; finally, the distance from the nth subcarrier in the matrix is ​​taken. i The mean of the 30 DTW distances of the row is used to represent the first row. i The DTW distance of each subcarrier is calculated and normalized. Then, singular value decomposition is performed on the matrix storing the information of 30 subcarriers. The diagonal values ​​of the resulting matrix are taken as the singular value decomposition contribution rates of the 30 subcarriers and normalized. Finally, the amplitude and phase of each of the 30 subcarriers are extracted and their information entropy is calculated. The information entropy of the amplitude and phase are added together and the negative value is taken. Finally, the information entropy of the 30 subcarriers is normalized. After calculating the above three indicators, the DTW distance, singular value decomposition contribution rate, and information entropy of the 30 subcarriers after normalization were obtained; for these three indicators, weight parameters were assigned respectively and they were added together. in, D For DTW distance, S Contribution rate to singular value decomposition I For information entropy, alpha The weight for the DTW distance is set to 0.4; beta The weight for the contribution rate of singular value decomposition is set to 0.4; gamma The information entropy weight is 0.2, and the singular value decomposition is 0.2; finally, the 10 subcarriers with the best scores are selected.

6. The contactless multimodal user identification method based on WiFi signals according to claim 5, characterized in that, Step 3 is as follows: Step 3.1: Extract the statistical features of amplitude and phase from the preprocessed CSI data in Step 2. The statistical features include standard deviation, range, and median absolute deviation (MAD). For a sequence {x1,x2,...,xn} with n values, the formula for calculating its sample standard deviation is as follows: The Median Absolute Deviation (MAD) measures the statistical dispersion of a dataset. Similarly, for a sequence of n values ​​X = {x1, x2, ..., xn}, the MAD is calculated as follows: The range is used to reflect the range of variation or fluctuation of data. Similarly, for a sequence of n values ​​X={x1,x2,...,xn}, the formula for calculating the range is as follows: in, The maximum value in X. The minimum value in X; Step 3.2: Extract the Doppler spectrum of the signal from the preprocessed CSI data in Step 2; The Doppler frequency shift spectrum is obtained by short-time Fourier transform for discrete signals. The short-time Fourier transform formula is: in, m It is the first m One time frame, k It is the first k Each frequency point, M It is frame shift, that is, the difference in the number of samples between each frame. N It is the window length, i.e., the number of samples in each frame; It is a window function; Step 3.3: Combine the statistical characteristics of amplitude and phase of each CSI data point with the Doppler spectrum into a data set; then, repeat steps 3.1 and 3.2 to construct the complete dataset; finally, divide the dataset according to the ratio of training set:validation set:test set = 7:1.5:1.

5.

7. The contactless multimodal user identification method based on WiFi signals according to claim 6, characterized in that, The multi-branch user identification model described in step 4 is built on self-attention and the VisionTransformer. This model processes these inputs through different branches and then fuses the processed features to achieve the final classification task.

8. The contactless multimodal user identification method based on WiFi signals according to claim 7, characterized in that, In step 4, the training set divided in step 3.3 is used as the input of the user recognition model. For the statistical features of amplitude and phase, a 3x3 convolution kernel is first used for convolution operation, then a self-attention mechanism is used to capture long-distance dependencies in the sequence, and finally two vectors of length 500 are obtained through a fully connected layer. For the Doppler spectrum, the Vision Transformer is directly applied to model the Doppler spectrum image, extract time-frequency mode information, and then a vector of length 500 is obtained through a fully connected layer; The amplitude, phase, and Doppler spectrum vectors are standardized individually. Then, a cross-attention operation is performed on each pair of these three vectors, and the resulting three vectors are standardized again. Finally, these three vectors are concatenated into a 1500-dimensional vector, fully connected, and then fed into a Bidirectional Long Short-Term Memory (BiLSTM) network. The parameters from the hidden layers are extracted and processed through another fully connected layer to complete the classification, thus completing the user recognition training. The model parameters that achieve the best recognition rate on the validation set are saved, resulting in the optimized identity recognition model. The training process used a learning rate of 1e-5, a batch size of 1, a cross-entropy loss function, and an adaptive moment estimation (Adam) optimizer.

9. The contactless multimodal user identification method based on WiFi signals according to claim 8, characterized in that, The test set obtained in step 3.3 is input into the user identification model obtained in step 4 for testing, classification is achieved, and the prediction result is finally obtained.

Images