Human body sensing system and method based on WiFi signal

Abstract

The invention discloses a human body sensing system and method based on a WiFi signal, and the method comprises the steps: 1, carrying out the initialization configuration of an ESP32-S3 module, sending a collection request to a router, and collecting the Wi-Fi CSI information related to the human body activity through a signal collection module; 2, processing data of the acquired Wi-Fi CSI information related to human body activities according to a sequence of data reading, subcarrier screening, amplitude calculation and preprocessing, filtering, denoising and normalization through a signal processing module; 3, performing multi-dimensional feature extraction on the processed Wi-Fi CSI information related to the human body activity through a feature extraction module, and extracting features related to the human body activity; 4, the extracted features are classified and recognized through a GRU model of a target recognition module, human body sensing information of the existence state and the action type of the human body is judged according to the change of the Wi-Fi CSI information features, and human body activity classification based on the WiFi signals is achieved. According to the invention, high-precision, low-time-delay, low-cost and high-privacy-protection human body state detection is realized.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of human perception system, and particularly relates to a human perception system and method based on WiFi signal. BACKGROUND

[0002] The vigorous development of Internet of Things technology promotes the intelligent home and public safety industry to deep transformation towards intelligence and automation, and there is an urgent need for high-precision, non-invasive human perception technology. For example, in the field of intelligent home and public safety, human perception is the core link of automation control. Traditional infrared sensors are easily disturbed by environmental factors and are difficult to accurately capture the micro-motion state of the human body in high-temperature and strong light environments; the camera solution has the risk of privacy leakage, and the cost of equipment procurement, installation and maintenance is high, and there are also visual blind areas, making it difficult to achieve global coverage.

[0003] Based on the advantages of wireless sensing technology without additional wiring and flexible deployment, WiFi-based wireless sensing technology has become a research hotspot in the field of human perception. However, the existing technology still has obvious shortcomings in terms of adaptability to complex environments and multi-target processing capability, which restricts the large-scale commercial application of this technology. Among them, the scheme relying on received signal strength indication (RSSI) is significantly affected by environmental object shielding and electromagnetic interference, making it difficult to distinguish subtle human movements; the technology based on channel state information (CSI) has higher resolution, but the data processing complexity is high, which cannot meet the real-time response requirements of intelligent home, and in the multi-target scene, the cross interference of target signals leads to insufficient detection accuracy.

[0004] For example, in the prior art, Ashraf Tahat et al. published a paper titled "A Lookat the Recent Wireless Positioning Techniques With a Focus on Algorithms for Moving Receivers" in IEEE Access. This paper uses RSSI intensity changes for target detection. While this method is simple, it is significantly affected by environmental factors and has a high false positive rate. Another example is Jiaqi Geng et al.'s paper titled "DensePose From WiFi." This method purifies CSI signals, converts them into 2D feature maps through modality conversion, and then uses the WiFi-DensePose RCNN architecture to generate a 3D human model. This method has some effectiveness under the same layout, but its generalization ability is weak, making it prone to misjudgment in different layouts or multi-person scenarios. Furthermore, data processing relies on high-performance computing equipment, making it difficult to deploy on low-power terminal devices. Another example is Yang Zhang et al.'s paper titled "WiFi-based non-contact human presence detection technology" published in Scientific Reports. This method uses a network card with modified firmware to collect CSI, then performs low-pass filtering and wavelet transform noise reduction, then extracts features using a self-organizing neural network, and finally classifies using a softmax classifier. In laboratory and conference room testing, detection accuracy was high and demonstrated stability across diverse environments and human body conditions. However, this technology can only detect the presence of a human in a room and cannot identify specific movements or the simultaneous actions of multiple people. Furthermore, it requires hardware modifications to terminal devices, resulting in poor compatibility and limited application scenarios.

[0005] To summarize, the core flaws of existing technologies include: poor environmental adaptability, insufficient real-time performance, high cost, and difficulty in balancing privacy and compatibility. Summary of the Invention

[0006] In order to overcome the shortcomings of the above-mentioned existing technologies, the purpose of the present invention is to provide a human perception system and method based on WiFi signals to achieve high-precision, low-latency, low-cost, and high-privacy protection human state detection to meet the requirements of scenarios such as automatic control of smart home devices and public safety abnormal behavior warning.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is:

[0008] A human body perception system based on WiFi signals, including a signal acquisition module, a signal processing module, a feature extraction module and a target recognition module;

[0009] The signal acquisition module is used for real-time acquisition of original WiFi signal data.

[0010] The signal processing module is used for preprocessing of the collected original WiFi signal.

[0011] The feature extraction module is used for extracting features related to human activity from the WiFi signal processed by the signal processing module.

[0012] The target recognition module is used for recognizing and classifying human activity according to the feature vector output by the feature extraction module.

[0013] The signal acquisition module collects original WiFi signals changed by human activity in the environment through the ESP32-S3 module arranged in a specific area, wherein the ESP32-S3 module establishes a wireless connection with a commercial router using IEEE 802.11n protocol and realizes real-time transmission of CSI data through UDP protocol, providing basic data for subsequent processing.

[0014] The specific area refers to a target area that needs human perception, such as the interior of a room in a smart home environment or a specific monitoring range in a public security scenario. For the arrangement of ESP32-S3 modules, it is generally recommended to arrange them in positions that can fully cover the target area and have stable WiFi signals. In a small room, arranging 1-2 modules can usually meet the basic signal acquisition requirements; while in larger spaces or complex structures, 3 or more modules may be needed to ensure effective signal coverage. The number and position of arrangements are closely related to the acquisition capability, and reasonable increase in the number of arrangements and optimization of the arrangement position can expand the signal acquisition range, improve the ability to capture changes in WiFi signals caused by human activity, and thus improve the perception accuracy and reliability of the system.

[0015] The ESP32-S3 module is a complete product purchased, and the specific model is YD-ESP32-S3 development board produced by Source Studio. After the ESP32-S3 module establishes a connection with the commercial WiFi router, it triggers the data acquisition process by sending a Ping packet. When the router replies to the Ping packet, the terminal WiFi module uses its own hardware capture unit to accurately extract the CSI data in the packet.

[0016] The signal processing module selects effective subcarrier signals through a subcarrier screening algorithm, removes noise interference in the signal using median filtering and bidirectional Butterworth filtering, and finally reduces the data dimension and reduces redundant information through principal component analysis (PCA) denoising, improving the signal quality and providing clean and effective data for feature extraction.

[0017] The subcarrier signal is a subdivision of the WiFi signal in the frequency domain. Under the 802.11n protocol, the WiFi signal is divided into multiple subcarriers, including pilot subcarriers and data subcarriers. The CSI (Channel State Information) data is a description of the state of the WiFi signal after passing through the channel during transmission, which is constructed by collecting the amplitude and phase information of the signal on each subcarrier. In the present application, the collected WiFi signal extracts the CSI data from the router response packet through the ESP32-S3 module, and the CSI data contains the real and imaginary part information of each subcarrier, reflecting the subtle disturbance characteristics of the WiFi signal caused by human activity, such as signal amplitude change, phase shift, etc. Through analysis and processing of the CSI data, information related to human activity is mined. In code implementation, such as through the get_data_subcarriers function, valid data subcarriers are filtered according to the IEEE 802.11 protocol, and the set of all subcarrier indices is S, the set of pilot subcarrier indices is P, then the set of data subcarrier indices D = S-P, thereby realizing the filtering of valid subcarrier signals in the CSI data and providing a basis for subsequent processing.

[0018] The feature extraction module mines the feature vector that can represent human action and position information by analyzing the amplitude information in the CSI data, and converts the original WiFi signal data into effective features for target recognition.

[0019] The CSI data is divided into groups of two elements each, representing the real and imaginary part information of each subcarrier. Each group of elements is set to form a vector x = [x1, x2, …, x n ], and the Euclidean norm of each group of elements is calculated, that is to obtain the amplitude value corresponding to each subcarrier;

[0020] In actual data processing, the amplitude value sequence calculated by the design is A = [A1, A2, …, A m ], the set of valid data subcarrier indices filtered is S = {s1, s2, …, s 48}, and the amplitude of the corresponding subcarrier is extracted from the amplitude value sequence A to form a feature vector

[0021] When the set valid subcarrier index is acquired, the amplitude value at the corresponding position in the amplitude value sequence is selected to form a feature vector used for model training and recognition. The target recognition module adopts a GRU neural network model, and the structure parameters thereof are as follows: the input layer receives a feature matrix with a receiving dimension of (x, y), x time steps x y-dimensional subcarrier features; the hidden layer includes 2 GRU units, adopts a tanh activation function, and is provided with a dropout regularization between layers; the output layer outputs a probability distribution of different activity states through a softmax activation function, an Adam optimizer is adopted for training, a sparse categorical crossentropy is selected as a loss function, and through learning and training of a large amount of sample data, different activity states of the human body, such as walking, sitting and standing, can be accurately judged, and the human body perception function based on the WiFi signal is realized.

[0022] A human body perception method based on a WiFi signal, comprising the following steps:

[0023] Step 1: The ESP32-S3 module is initialized and configured to send a collection request to the router at a certain frequency, and the Wi-Fi CSI information related to human activity is collected through the signal collection module; these information will be used as the original data basis for judging the human body perception information such as the existence state and the action type of the human body in the future;

[0024] Step 2: The Wi-Fi CSI information related to human activity collected is processed in the order of data reading, subcarrier screening, amplitude calculation and preprocessing, filtering, denoising and normalization through the signal processing module; so as to improve the signal quality and data effectiveness, and prepare for the subsequent human activity feature extraction;

[0025] Step 3: The Wi-Fi CSI information related to human activity processed is subjected to multi-dimensional feature extraction through the feature extraction module, and the features related to human activity are extracted, such as signal fluctuation amplitude, frequency change and the like, which will be directly used for classification and recognition of human activity;

[0026] Step 4: The extracted features are classified and recognized through the GRU model of the target recognition module, the existence state and the action type of the human body are judged according to the change of the Wi-Fi CSI information features, the human body perception information is realized, and the human body activity classification based on the WiFi signal is realized.

[0027] In step 1, the signal collection module collects Wi-Fi CSI data containing human activity information, and the specific collection steps are as follows:

[0028] Hardware initialization: Initialize the ESP32-S3 module, configure it to work in STA mode, set the WiFi connection parameters (including router SSID and password), and initialize the serial communication interface (CH343P-USB to TTL) for data upload;

[0029] Establish connection: ESP32-S3 module establishes WiFi connection with commercial router, completes identity authentication and network access;

[0030] Data request: ESP32-S3 module sends Ping packet (ICMP EchoRequest) to router at a frequency of 100Hz, triggering the router to generate a response signal; this signal will carry the environmental interference information generated by human activity;

[0031] Data capture: When the router returns the Ping response packet, the ESP32-S3 module uses its built-in WiFi hardware capture unit to extract CSI data (including the real and imaginary parts of each subcarrier) from the response packet; these CSI data contain the subtle disturbance characteristics of WiFi signal caused by human walking, waving, sitting, etc. Different activity states such as signal amplitude change, phase shift, etc.

[0032] Data caching and uploading: The captured CSI data is temporarily stored in the external PSRAM of the ESP32-S3 module, and then uploaded to the host computer in CSV format through the serial port in real time; the host computer stores the data and associates it with the corresponding human activity scenario, and records the time stamp for subsequent human activity classification model training to provide data basis;

[0033] Exception handling: During data collection, if the router does not respond for 5 consecutive times or data transmission errors occur, the module will trigger the RST button to reset and restart, and re-execute the above collection process to ensure the continuity and integrity of human activity data collection.

[0034] In step 2, the signal processing module processes the original WiFi signal data to improve data quality and provide support for subsequent steps, including data reading and analysis, effective subcarrier selection, amplitude calculation and preprocessing, filtering, principal component analysis (PCA) denoising, and normalization processing.

[0035] First, data reading and analysis are performed to convert the original CSV data into a processable format and remove invalid data to provide a basis for subsequent processing;

[0036] Next, effective subcarrier selection is performed to filter out non-pilot subcarriers from the analyzed data, reducing data volume and removing irrelevant information;

[0037] Then the amplitude calculation and pretreatment are carried out, the amplitude value is calculated based on the screened effective subcarriers, and the CSI data in complex form is converted into amplitude information which is easier to process;

[0038] Then filtering processing is carried out, first, the median filtering is used to remove impulse noise, and then the bidirectional Butterworth filtering is used to suppress high-frequency noise, so as to further purify the signal; the filtered signal enters the PCA denoising link, reduces the data dimension and removes the redundant noise components;

[0039] Finally, normalization processing is carried out, the data is standardized to the interval [0, 1], the dimension influence is eliminated, and standardized data input is provided for feature extraction and model training.

[0040] The specific processing steps are as follows:

[0041] Data reading and analysis: the ESP32-S3 module deployed in the monitoring area collects WiFi signals containing human activity information, connects with the host computer and stores the original data in CSV format, then the pd.read_csv function is used to read the original data in CSV format, the data is stored in DataFrame, for the "data" column containing CSI data, the parse_data function is used for analysis, the string is converted into a list by using ast.literal_eval, and the effectiveness is screened according to the data length, the data that does not meet the requirements is marked as np.nan; the "timestamp" column is converted into date and time format, and the data rows with parsing errors are removed;

[0042] Effective subcarrier screening: according to the IEEE 802.11 protocol, the get_data_subcarriers function is used to screen effective data subcarriers, the set of all subcarrier indexes is S, the set of pilot subcarrier indexes is P, and then the set of data subcarrier indexes D = S-P is obtained;

[0043] Amplitude calculation and pretreatment: the parsed CSI data is reshaped into a (x, y) two-dimensional matrix, for each subcarrier i, the amplitude value is calculated by the formula (where Re i and Im i are the real part and imaginary part of the i-th subcarrier respectively), and the amplitude of the effective subcarrier is extracted to form the amplitude matrix A;

[0044] Filtering processing: the median filtering and bidirectional Butterworth filtering are combined, the median filtering is used to process the signal x(t) with a window size w = 3, and the formula is:

[0045]

[0046] Bi-directional Butterworth filtering according to the sampling rate f s = 100 Hz, the cut-off frequency f c = 20 Hz and the order n = 4, generate filter coefficients, forward and backward filtering of the median filtered signal, suppress noise;

[0047] Principal Component Analysis (PCA) denoising: perform PCA denoising using the PCA Denoiser class, set the amplitude matrix as X, transpose it and convert it through PCA: Y = PCA (X T ), set the first principal component of the converted data to zero, and reconstruct the signal through inverse conversion: (where Y' is the modified PCA conversion result);

[0048] Normalization processing: for the signal matrix Z after PCA denoising, normalize each column data z j

[0049]

[0050] where Z represents the signal matrix after PCA denoising; z j represents the jth column data of the matrix Z; min(z j ) is the minimum value in the jth column data; max(z j ) is the maximum value in the jth column data; is the jth column data after normalization processing.

[0051] In step 3, the feature extraction module extracts features from the preprocessed signal data; for CSI-based signals, the effective data subcarrier information processed by the signal processing module is used for feature extraction;

[0052] From the preprocessed CSI data, extract its amplitude information as the key feature; specifically, divide the CSI data into groups of two elements each, representing the real and imaginary parts of each subcarrier. Assume that each group of elements forms a vector x = [x1, x2, …, x n , calculate the Euclidean norm of each group of elements, that is to get the amplitude value corresponding to each subcarrier;

[0053] In actual data processing, the designed amplitude value sequence is A = [A1, A2, …, A m , and the selected effective data subcarrier index set is S = {s1, s2, …, s 48 ​}(selected by the get_data_subcarriers function, excluding 4 pilot subcarriers), the amplitudes of the corresponding subcarriers are extracted from the amplitude value sequence A to form a feature vector

[0054] When the set valid subcarrier index is obtained, the amplitude value at the corresponding position in the amplitude value sequence is selected to form a feature vector used for model training and recognition, which will be subsequently input into the GRU neural network model of the target recognition module for judging the presence state, action type and other information of the human body.

[0055] In step 4, the target recognition module uses a GRU neural network model to realize classification and recognition of the human activity state.

[0056] The input layer of the GRU neural network model receives the feature vector output by the feature extraction module. Considering that the human activity signal has time series characteristics, the input shape is set to (x, y), that is, it contains x time steps, and each time step corresponds to the amplitude features of y valid subcarriers.

[0057] The GRU neural network model contains two layers of N GRU units. The weights are initialized in the "he_normal" mode. The first layer of GRU units returns a sequence, which is used to capture the long-term dependence of human actions in the time dimension, such as the periodic variation of walking actions. The second layer of GRU units no longer returns a sequence, but abstracts and refines the features. Between the two layers of GRU units and between the hidden layer and the output layer, there are Dropout layers to effectively prevent overfitting of the model during training and improve the generalization ability of the model. The output layer contains M neurons, which are activated by a linear rectifier function (ReLU) to enhance the model's ability to express complex features. Finally, through a fully connected layer with the corresponding number of classifications, combined with the softmax activation function, the classification results of human activities are output.

[0058] Advantages of the invention:

[0059] Detection accuracy: The existing RSSI threshold detection method is easily affected by environmental interference, has a high false positive rate in complex scenes, and is difficult to distinguish subtle human actions. This invention removes noise interference by median filtering and bidirectional Butterworth four-order low-pass filtering on CSI data, and then selects 48 valid subcarriers and performs PCA denoising. The Euclidean norm of the real and imaginary parts of each subcarrier The amplitude dynamic feature is extracted as a core input, a two-layer N-unit GRU neural network with a Dropout layer is combined to deeply mine the time sequence feature, subtle changes of the human activity state can be accurately captured, whether it is a complex indoor human activity scene or a through-wall special environment, the recognition accuracy can be effectively improved, and compared with the traditional method, the accuracy of human motion recognition is greatly improved.

[0060] Data processing efficiency: the traditional machine learning method based on CSI is difficult to realize real-time response due to complex data processing and high requirement for computing resources. The application optimizes the signal processing process, adopts a composite filtering algorithm and a subcarrier screening strategy to reduce the data processing amount; meanwhile, a lightweight GRU neural network model is designed to reduce the calculation complexity on the premise of ensuring the recognition accuracy, effectively shorten the data processing time and meet the real-time demand in actual application.

[0061] Cost control: the hardware enhancement scheme often needs to modify the WiFi router or terminal device, resulting in a substantial increase in device cost and poor compatibility. The application constructs a system based on a commercial router and an ESP32-S3 module, without the need for additional modification of hardware, greatly reducing the system construction cost, and at the same time, having good compatibility and being suitable for a variety of commercial devices on the market.

[0062] Privacy protection: the traditional camera scheme has a serious risk of privacy leakage, and some existing WiFi sensing technologies may also involve user privacy data collection. The application is based on WiFi signal analysis only, without involving any image or privacy information collection, and uses a pure signal processing technology to realize human activity sensing, fundamentally eliminating the risk of privacy leakage and meeting the strict requirements of users for privacy protection. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1 It is a schematic diagram for the signal acquisition module in the application.

[0064] Figure 2 It is a schematic diagram of human motion direction in different test environments in the application.

[0065] Figure 3 It is a processing process of a signal processing module for a received signal, which is a time domain original signal, a filtered and denoised signal, and a normalized signal.

[0066] Figure 4 It is the recognition result of different actions of a target recognition module; (a) classification confusion matrix, (b) recognition accuracy. (mo - multiple people walking, no - no one, st - single person standing, th - walking through the wall, wa - single person walking). DETAILED DESCRIPTION

[0067] The application will be further described in detail below with reference to the drawings.

[0068] The human body sensing system based on WiFi signals provided by the application is composed of a signal acquisition module, a signal processing module, a feature extraction module and a target recognition module.

[0069] The signal acquisition module is used for real-time acquisition of WiFi signal data.

[0070] The signal processing module is used for pre-processing of the acquired original WiFi signal.

[0071] The feature extraction module is used for extracting features related to human activities from the WiFi signal processed by the signal processing module.

[0072] The target recognition module is used for recognizing and classifying human activities according to the feature vector output by the feature extraction module.

[0073] As shown in Figure 1 The signal acquisition module is composed of a terminal WiFi module and a commercial WiFi router. The terminal WiFi module adopts an ESP32-S3 module, which is equipped with an Xtensa dual-core 32-bit LX7 CPU with a main frequency of 240MHz, which can efficiently execute instructions during data acquisition and preliminary processing. In terms of storage, the built-in SRAM 512K can quickly cache temporary data, the built-in ROM 384K is used to store the basic program, and the externally expanded PSRAM 8M and externally expanded FLASH 16M meet the storage needs of a large amount of CSI data. The PCB antenna equipped with the terminal WiFi module supports the 2.4GHz frequency band, and data transmission with the host computer is realized through ESP32-S3 direct connection USB or CH343P-USB to TTL. In the actual debugging process, the collected data can be conveniently transmitted to the computer for analysis. The RST and BOOT keys on the terminal WiFi module are convenient for resetting operation when the program is abnormal, and entering a specific mode when programming, and the working voltage of 3V-3.6V makes it can adapt to a variety of power adapters. After the terminal WiFi module and the commercial WiFi router are connected, the data acquisition process is triggered by sending a Ping packet. When the router replies to the Ping packet, the terminal WiFi module uses its own hardware capture unit to accurately extract the CSI data in the packet.

[0074] The signal processing module: The signal processing module processes the original WiFi signal data to improve the data quality and provide support for the subsequent links, and its main processing steps are as follows:

[0075] Data reading and parsing: The raw data in CSV format is read by the pd.read_csv function and stored in a DataFrame. For the "data" column containing CSI data, the parse_data function is used for parsing, converting the string to a list using ast.literal_eval, and filtering the data based on its length. Data that does not meet the requirements is marked as np.nan. At the same time, the "timestamp" column is converted to date and time format, and data rows with parsing errors are removed.

[0076] Effective subcarrier screening: According to the IEEE 802.11 protocol, the get_data_subcarriers function is used to screen effective data subcarriers. Let S be the set of all subcarrier indices, P be the set of pilot subcarrier indices, then D = S - P is the set of data subcarrier indices.

[0077] Amplitude calculation and preprocessing: The parsed CSI data is reshaped into a (x, y) two-dimensional matrix. For each subcarrier i, the amplitude value is calculated using the formula (where Re i and Im i are the real and imaginary parts of the i-th subcarrier, respectively), and the amplitudes of the effective subcarriers are extracted to form the amplitude matrix A.

[0078] Filtering: A combination of median filtering and bidirectional Butterworth filtering is used. The median filtering is performed on the signal x(t) with a window size w = 3, and the formula is: Bidirectional Butterworth filtering: According to the sampling rate f s = 100 Hz, the cutoff frequency f c = 20 Hz, and the order n = 4, the filter coefficients are generated, and the median filtered signal is forward and backward filtered to suppress noise.

[0079] Principal Component Analysis (PCA) denoising: The PCADenoiser class is used to perform PCA denoising. Given the amplitude matrix X, it is transposed and converted by PCA: Y = PCA(X T ), the first principal component of the converted data is set to zero, and the signal is reconstructed by inverse transformation: (where Y' is the modified PCA conversion result).

[0080] Normalization: For the signal matrix Z after PCA denoising, each column data z j is normalized:

[0081]

[0082] Through the above processing, high-quality signal data is obtained, which provides a basis for subsequent feature extraction and target recognition.

[0083] Feature extraction module: The feature extraction module extracts features from the preprocessed signal data. For CSI-based signals, the effective data subcarrier information processed by the signal processing module is used for feature extraction. From the preprocessed CSI data, the amplitude information is extracted as the key feature. Specifically, the CSI data is divided into groups of two elements, representing the real and imaginary parts of each subcarrier. By calculating the Euclidean norm of each group of elements, the amplitude value corresponding to each subcarrier is obtained. In actual data processing, the effective data subcarrier index selected is used to extract the amplitude of the corresponding subcarrier from the calculated amplitude value sequence to form a feature vector. For example, when the set of effective subcarrier indexes is obtained, the amplitude values at the corresponding positions in the amplitude value sequence are selected to form the final feature vector used for model training and recognition. These feature vectors are subsequently input into the GRU neural network model of the target recognition module to determine the presence state, action type, and other information of the human body.

[0084] Target recognition module: The target recognition module uses a GRU neural network model to realize the classification and recognition of human activity states. The input layer of the model receives the feature vectors output by the feature extraction module. Considering the time series characteristics of human activity signals, the input shape is set to (x, y), which includes x time steps, and each time step corresponds to y amplitude features of effective subcarriers. The model contains two layers of N-unit GRU layers, and the weights are initialized using the "he_normal" method. The first layer of GRU units returns a sequence, which is used to capture the long-term dependence of human actions in the time dimension, such as the periodic variation of walking actions. The second layer of GRU units no longer returns a sequence, but abstracts and refines the features. Between the two layers of GRU units and between the hidden layer and the output layer, Dropout layers are set to effectively prevent overfitting of the model during training and improve the generalization ability of the model. The output layer contains M neurons, which use the linear rectifier function (ReLU) activation function for nonlinear transformation to enhance the model's ability to express complex features. Finally, through a fully connected layer with a corresponding number of classifications, combined with the softmax activation function, the classification results of human activity are output.

[0085] Method flow:

[0086] The signal acquisition module sends a collection request to the router at a frequency of 100Hz, the signal processing module processes the data in the order of outlier rejection, filtering, normalization, and subcarrier selection, the feature extraction module extracts multi-dimensional features and inputs them into the target recognition module, and the GRU model performs classification and recognition to determine the presence state, action type, and other information of the human body.

[0087] 1. Selection of terminal device: The present application preferably uses ESP32-S3 module, but ESP32-C3 module is also feasible, as long as the device has similar WiFi signal collection and basic processing capability, it can be used as an alternative.

[0088] 2. Selection of filtering algorithm: The present application uses median filtering and bidirectional Butterworth four-order low-pass filtering, and the combination of Gaussian filtering and mean filtering can also achieve signal denoising. Gaussian filtering can effectively suppress Gaussian noise, and mean filtering can smooth the signal. The combination of the two can remove noise interference while preserving signal characteristics. As long as the filter combination can improve signal quality, it can be used as an alternative solution.

[0089] 3. Subcarrier screening strategy: The present application screens subcarriers according to IEEE 802.11 protocol, and can also use a screening strategy based on signal-to-noise ratio (SNR). By calculating the signal-to-noise ratio of each subcarrier, the subcarriers with higher signal-to-noise ratio are preferentially retained, which can effectively reduce interference and does not affect subsequent feature extraction and target recognition. It can be used as an alternative method for subcarrier screening.

[0090] 4. CSI signal feature extraction method: The present application extracts the amplitude information of the CSI signal, and can also extract the frequency offset feature of the signal. By analyzing the frequency offset of the WiFi signal under the influence of human activity, combined with the amplitude information, it can also represent the human activity state, and effectively identify the human activity.

[0091] 5. Selection of neural network model: Although GRU neural network is the preferred model of the present application, LSTM (Long Short Term Memory Neural Network) can also achieve the same function. LSTM can process input sequences from both forward and reverse directions, making full use of context information. In complex human motion recognition scenarios, as long as the feature vectors can be effectively classified, they can be used as alternative solutions.

[0092] 6. Application of machine learning algorithm: XGBoost (Distributed Gradient Boosting Library) algorithm integrates multiple weak learners through gradient boosting, which has good performance and generalization ability when dealing with small sample and high-dimensional data. Using XGBoost algorithm instead of neural network model to classify the extracted feature vectors can also achieve the purpose of recognizing human activity state.

[0093] Example 1

[0094] In the data collection link, set up two core test environments of indoor and through-wall:

[0095] Indoor test: The distance between the signal transmitting end (TX) and the receiving end (RX) is fixed at 3m, simulating various human activity scenarios. Among them, the single person static scene covers standing, sitting, and supine three typical postures; the single person walking is set up two different direction paths, such as Figure 2 (a) shows; two volunteers are arranged in the same area at different speeds, such as Figure 2 (b) shows, the WiFi signal data of different human body states are collected.

[0096] Wall test: TX and RX are respectively arranged on the two sides of a wall with a thickness of 30 cm, and the distance from the wall is 1.2 m, and the signal of four different positions of human body activity is collected in this environment, such as Figure 2 (c) shows, and the influence of the wall on the WiFi signal and human perception is explored.

[0097] For the data collected in the embodiment 1, the functions such as pd.read_csv are used to complete data reading, analysis and subcarrier selection, and the processed signals are subjected to median filtering, Butterworth filtering, PCA denoising and normalization, and the processing results are as shown in Figure 3 Subsequently, the CSI data amplitude feature vector is extracted, and input into the GRU neural network model. The model contains two layers of 128-unit GRU layers and Dropout layers, and the classification results are output through ReLU activation, full connection layer and softmax function, and the training is completed through the Adam optimizer, and the model confusion matrix is obtained, and the results are as shown in Figure 4 The proposed model can realize human activity state judgment with an identification accuracy of 98%.

[0098] In the automatic control of smart home devices, the human perception system and the smart home control system are integrated. When the system detects the "single person stationary" state in the room (such as the user sitting on the sofa), the light brightness can be automatically adjusted to a soft mode, the TV volume can be reduced or the energy-saving mode of the air conditioner can be turned on; when the "single person walking" state is detected, the light system can be linked to realize the effect of turning on the light when a person comes and turning off the light when a person leaves, and the environmental parameters (such as temperature and humidity) of the corresponding area can be adjusted in advance according to the walking path; if the "no one" state is detected, the system can automatically turn off unnecessary electrical equipment (such as lights, televisions and air conditioners) to achieve energy saving. In addition, for the elderly family, the system can identify the "walk through the wall" state to determine whether the old people are active in the room, and if no activity is detected for a long time, the reminding function can be triggered to notify the family.

[0099] In the aspect of public security abnormal behavior early warning, the system is deployed in public places such as shopping malls, office buildings, nursing homes, etc. When detecting the "multiple people moving" state anomaly (such as sudden gathering, rapid running), an early warning signal can be sent to the security system to prompt the security personnel to pay attention to whether an emergency has occurred in the area; for places such as nursing homes, the system can accurately identify whether the "single person stationary" state of the elderly is abnormal (such as long-time stationary may be a fall or illness), and once an anomaly is found, an alarm is triggered immediately and medical personnel are notified; in some areas where entry is prohibited, if the system detects "single person moving" or "multiple people moving" state, an alarm can be issued in time and the monitoring system can be linked to track the area, preventing safety accidents from occurring. At the same time, since the system is based on WiFi signal non-contact sensing, it avoids the privacy problems brought by camera monitoring, and is suitable for popularization and application in various public and private places.

Claims

1. A human sensing system based on WiFi signals, characterized in that, The system comprises a signal acquisition module, a signal processing module, a feature extraction module and a target recognition module. The signal acquisition module is used for real-time acquisition of original WiFi signal data. The signal processing module is used for preprocessing of the acquired original WiFi signal. The feature extraction module is used for extracting features related to human activities from the WiFi signal processed by the signal processing module. The target recognition module is used for identifying and classifying human activities according to the feature vector output by the feature extraction module.

2. The human perception system based on WiFi signals according to claim 1, characterized in that, The signal acquisition module collects original WiFi signals changed by human activities in the environment through ESP32-S3 modules arranged in a specific area, wherein the ESP32-S3 modules establish wireless connection with commercial routers using IEEE 802.11n protocol and realize real-time transmission of CSI data through UDP protocol. The specific area refers to a target area requiring human perception.

3. The human perception system based on WiFi signals according to claim 1, characterized in that, The signal processing module selects effective subcarrier signals through a subcarrier screening algorithm, removes noise interference in the signals by using median filtering and bidirectional Butterworth filtering, and finally denoises by principal component analysis (PCA). The collected WiFi signals are extracted from the CSI data in the router response packet by the ESP32-S3 module, and the CSI data contains real and imaginary part information of each subcarrier, reflecting the subtle disturbance characteristics of human activities on WiFi signals. In code implementation, effective data subcarriers are selected according to IEEE 802.11 protocol, and the index set of all subcarriers is S, the index set of pilot subcarriers is P, and the index set of data subcarriers is D = S-P. The CSI data is divided into groups of two elements, representing the real and imaginary information of each subcarrier, and each group of elements is set to form a vector x = [x1, x2, …, xn], and the Euclidean norm of each group of elements is calculated, that is n The amplitude value corresponding to each subcarrier is obtained ​ In actual data processing, the amplitude value sequence designed and calculated is A = [A1, A2, …, A m ], the screened effective data subcarrier index set is S = {s1, s2, …, s 48}, the amplitudes of the corresponding subcarriers are extracted from the amplitude value sequence A to form a feature vector The feature extraction module analyzes the amplitude information in the CSI data to extract feature vectors representing human actions and position information, and converts the original WiFi signal data into effective features for target recognition.

4. The human-aware system based on WiFi signals according to claim 1, wherein, After obtaining the set of effective subcarrier indices, the amplitude values at the corresponding positions are selected from the amplitude value sequence to form the feature vector used for model training and recognition.

5. The method of awareness of a human-aware system based on WiFi signals according to any of claims 1-4, characterized in that, The target recognition module uses a GRU neural network model with the following structure parameters: the input layer receives a feature matrix with dimensions (x, y), x time steps x y-dimensional subcarrier features; the hidden layer contains 2 GRU units, uses a tanh activation function, and sets dropout regularization between layers; the output layer outputs the probability distribution of different activity states through a softmax activation function, uses an Adam optimizer for training, and selects a sparse categorical crossentropy as the loss function. The system comprises the following steps: Step 1: initialize and configure the ESP32-S3 module to send acquisition requests to the router at a certain frequency, and collect Wi-Fi CSI information related to human activities through the signal acquisition module. Step 2: The collected Wi-Fi CSI information related to human activity is processed by the signal processing module in the order of data reading, subcarrier selection, amplitude calculation and preprocessing, filtering, denoising, and normalization. Step 3: The processed Wi-Fi CSI information related to human activity is extracted by the feature extraction module in multiple dimensions, extracting features related to human activity. Step 4: The extracted features are classified and recognized by the GRU model of the target recognition module, and the existence state and action type of the human body are determined based on the changes in the Wi-Fi CSI information features, realizing human activity classification based on WiFi signals.

6. The human body sensing method based on WiFi signals according to claim 5, characterized in that, In step 1, the specific collection steps are as follows: Hardware initialization: Initialize the ESP32-S3 module, configure it to work in STA mode, set the WiFi connection parameters, and initialize the serial communication interface for data upload; Establish a connection: ESP32-S3 module establishes a WiFi connection with a commercial router, completes identity authentication and network access; Data request: ESP32-S3 module sends Ping packets to the router at a frequency of 100Hz, triggering the router to generate response signals; This signal will carry environmental interference information caused by human activity; Data capture: When the router returns the Ping response packet, the ESP32-S3 module uses its built-in WiFi hardware capture unit to extract CSI data from the response packet; the CSI data contains the subtle disturbance characteristics of WiFi signals caused by different activity states such as walking, waving, and sitting; Data caching and uploading: The captured CSI data is temporarily stored in the external PSRAM of the ESP32-S3 module, and then uploaded to the host computer in CSV format through the serial port in real time; when the host computer stores the data, it associates and labels the data with the corresponding human activity scene, and records the collection timestamp; Exception handling: During data collection, if the router does not respond for 5 consecutive times or data transmission errors occur, the module will trigger the RST button to reset and restart, and re-execute the above collection process to ensure the continuity and integrity of human activity data collection.

7. The human body sensing method based on WiFi signals according to claim 6, characterized in that, In step 2, the signal processing module processes the original WiFi signal data as follows: First, data reading and analysis are performed to convert the original CSV data into a processable format and remove invalid data; Next, valid subcarrier selection is performed to filter out non-pilot subcarriers from the analyzed data, reducing data volume and removing irrelevant information; Then, amplitude calculation and preprocessing are performed based on the selected valid subcarriers to calculate amplitude values and convert complex CSI data into more easily processed amplitude information; After that, filtering is performed to remove impulse noise through median filtering and suppress high-frequency noise through bidirectional Butterworth filtering, further purifying the signal; the filtered signal enters the PCA denoising link to reduce data dimension and remove redundant noise components; Finally, normalization is performed to standardize the data to the interval [0, 1], eliminating the dimension effect and providing standardized data input for feature extraction and model training.

8. The human body sensing method based on WiFi signals according to claim 7, characterized in that, The specific processing steps are as follows: Data reading and analysis: ESP32-S3 modules deployed in the monitoring area collect WiFi signals containing human activity information, connect with the host computer and store as raw data in CSV format. Then use the pd.read_csv function to read the CSV format raw data, store the data in DataFrame, and for the "data" column containing CSI data, use the parse_data function to analyze, convert the string to a list using ast.literal_eval, and filter the data according to the data length. Data that does not meet the requirements is marked as np.nan; The "timestamp" column is converted to date and time format, and the data rows with parsing errors are removed; Effective subcarrier screening: According to the IEEE 802.11 protocol, use the get_data_subcarriers function to screen effective data subcarriers. Set the set of all subcarrier indices as S, and the set of pilot subcarrier indices as P. Then the set of data subcarrier indices D = S - P; Amplitude calculation and pre-processing: the parsed CSI data is reshaped into a (x, y) two-dimensional matrix, for each subcarrier i, the formula The amplitude value is calculated, where Re i and Im i are the real and imaginary parts of the i-th subcarrier, respectively, and the amplitude corresponding to the effective subcarrier is extracted to form the amplitude matrix A; Filtering: Use a combination of median filtering and bidirectional Butterworth filtering. The median filtering uses a window size w = 3 to process the signal x(t), and the formula is: The bidirectional Butterworth filter according to the sampling rate f s = 100 Hz, the cutoff frequency f c = 20 Hz and the order n = 4, generate filter coefficients, and perform forward and backward filtering on the median filtered signal to suppress noise. Principal Component Analysis (PCA) denoising: PCA denoising is performed using the PCA Denoiser class, with the magnitude matrix X being transposed and converted by PCA: Y = PCA(X T ), the first principal component of the converted data is set to zero, and the signal is reconstructed by inverse conversion: where Y' is the modified PCA conversion result; Normalization: For the signal matrix Z after PCA denoising, normalize each column data z j : wherein Z represents a signal matrix after PCA denoising; z j represents the jth column data of matrix Z; min(z j ) is the minimum value in the jth column data; max(z j ) is the maximum value in the jth column data; is the jth column data after normalization processing.

9. The human body sensing method based on WiFi signals according to claim 8, characterized in that, In step 3, the amplitude information of the preprocessed CSI data is extracted as a key feature; the CSI data is divided into groups of two elements, representing the real and imaginary parts of each subcarrier. Assuming that each group of elements forms a vector x = [x1, x2, …, xn], the Euclidean norm of each group of elements is calculated, i.e. n The amplitude value corresponding to each subcarrier is obtained. ​ In actual data processing, the amplitude value sequence designed and calculated is A = [A1, A2, …, A m ], the screened effective data subcarrier index set is S = {s1, s2, …, s 48}, the amplitudes of the corresponding subcarriers are extracted from the amplitude value sequence A to form a feature vector After obtaining the set of effective subcarrier indices, select the amplitude values at the corresponding positions from the amplitude value sequence to form the feature vector used for model training and recognition. The feature vector will be input into the GRU neural network model of the target recognition module to determine the presence state and action type information of the human body.

10. The human body sensing method based on WiFi signals according to claim 9, characterized in that, In step 4, the GRU neural network model input layer receives the feature vector output by the feature extraction module. The input shape is set to (x, y), which means it contains x time steps, and each time step corresponds to y amplitude features of the effective subcarriers. The GRU neural network model contains two layers of N GRU units. The weights are initialized using the "he_normal" method. The first layer of GRU units returns a sequence. The second layer of GRU units no longer returns a sequence. Between the two layers of GRU units and between the hidden layer and the output layer, there are Dropout layers. The output layer contains M neurons and uses the rectified linear unit (ReLU) activation function for nonlinear transformation. Finally, through a fully connected layer with the corresponding number of classifications, combined with the softmax activation function, the classification results of human activity are output.

Images