A human posture estimation method based on wi-fi channel state information

Abstract

The present disclosure provides a human posture recognition method based on Wi-Fi channel state information, which firstly constructs a wireless human posture estimation data set, constructs a human posture video and CSI data under different postures of the human body, and adopts a mode of changing the last 32 bits of the hardware NIC timestamp in the CSI data packet into the last 32 bits of the system timestamp for software time alignment. The present disclosure further constructs a human posture key point prediction deep learning model of a pyramid dilated convolution + residual network; when prediction is needed, the CSI data to be predicted under any action of the human body is obtained, and the human posture action representation is obtained by inputting the model. The present disclosure can realize human posture recognition based on Wi-Fi channel state information, and simultaneously solve the data synchronization problem.

Description

Technical Field

[0001] This invention belongs to the field of wireless human pose estimation, specifically relating to a human pose estimation method based on Wi-Fi channel state information. Background Technology

[0002] With the rapid development of technology and the improvement of people's living standards, smart living has become an inevitable trend in modern society. Human body sensing technology plays a crucial role in this trend, providing strong technical support for applications such as smart homes and virtual reality. Traditional human behavior sensing technologies mostly rely on physical devices or wearable devices, which not only limit the user's range of movement but also raise numerous privacy concerns. Therefore, developing a non-contact, high-precision human behavior sensing technology is of great significance.

[0003] In recent years, fine-grained human pose estimation based on Wi-Fi channel state information (CSI) has attracted widespread attention. Wi-Fi, as a widely deployed wireless communication technology, offers advantages such as wide coverage, low cost, and easy accessibility. By extracting features from Wi-Fi-based CSI signals, rich environmental and human pose information can be obtained, thereby enabling human pose estimation. This technology overcomes some limitations of traditional human pose estimation techniques, providing a new perspective for human behavior perception methods and possessing enormous application development potential.

[0004] However, Wi-Fi signals differ significantly from traditional dedicated wireless sensing devices in their signal characteristics. Traditional dedicated wireless sensing devices are known for their large bandwidth and high resolution, while Wi-Fi signals, on the other hand, are limited by smaller bandwidth and lower spatial resolution, making it difficult to achieve high accuracy and real-time human pose estimation. Summary of the Invention

[0005] In view of this, the present invention proposes a human posture recognition method based on Wi-Fi channel state information. Based on a custom wireless human posture estimation dataset, a suitable model framework is established to obtain a deep learning model for predicting human posture key points, thereby realizing human posture recognition based on Wi-Fi channel state information. At the same time, the data synchronization problem is solved, thus solving the problems of large model size, slow speed and low accuracy of existing deep learning methods.

[0006] To solve the above-mentioned technical problems, the present invention is implemented as follows.

[0007] A method for human posture recognition based on Wi-Fi channel state information includes:

[0008] Step 1: Establish a wireless human pose estimation dataset: Simultaneously collect channel state information (CSI) data and human pose videos under different human poses and perform time alignment; obtain human pose images from human pose videos and label the coordinates and confidence scores of human pose key points in the human pose images to obtain labeled supervision data; CSI data and labeled supervision data form training samples to establish a wireless human pose estimation dataset.

[0009] The time alignment includes: during data acquisition, the video recording time is the system timestamp T1. The CSI data is collected by open-source measurement software and carried in the CSI data packet, which is provided to the sender. It includes the hardware NIC timestamp information T2 at the time of transmission. There is a difference between the system timestamp T1 and the hardware NIC timestamp T2. The underlying code of the open-source measurement software for CSI data is modified so that the last 32 bits of the original hardware NIC timestamp in the CSI data packet sent by the software are directly changed to the last 32 bits of the system timestamp, i.e., the system timestamp offset information. The device receiving the CSI data packet extracts the timestamp from the CSI data. The device calculates the time when the last 32 bits of the system timestamp is 0, which is closest to the current time. The system timestamp offset information of the last 32 bits in the CSI data packet is added to obtain the system time when the CSI data packet was sent.

[0010] Step 2: Establish and train a deep learning model for predicting human pose key points: Design a network model that maps CSI data to the coordinates of human pose key points. This network model includes pyramid dilated convolutions and residual networks; input the wireless human pose estimation dataset into the network model for training.

[0011] Step 3: Human pose prediction: Obtain the CSI data to be predicted under any human movement, input it into the human pose key point prediction deep learning model, and obtain the human pose key point sequence corresponding to the current CSI data; combine the human pose key point sequence according to the time dimension to obtain the human pose action representation.

[0012] Preferably, in step 1, different human postures include postures of different body types in standing, walking, sitting, lying down, bending, stretching, and rotating.

[0013] Preferably, in step 1, after acquiring CSI data, further CSI data packet loss processing is performed, specifically as follows:

[0014] Import the .dat file generated by the Linux 802.11n CSI tool into the Matlab software;

[0015] The timestamp difference between two consecutive CSI data packets is determined based on the sampling frequency.

[0016] Based on the tolerable error range, the threshold m for the time difference between the reception of two CSI data packets under normal circumstances is determined based on the timestamp difference.

[0017] Calculate the timestamp difference between the two CSI data packets actually received and compare it with the threshold m. If it is greater than the threshold m, it is determined that there is packet loss in the data packet transmission.

[0018] Nearest neighbor interpolation was used to fill in the missing CSI data due to packet loss.

[0019] Preferably, the annotation of the coordinates and confidence scores of key human pose points in the human pose image is performed by using Alphapose to annotate the coordinates and confidence scores of key human pose points in the human pose image.

[0020] Preferably, the deep learning model for predicting human pose key points is configured as follows: after inputting CSI data into dilated convolutions with dilation rates of {1, 2, 4, 8}, the resulting data are fused by summing the features, followed by feature extraction, regularization, and activation operations. The data is then input into a 50-layer residual network with residual blocks numbered {3, 4, 6, 3}, and the residual blocks mainly contain convolution and activation operations. Finally, a fully connected layer is added to output 34 key points.

[0021] In deep learning models, the loss function metric, Loss, is the weighted sum of the Euclidean distance confidence scores between the predicted and actual values ​​of each human pose keypoint.

[0022]

[0023] Among them, C i pd represents the confidence level of the i-th human pose keypoint in the training samples. i Let gt be the predicted value of the i-th human pose key point. i Let be the true value of the key point of the i-th human posture; The symbol in the middle indicates Euclidean distance calculation.

[0024] The key points of human posture include 17 key points, which represent the nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle.

[0025] Beneficial effects:

[0026] (1) This method achieves synchronization between CSI reception and human posture video recording by calculating timestamps, which solves the problem that the antenna transmission and camera cannot be controlled simultaneously due to hardware limitations. It ensures the correspondence between CSI data and human posture supervision information, and provides a reliable data foundation for human posture estimation.

[0027] (2) This invention is applied to the field of wireless human pose recognition. It uses a residual network model with a pyramid-shaped dilated convolutional structure to perform supervised training on Wi-Fi signals. The pyramid-shaped dilated convolutional structure can achieve multi-scale information fusion, increase the receptive field of the convolutional layer without losing spatial resolution, and retain the rich spatial and variation information contained in the Wi-Fi signal. The use of the residual network can avoid the model degradation problem caused by the increase of network depth, and has a good fitting effect. At the same time, it achieves high accuracy and real-time human pose estimation. Attached Figure Description

[0028] Figure 1 This is a flowchart of the algorithm processing according to an embodiment of the present invention.

[0029] Figure 2 This is a diagram illustrating the changes in data volume during CSI data interpolation.

[0030] Figure 3 This is a schematic diagram of CSI data packet loss interpolation.

[0031] Figure 4 This is a flowchart illustrating the data alignment process.

[0032] Figure 5 This is a schematic diagram of the network structure in step 3 of the present invention; wherein Figure 5 (a) is a schematic diagram of a pyramid-shaped dilated convolutional network. Figure 5 (b) is a schematic diagram of ResNet50. Figure 5 (c) is a schematic diagram of the overall network structure process.

[0033] Figure 6 This is a diagram of the actual experimental scenario.

[0034] Figure 7 This is a schematic diagram of the model training results. Figure 7 (a) and Figure 7 (b) shows two examples, with the left side showing the true human pose and the right side showing the predicted human pose.

[0035] Figure 8 A schematic diagram of the key points of 17 human body postures. Detailed Implementation

[0036] The present invention will be further described below with reference to the accompanying drawings and embodiments, but the scope of the invention is not limited in any way.

[0037] The purpose of this invention is to solve the problems of large network size and low accuracy in existing networks, and to propose a deep learning method for human pose recognition. Figure 1 This is a flowchart of the algorithm processing according to an embodiment of the present invention. Figure 1 As shown, the present invention is achieved through the following steps:

[0038] Step 1: Modeling the effect of human motion on CSI signals

[0039] Channel state information (CSI) describes the attenuation factor of a signal on each transmission path, that is, the value of each element in the channel gain matrix H, such as signal scattering, environmental attenuation, distance attenuation, etc., which can reflect the characteristics of the wireless channel in a fine-grained manner.

[0040] In an OFDM system, the signal received by the receiver can be represented as:

[0041] Y = HX + R

[0042] Where X and Y represent the transmitted and received signals, respectively, H is the Channel State Information (CSI) matrix, and R is Gaussian white noise. Therefore, the CSI matrix can be approximately represented as:

[0043]

[0044] To further accurately describe the characteristics of multiple signal propagation paths, H can be represented as:

[0045]

[0046] Where a n (t) is the amplitude attenuation factor, τ n (t) is the propagation delay, f is the carrier frequency, and t is the current time. The channel frequency response at a specific carrier frequency is the CSI signal.

[0047] Let N be the number of subcarriers, then at time i, the channel state information H(i,j) of the j-th receiving antenna can be expressed as:

[0048] H(i,j)=[H(i,j,1),H(i,j,2),…,H(i,j,N)]

[0049] The channel state information of the k-th subcarrier can be represented as:

[0050]

[0051] in, Phase information reflects the phase shift during wireless signal transmission.

[0052] In a typical indoor space, wireless signals have multiple propagation paths. In addition to the line-of-sight propagation path (the primary propagation path), there are also multiple reflection paths caused by other environmental factors such as walls and furniture. When someone enters the room, these moving factors introduce additional propagation paths related to the human body.

[0053] In wireless communication systems, line-of-sight propagation signals are primarily affected by free-space path loss. Based on the free-space model and Friss free-space equations, the received power when the signal propagates from the transmitting antenna to the receiving antenna at a distance d can be expressed as:

[0054]

[0055] Among them, P t For transmission power, P r (d) represents the received power, which is a function of the line-of-sight propagation distance d. t For the transmit antenna gain, G r Let λ be the receiving antenna gain, and λ be the electromagnetic wave wavelength. Besides considering line-of-sight propagation, the impact of other critical reflection paths on the received power must also be taken into account. After introducing environmental factors related to reflection paths, the received power can be expressed as:

[0056]

[0057] Where h is the distance from other reflection paths to the line-of-sight propagation path (LoS path).

[0058] When a human body is in an indoor environment, multiple scattered paths are generated. The scattered power of these paths is added to the final received power. The human body scattered power can be expressed as:

[0059]

[0060] Where Δ is the extra path length caused by the human body.

[0061] If a person is stationary in an environment, P r The body remains stable. However, with human movement, the positions of the joints change. Taking a fall as an example, as a person falls, the body's scattering point becomes lower, directly causing a change in the body's scattering path. In an actual fall, the scattering path changes rapidly, directly leading to a change in received power.

[0062] The total received power of the receiver is the sum of the received power from each propagation path. Assume the intensities of the line-of-sight path, the indoor environment (floor, walls, etc.) reflection path, and the human body scattering path are E... LoS E ref and E Sca Then the total received power can be expressed as:

[0063] P∝|E LoS +E ref +E Sca | 2

[0064] In the above formula, under specific circumstances, due to the invariance of the environment, E LoS and E ref Generally a constant, when the human body is at rest, the component E... Sca It is also basically in a constant state; when the human body is in motion, E Sca It will change over time, and this can be used to represent human movement.

[0065] Wi-Fi signals can reach various parts of the human body, and these parts will cause reflection and attenuation during propagation, thus affecting the CSI value. Furthermore, the changes in indoor communication links caused by the scattering paths introduced by the human body can be modeled: Under indoor multipath conditions, CSI can be expressed as:

[0066]

[0067] Among them, H s This represents the channel characteristics of static signal propagation paths introduced by the indoor environment. The subscript l indicates the static signal propagation path index, and L is the number of static signal propagation paths; H m This represents the channel characteristics of the dynamic signal propagation path introduced by the moving human body, where the subscript d indicates the dynamic signal propagation path index, and D is the number of dynamic signal propagation paths. From the above definition, we can see that H... s Time-invariant information, while H m It contains human motion information that changes over time. The presence of human motion information in H provides theoretical support for the model to extract human posture information from CSI signals.

[0068] Step 2: Data Acquisition and Preprocessing

[0069] To establish a wireless human pose estimation dataset, this invention takes the following steps:

[0070] During the data acquisition phase, this solution simultaneously acquires CSI data and captures corresponding human pose video images using a camera with self-encoding capabilities. After obtaining the human pose estimation data, to ensure the synchronization between the CSI data and the human pose images, the CSI data and the human pose video data need to be processed separately to construct an accurate Wi-Fi wireless human pose estimation dataset.

[0071] For CSI data, since CSI signals are easily affected by environmental and equipment factors, receiving antennas may struggle to capture all CSI data values, leading to data loss and outliers. Therefore, preprocessing of the acquired raw CSI data is necessary, including amplitude information extraction, nearest neighbor interpolation, Kalman filtering, and bilinear interpolation, to obtain accurate and reliable data.

[0072] Specifically, each data set during the measurement process contains 3000 data packets, with the actual received data size ranging from approximately 2500 to 3000. To address these issues, this solution first uses the nearest neighbor interpolation method based on time series data to augment the CSI data volume to 3000. Kalman filtering is then used to remove outliers, resulting in a CSI signal data size of 3000×3×3×30. Due to the data volume requirements of deep learning, each large data set is split into smaller data blocks of 10×3×3×30 using Python processing. After resizing, the data size is changed to 300×3×3. To improve the smoothness of the CSI data and increase the data volume, bilinear interpolation is finally used to interpolate the data to a size of 300×136×136. The changes in CSI data size during processing are shown below. Figure 2 As shown.

[0073] For human pose motion videos, this solution first uses the OpenCV-Python library to decompose the video frame by frame. The resulting image set is then uniformly sampled to match the CSI data. Next, this invention uses Alphapose to annotate the keypoint coordinates and confidence scores of the human pose in the images, replacing manual dataset annotation and significantly improving annotation speed and accuracy. The resulting JSON file is then processed to extract the keypoints and confidence scores for each image, thus obtaining complete label supervision data.

[0074] In actual data processing, packet loss and data alignment are problems that need to be addressed.

[0075] (1) CSI data packet loss

[0076] During data acquisition, given the complexity of wireless network communication, factors such as channel interference, transmission errors, and hardware failures are often unavoidable. These factors may cause some CSI data packets to fail to reach the receiving end or be completely recorded during transmission. For deep learning models, especially when building human pose estimation models, the consistency of data volume and the accuracy of data quality are particularly critical. Therefore, the integrity and accuracy of CSI data have a decisive impact on the model training effect, making the resolution of CSI data packet loss a crucial issue. The processing steps are as follows: Figure 3 As shown.

[0077] Since the .dat data structure obtained during CSI acquisition does not record packet loss locations, we need to determine the packet loss location by judging its transmission time. In the receiver data processing stage, this solution first imports the .dat file generated by the Linux 802.11n CSI Tool measurement into Matlab software to further extract CSI amplitude information. The timestamp_low index in each obtained CSI data packet is the timestamp record of its transmitting NIC network card, which is the last 32 bits of the network card time, with a standard data sampling rate of 10. 6 Hz. That is to say, assuming that the sampling frequency is set to 100Hz in this experimental scheme, the difference in standard hardware timestamps between two consecutive CSI data packets should be 10000.

[0078] Therefore, the time on the horizontal axis of the waveform can be determined using the timestamp physical quantity. Combined with the aforementioned standards, and within acceptable error limits, the difference in timestamps between two consecutive data packets is calculated, and a variable threshold m is set to effectively identify whether packet loss occurs during data packet transmission. If packet loss is detected, to ensure data sequence coherence and model training quality, this scheme employs nearest neighbor interpolation to fill in the missing CSI data due to packet loss. This method not only improves data integrity but also helps enhance the accuracy and stability of subsequent human pose estimation models. Through this series of data processing measures, we provide more reliable and effective data support for the training of deep learning models.

[0079] (2) CSI data and RGB image alignment issues

[0080] To ensure a correct mapping between human poses in images and human movements contained in CSI data, CSI data acquisition and video recording should begin simultaneously during data acquisition. To guarantee this experimental accuracy, solutions can be implemented using both hardware and software processing, as illustrated in the diagram below. Figure 4 As shown.

[0081] (1) Hardware-based solution: During the data acquisition process, two computers are used to control the antenna transmission and reception, and the time of the two computer hardware NICs is synchronized. A script is used on one of the computers to control the antenna reception and camera operation at the same time, so as to achieve data synchronization during the measurement phase.

[0082] (2) Software-based solution: A timestamp-based method is used to synchronize CSI data and video frames. This method requires recording a reference point timestamp at the start of data acquisition, typically the send timestamp by default. Then, at the start of CSI data acquisition and video recording, their respective timestamps are recorded. These timestamps can be obtained from the system time or other precise time sources.

[0083] After data acquisition and recording are completed, CSI data and video images are stored in corresponding files, with timestamp information included in the file names or metadata to facilitate data alignment in subsequent data processing stages. During data processing, the CSI data and video image files are read and aligned based on the timestamps. By comparing the timestamp differences, the time offset between the CSI data and video images can be determined. Then, one of the data sources (e.g., CSI data or video frames) is time-corrected to ensure temporal alignment between the two data sources.

[0084] However, due to hardware limitations, we were unable to control the antenna receiver and camera simultaneously on a single computer in this experiment. Therefore, this invention uses a software-based approach and timestamp calculation to synchronize CSI data and video frames.

[0085] When using software processing methods, it's important to note the difference between hardware and software timestamps. A timestamp is the number of seconds elapsed since January 1, 1970 (midnight UTC / GMT) (excluding leap seconds), used to represent a point in time. Initially, the hardware and software timestamps may not be the same. The timestamp in the CSI data structure collected by the Linux 802.11n CSI Tool is the last 32 bits of the NIC's time, incremented by 1 every microsecond, repeating approximately every 4300 seconds (72 minutes). 0 ~2 32 (Approximately 4300 seconds). Therefore, the actual time for data packet transmission can be calculated from two aspects:

[0086] ① Use PTP Precision Time Protocol

[0087] Precision Time Protocol (PTP) is a protocol used for clock synchronization in a network. When used in conjunction with hardware, PTP can achieve sub-microsecond accuracy. During experimental measurements, the PTP protocol can be used to achieve complete synchronization between the software time (computer time) and the NIC (Network Interface Card) time, allowing direct reading of time information from the CSI (Content Query Scale) data. However, it's important to note that the PTP protocol has high equipment requirements; some classic network cards do not support this function, such as the Intel 5300 series used in this solution.

[0088] ② Improve the software

[0089] Since the Intel 5300 does not support the PTP synchronization protocol, this paper modifies the underlying code of the Linux 802.11n CSI Tool. Simultaneously with sending data packets, a corresponding software timestamp is written to a specific text file, thus bypassing hardware and software time synchronization to obtain CSI time information. After obtaining the CSI timestamp information, this solution uses a self-written program to calculate the closest last 32 bits of the software timestamp to the current time (which is 0), and then superimposes this value onto the last 32 bits of the software timestamp from the CSI data to obtain the complete time information at the time the data packet was sent.

[0090] (3) Dataset completeness

[0091] The completeness of the Wi-Fi wireless human pose estimation dataset is crucial for solution validation and model design. A complete dataset needs to provide diverse, real-world samples, covering various pose types, pose variations, and angles. Such a dataset can help train a more robust and generalizable model, enabling it to more accurately estimate human pose key points in practical applications. In the initial experimental phase of this solution, the model output results were consistent, approximating the average value of key points from each sample. After excluding the influence of algorithmic and other related factors, the completeness of the dataset is considered to have a significant impact on human pose estimation based on Wi-Fi signals.

[0092] The completeness of a dataset is crucial not only for training but also for testing. An incomplete dataset often results in unrepresentative test results. This is because, in actual experiments, deep learning for wireless human pose estimation inevitably learns inherent characteristics of the dataset itself. For example, in keypoint estimation, if human behavior data contains too many hand movements and too few movements involving leg joints such as running and jumping, the accuracy of hand joint estimation will be far lower than that of leg joint estimation, and the output human pose image will tend to maintain a single leg joint. Using such a dataset for training or testing is undoubtedly subjective, and the results will lack representativeness. Therefore, the completeness of the dataset must be given special attention during its acquisition.

[0093] A complete dataset should consider both the size of the dataset and the diversity of human poses, comprehensively covering variations in different human poses, as well as pose samples from different individuals (tall, short, fat, thin). This helps to improve the diversity and representativeness of the dataset, ensuring that the dataset has broader applicability in keypoint pose recognition.

[0094] Therefore, when constructing the dataset, this invention collects data on different human postures, including postures of different body types such as height, weight, and build, in standing, walking, sitting, lying down, bending, stretching, and rotating positions.

[0095] Step 3: Build and train a human pose prediction network model;

[0096] Pyramid dilated convolution is a multi-scale convolution operation that can extract features across different receptive fields. By using pyramid dilated convolution to capture pose features at different scales, and applying different dilation rates to convolutional layers, feature representations at different levels can be obtained, achieving effective capture of pose information from local to global perspectives.

[0097] Residual networks are a method that addresses the vanishing and exploding gradient problems in deep network training by using skip connections. This invention introduces residual connections to construct deep network structures. Residual connections can directly pass features from the previous layer to subsequent layers, allowing the network to better learn and fit information, avoiding information attenuation during network transmission, and helping to improve the network's convergence speed and model performance.

[0098] By combining pyramidal dilated convolutions and residual networks, the model can model pose features at different scales, and retains detailed information and contextual relationships through residual connections, thereby improving the prediction accuracy of pose keypoints. Furthermore, based on the size and format of the input CSI data, this invention improves the channel size and connection layers of the original network to adapt to different data characteristics and model complexity, thus improving the model's performance and applicability. The network structure diagram is shown below. Figure 5 As shown.

[0099] Specifically, CSI data is input into dilated convolutions with dilation rates of {1, 2, 4, 8}. The resulting data is then fused by summing the features. After simple feature extraction, regularization, and activation operations, the data is fed into a 50-layer residual network with residual blocks of {3, 4, 6, 3}, primarily containing convolutions and activation operations. Finally, a fully connected layer is added to output 34 keypoints.

[0100] When using AlphaPose to extract 17 key nodes in human pose estimation, not only were the locations of these key points obtained, but also the corresponding confidence scores, which reflect the reliability of the AlphaPose algorithm's prediction of the location of a specific key point.

[0101] Therefore, to accurately assess the difference between the predicted human pose joints and the actual pose, and to guide model optimization, this scheme defines a loss function index for single-person human pose estimation, as shown in the following formula. This loss function index is the weighted sum of the Euclidean distance confidence scores between the predicted and actual values ​​of each human pose keypoint. The loss contribution of each keypoint depends not only on the Euclidean distance between its predicted and actual positions but also on the confidence score of that point.

[0102]

[0103] Among them, C i pd represents the confidence level of the i-th person's pose keypoint in the sample. i Let gt be the predicted value of the key point of the i-th person. Similarly, gt i This is the actual value. The slash symbol is used to represent the calculation of Euclidean distance.

[0104] Based on the above design principles and improvement measures, this human posture prediction model can effectively extract features from CSI data and accurately predict the location of key points in human posture.

[0105] Input the training samples obtained in step 2 into the human pose prediction model and train it to obtain a trained prediction model.

[0106] Step 4: Human posture prediction;

[0107] In the human pose prediction stage, data on arbitrary human movements are collected to obtain the CSI signal to be predicted. The collected CSI signal is preprocessed and segmented before being input into the human pose keypoint prediction model for analysis and inference.

[0108] The preprocessed signal is input into the human pose keypoint prediction model. Through keypoint value regression analysis, the sequence of human pose keypoints corresponding to the CSI signal can be obtained. Finally, these keypoint sequences are combined according to the time dimension to obtain a complete representation of human pose and movement.

[0109] Through the above steps, this invention constructs a wireless human pose estimation scheme. It utilizes a deep learning model to process and predict CSI data, achieving accurate prediction of key points in human pose. This model combines pyramidal dilated convolutions and residual networks, and optimizes the network channel size based on the characteristics of the CSI data to improve the performance and accuracy of pose estimation.

[0110] To verify the proposed human pose estimation method based on Wi-Fi channel state information, data measurement of single-person pose was performed in an indoor scene. The scene layout is as follows: Figure 6 As shown.

[0111] This embodiment utilizes an Intel 5300 wireless network card equipped with three 5G antennas, the Linux 802.11n CSI Tool, and the Ubuntu 14.04 LTS operating system for CSI data acquisition, and a Hikvision Os-E14a camera for human posture video capture. The experiment employs a 100Hz CSI data acquisition frequency, corresponding to a video frame rate of 10fps. Every 10 frames of CSI data correspond to one human posture key point monitoring information. Data processing mainly includes filtering, interpolation, and other related operations to improve data quality and accuracy.

[0112] The constructed wireless human pose estimation dataset contains 36,000 valid data sets. 90% of the data was used as the training set for model parameter learning and optimization. The remaining 10% served as an independent test set to evaluate the model's performance and generalization ability. The processed dataset was input into the algorithm model for iterative training using the Adam optimizer with a learning rate of 0.001. Good results were achieved after 501 training iterations. Two simple test prediction graphs are shown below. Figure 7 As shown in Figures (a) and (b) in the text.

[0113] Considering the keypoint annotation of AlphaPose, and with slight modifications based on the Coco dataset, the inventors used the percentage of correct keypoints (PCK) as the metric to measure algorithm performance:

[0114]

[0115] Where P is the number of people in the test frame, and i represents the index of a body joint, i∈{1,2,...,17) For the predicted value of the key point of the i-th person in the P-th sample, similarly, This is the actual value. p h is the width of the human pose bounding box. p The height of the human pose bounding box. It is the Euclidean pixel distance between the predicted and true values, expressed as the length of the diagonal of the human bounding box. Normalization is performed. Ⅱ represents a logical judgment operation; if the normalization error value is less than 'a', the logical value is 1; otherwise, it is 0. To obtain the human bounding box, the body joint coordinates of the AlphaPose are aligned with the bounding boxes of the rendered human pose key points during processing.

[0116] The human pose estimation implemented in this scheme has 17 key points. Based on PCK, the points are averaged and segmented to establish the MPCK index as shown in the following formula:

[0117]

[0118] The PCK results obtained by evaluating metrics using the existing test set are shown in the table below:

[0119]

[0120] These 17 key points represent the nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle, respectively. Figure 7 As shown.

[0121] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for human posture recognition based on Wi-Fi channel state information, characterized in that, The application relates to a wireless human posture estimation method and device. Step 1: establishing a wireless human posture estimation dataset: simultaneously collecting channel state information (CSI) data and human posture videos under different human postures, and performing time alignment; Obtaining human posture images from human posture videos, and labeling human posture key point coordinates and confidence thereof in the human posture images to obtain label supervision data; The CSI data and the label supervision data constitute training samples, so that the wireless human posture estimation dataset is established; The time alignment comprises the following steps: in the data collection process, the video recording time is a system timestamp, the CSI data is collected by open-source measurement software and provided in a CSI data packet, the hardware NIC timestamp information is sent, and a difference exists between the system timestamp and the hardware NIC timestamp; the open-source measurement software underlying code of the CSI data is modified, the original hardware NIC timestamp in the CSI data packet sent by the open-source measurement software is directly changed into the last 32 bits of the system timestamp, that is, system timestamp offset information; after receiving the CSI data packet, the timestamp in the CSI data is extracted, the system time when the last 32-bit system timestamp is 0 closest to the current time is calculated by the equipment, and the last 32-bit system timestamp offset information in the CSI data packet is superimposed, so that the system time when the CSI data packet is sent can be obtained; Step 2: establishing and training a human posture key point prediction deep learning model: designing a network model for mapping the CSI data to human posture key point coordinates, the network model comprising a pyramid dilated convolution and a residual network; and inputting the wireless human posture estimation dataset into the network model for training; Step 3: human posture prediction: obtaining CSI data to be predicted under any action of a person, inputting the CSI data into the human posture key point prediction deep learning model to obtain a human posture key point sequence corresponding to the current CSI data; and combining the human posture key point sequence according to the time dimension to obtain a human posture action representation.

2. The method of claim 1, wherein, In step 1, the different human postures comprise postures of different body types in standing, walking, sitting, lying, bending, stretching and rotating.

3. The method of claim 1, wherein, In step 1, after the CSI data is collected, further CSI data packet loss processing is performed, and the processing specifically comprises the following steps: Importing a.dat file generated by a Linux 802.11n CSI tool into Matlab software; Determining a timestamp difference value of two CSI data packets according to a sampling frequency; Determining a threshold value m of a time difference of two CSI data packets under normal conditions according to a tolerable error range and based on the timestamp difference value; Comparing a timestamp difference value of two actually received CSI data packets with the threshold value m, and if the timestamp difference value is greater than the threshold value m, it is determined that there is a packet loss in data packet transmission; Performing data filling by using a nearest neighbor interpolation method to fill the missing CSI data due to the packet loss.

4. The method of claim 1, wherein, The labeling of human posture key point coordinates and confidence thereof in the human posture images is performed by using Alphapose.

5. The method of claim 1, wherein, The human posture key point prediction deep learning model is set as follows: after the CSI data is input into the dilated convolution with an expansion rate of {1, 2, 4, 8}, the obtained data are fused in an additive manner, then feature extraction, regularization and activation operations are performed, and the data are input into a 50-layer residual network, the number of residual blocks is {3, 4, 6, 3} in sequence, and the residual blocks mainly contain convolution and activation operations; finally, a full connection layer is added to realize the output of 34 key points; The loss function index Loss in the deep learning model is the weighted cumulative sum of the Euclidean distance confidence between each human posture key point prediction value and the true value: wherein C i is the confidence of the i-th human pose keypoint of the training sample, pd i is the predicted value of the i-th human pose keypoint, gt i is the ground truth value of the i-th human pose keypoint. The symbol in the middle represents the Euclidean distance calculation.

6. The method of claim 1, wherein, The human posture key points include 17 key points, which respectively represent nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle and right ankle.

Images