Fine-grained human body posture estimation method based on wireless radio frequency signals

Abstract

The invention discloses a fine-grained human body posture estimation method based on a wireless radio frequency signal, and belongs to the field of wireless communication and computer vision, a Wi-Fi field is established through an antenna of a wireless transmitting end, and a wireless receiving end receives a penetrated or reflected Wi-Fi signal. In addition, CSI values related to human body postures are obtained through real-time calculation and are recorded together with video pictures captured by the synchronous camera, and data collection is completed. And the collected CSI value is subjected to noise filtering and standardization processing. A deep learning network is constructed, and a training data set including synchronous CSI values and video frames is established by using collected data. The network employs a teacher-student architecture and uses a loss function to optimize a prediction matrix. And decoding the real-time CSI value through the trained deep learning network so as to estimate the human body posture. The fine-grained human body posture estimation method based on the wireless radio frequency signals has the advantages of being low in cost, good in privacy, high in environmental adaptability and high in estimation precision, and is suitable for various application scenes.

Description

Technical Field

[0001] The invention relates to the technical field of wireless communication and computer vision, and in particular to a fine-grained human body posture estimation method based on wireless radio frequency signals. Background Art

[0002] With the development of science and technology and the improvement of living standards, smart home and virtual reality technologies are advancing rapidly. In the field of computer vision, such as video surveillance, human-computer interaction, motion analysis and virtual reality, the recognition of human movements and behaviors has become an indispensable part and is a very active research field. Human posture estimation involves identifying the location of human bones and their interconnections from a single RGB image or video, and finally outputting the overall or local limb parameters of the human body, such as human body contour, head position and orientation, joint position and category. This technology shows broad application prospects in the fields of intelligent monitoring, human-computer interaction and gesture recognition.

[0003] To achieve fine-grained human perception, three main types of sensors are currently used: cameras, radars, and lidars. These sensors can directly capture the image of the human body with high spatial resolution in 2D images, depth maps, or 3D point clouds. For example, a camera with a resolution of up to 300*300 pixels, a radar with a depth resolution of about 2 cm, or a lidar with 32 beams.

[0004] Camera-based human segmentation and pose estimation can capture high-resolution human images directly in 2D images, but is limited by technical challenges such as clothing, background, lighting, and occlusion, and faces social issues such as privacy. Radar-based methods can capture high-resolution human images in depth maps, but require the use of specialized hardware, such as frequency modulated continuous wave (FMCW) technology, a wide bandwidth of 1.78 GHz, and a carefully configured 16+4T antenna array. Although high-resolution lidar-based methods can directly capture high-resolution human images in 3D point clouds, they are difficult to be widely used in daily life and home environments due to their high cost and power consumption. Summary of the invention

[0005] The purpose of the present invention is to provide a fine-grained human posture estimation method based on wireless radio frequency signals, which uses commercial Wi-Fi devices to identify human posture changes by analyzing the change pattern of channel state information (CSI) to achieve accurate capture and analysis of human movements.

[0006] To achieve the above object, the present invention provides a fine-grained human posture estimation method based on wireless radio frequency signals, comprising the following steps:

[0007] S1. Establishment and synchronization system of Wi-Fi field, using commercial Wi-Fi router as wireless transmitter, deploying camera next to the wireless transmitter for synchronous picture shooting, and finally synchronizing the video frame with the Wi-Fi signal for time stamp;

[0008] S2. Data collection and preprocessing. The wireless receiver receives the penetrating or reflected Wi-Fi signals and calculates the channel state information (CSI) related to the human body posture in real time. CSI is a set of indicators that describe the state of wireless signals by analyzing the changes in Wi-Fi signals. It is suitable for capturing physical changes in the environment. Subsequently, the video images captured by the synchronous camera are recorded synchronously with the CSI values ​​to complete the data collection. After the collected CSI values ​​are noise filtered and standardized, the key frames are extracted from the video and timestamps are matched with the CSI values ​​to form a synchronous data set.

[0009] S3, build a deep learning network;

[0010] S4, training and optimization, using the collected data to establish a training dataset, including synchronized CSI values ​​and video frames, using a teacher-student network architecture and a loss function to optimize the prediction matrix;

[0011] S5. Posture estimation: Use the trained deep learning network to decode the real-time CSI value to obtain the estimated result of the human body posture.

[0012] Preferably, the Wi-Fi field is established by using a wireless transmitter and a wireless receiver having three antennas.

[0013] Preferably, the method for obtaining the channel state CSI value in step S2 is specifically as follows:

[0014] CSI=m×a×b×c

[0015] Among them, m is the number of WiFi packets received, a is the number of OFDM carriers, b and c are the number of antennas at the wireless transmitter and the wireless receiver, respectively.

[0016] Preferably, the deep learning network in step S3 includes:

[0017] Encoding module: upsamples and compresses the original data of CSI values;

[0018] Feature extraction module: uses deep learning algorithms to extract key features from CSI values;

[0019] Decoding module: Upsample the compressed feature map and convert it into a prediction matrix of human posture.

[0020] Preferably, the loss function used is the weighted sum of the mean square error loss MSE of a single point and the mean square error loss of the overall position; the loss function is designed as follows:

[0021] MSE loss of a single point: used to measure the position error between each predicted key point and the actual key point. For n key points, the expression is:

[0022]

[0023] Among them, p i represents the position of the key point of the i-th prediction, g i Represents the jth actual key point position;

[0024] MSE loss of the overall position: used to evaluate the overall error between the set of all predicted key points and the actual key point set in spatial layout, by calculating the error between the geometric center of the predicted key point set and the geometric center of the actual key point set. Assume that C P To predict the geometric center of the key point, C G is the geometric center of the actual key point, and the loss expression is:

[0025] L global =||C P -C G || 2

[0026] in,

[0027] Combined loss function: The final loss function is the weighted sum of the MSE loss of a single point and the MSE loss of the overall position:

[0028] L=L point +L global

[0029] Among them, α and β are weight coefficients used to adjust the relative importance of single point loss and overall position loss in the total loss.

[0030] Preferably, the prediction matrix includes a multi-dimensional data combination predicted for each key point position;

[0031] Spatial coordinates: The prediction matrix first includes the three-dimensional spatial coordinates of each key point, and the coordinates are expressed in x i ,y i , z i Indicates that i represents the index of the key point, and the mathematical formula is expressed as:

[0032] P i =(x i ,y i , zi );

[0033] Timestamp: The prediction of each keypoint position includes the spatial coordinates t i And the corresponding timestamp, the timestamp is expressed as follows:

[0034] T i =t i ;

[0035] Confidence score: The prediction matrix also contains the confidence score c i , which is used to evaluate the model's confidence in the accuracy of each predicted key point position, indicating the reliability of the model's prediction of each key point position, and is mathematically expressed as:

[0036] C i =c i ;

[0037] Keypoint identification: Each keypoint has an identifier k i , which is used to distinguish different body parts, such as head, arms, legs, etc., and is mathematically expressed as:

[0038] K i =k i ;

[0039] The prediction matrix is ​​expressed as a multidimensional matrix, where each row represents a key point and each column represents an attribute of the point, spatial coordinates, timestamp, confidence, and identifier, which is mathematically expressed as:

[0040]

[0041] Where n is the total number of key points.

[0042] Therefore, the present invention adopts the above-mentioned fine-grained human posture estimation method based on wireless radio frequency signals, which has the following beneficial effects:

[0043] (1) Utilize existing Wi-Fi equipment, significantly reducing costs.

[0044] (2) Privacy protection: Since it does not rely on video images, human posture estimation can be achieved without infringing personal privacy;

[0045] (3) Environmental adaptability: Wi-Fi signals are not sensitive to environmental changes (such as lighting conditions), which makes the present invention more environmentally adaptable.

[0046] (4) Fine-grained estimation: Through the optimized design of the deep learning network, the present invention can achieve high-precision and fine-grained estimation of human body posture.

[0047] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A flowchart of a fine-grained human posture estimation method based on wireless radio frequency signals according to the present invention;

[0049] Figure 2 A schematic diagram of establishing a Wi-Fi field for a fine-grained human posture estimation method based on wireless radio frequency signals according to the present invention;

[0050] Figure 3 A data collection flow chart of a fine-grained human posture estimation method based on wireless radio frequency signals according to the present invention;

[0051] Figure 4 A deep learning network structure diagram of a fine-grained human posture estimation method based on wireless radio frequency signals according to the present invention;

[0052] Figure 5 This is an example diagram of posture estimation results of a fine-grained human posture estimation method based on wireless radio frequency signals in the present invention. DETAILED DESCRIPTION

[0053] Example

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] like Figure 1 , a fine-grained human posture estimation method based on wireless radio frequency signals, the specific steps are as follows:

[0056] S1. Use a commercial Wi-Fi router with at least three antennas as the wireless transmitter to ensure stable signal coverage. Deploy a synchronized camera next to the transmitter and implement timestamp synchronization to capture the video of human posture.

[0057] Reference Figure 2 , configure Wi-Fi routers to enhance signal coverage and stability, and ensure that antennas are correctly placed in indoor environments to maximize coverage. At the same time, install synchronized cameras at selected locations within the Wi-Fi field to ensure that they can cover the entire signal area, and set up a time synchronization mechanism between the camera and the router to achieve synchronous acquisition of video images and Wi-Fi signal data.

[0058] S2. Data collection and preprocessing. In the Wi-Fi field, the human body is a dynamic target. Its movement and posture changes will cause changes in the CSI value. Figure 3 ,The wireless receiving end receives the penetrating or reflected Wi-Fi signals, and calculates the CSI value related to the human posture in real time. ,The video images captured by the synchronous camera are recorded synchronously with the CSI value to provide ,annotated data for subsequent training. ,The collected CSI values ​​are noise filtered and standardized to improve the data quality. ,Key frames are extracted from the video and timestamped with the CSI value to form a synchronous ,dataset.

[0059] S3. Build a deep learning network, refer to Figure 4 , a deep learning network is constructed, including an encoding module, a feature extraction module and a decoding module; the encoding module is responsible for upsampling and feature compressing the original data of the CSI value to adapt to the network input; the feature extraction module uses deep learning algorithms, such as convolutional neural networks (CNN), to extract key features in the CSI value; the decoding module upsamples the compressed feature map and converts it into a prediction matrix of human posture, uses a synchronized data set to train the network, and uses appropriate loss functions and optimization algorithms to adjust the network parameters.

[0060] S4, training and optimization, uses the collected data to establish a training dataset, including synchronized CSI values ​​and video frames, using a teacher-student network architecture and a loss function to optimize the prediction matrix.

[0061] The loss function used is the weighted sum of the mean square error (MSE) loss of a single point and the mean square error loss of the overall position. This loss function scheme takes into account both the accuracy of the position prediction of each key point and the accuracy of the overall layout of all key points, thereby comprehensively improving the effect of human posture estimation.

[0062] The loss function is designed in detail as follows:

[0063] MSE loss of a single point: The MSE loss of a single point is used to measure the position error between each predicted key point and the actual key point. For n key points, its expression is:

[0064]

[0065] Among them, p i represents the position of the key point of the i-th prediction, g i Represents the jth actual key point position.

[0066] MSE loss of overall position: The MSE loss of overall position is used to evaluate the overall error between the set of all predicted key points and the set of actual key points in spatial layout. This can be achieved by calculating the error between the geometric center of the predicted key point set and the geometric center of the actual key point set. Assume C P To predict the geometric center of the key point, C G is the geometric center of the actual key point, and the loss expression is:

[0067] L global =||C P -C G || 2

[0068] in,

[0069] Combined loss function: The final loss function is the weighted sum of these two losses:

[0070] L=L point +L global

[0071] Among them, α and β are weight coefficients used to adjust the relative importance of single point loss and overall position loss in the total loss.

[0072] Through this design, the model can not only improve the prediction accuracy of a single key point position, but also ensure that the predicted human body posture is consistent with the actual posture as a whole, thereby effectively improving the overall estimation accuracy. The weighted summation method of the loss function provides flexibility, allowing α and β to be adjusted according to specific application requirements to optimize model performance. It is particularly suitable for human body posture estimation tasks that need to pay attention to both key point positioning accuracy and overall structural fidelity. Before actual deployment, it is very necessary to adjust and optimize these weights through multiple iterative experiments to ensure that the model achieves the best results in specific applications.

[0073] The prediction matrix includes a multi-dimensional data combination of the predicted positions of each key point. This prediction matrix contains the three-dimensional spatial coordinates of each key point, the timestamp information associated with it, and the confidence score of each predicted point.

[0074] Spatial coordinates: The prediction matrix first includes the three-dimensional spatial coordinates of each key point, which define the position of the key point in space. Specifically, the coordinates are expressed in x i ,y i , z i , where i represents the index of the key point. This part of the data is obtained through in-depth analysis of the wireless radio frequency signal, especially through the changes in the channel state information (CSI). Mathematically, it can be expressed as:

[0075] P i =(x i ,y i , z i );

[0076] Timestamp: The prediction of each keypoint position includes not only the spatial coordinates t i , and also includes the corresponding timestamps, which enables the prediction matrix to reflect the dynamic changes of key points over time. Timestamps are crucial for analyzing dynamic postures and behavior patterns, especially in applications such as motion analysis and health monitoring. The representation is as follows:

[0077] T i =t i ;

[0078] Confidence score: To assess the model’s confidence in the accuracy of each predicted keypoint location, the prediction matrix also contains a confidence score c i This score is a value between 0 and 1, which indicates the reliability of the model's prediction of each key point location. The mathematical expression is:

[0079] C i =c i ;

[0080] Key point identification: Each key point also requires an identifier k i , which is used to distinguish different body parts, such as head, arms, legs, etc. This helps to quickly identify and locate specific body parts in subsequent processing. The mathematical expression is:

[0081] K i =k i ;

[0082] Putting all the above together, the prediction matrix can be expressed as a multi-dimensional matrix where each row represents a keypoint and each column represents an attribute of the point (spatial coordinates, timestamp, confidence, and identifier). This can be mathematically expressed as:

[0083]

[0084] Where n is the total number of key points. This matrix provides a comprehensive data foundation for fine-grained human posture estimation based on wireless RF signals, so that every aspect of the key points is accurately recorded and analyzed to support further data processing and application.

[0085] S5, posture estimation, uses the trained deep learning network to decode the real-time CSI value to obtain the estimated result of human posture, such as Figure 5 shown.

[0086] Compare the predicted pose with the actual pose to evaluate the accuracy and reliability of the estimation. Adjust the network structure or parameters based on the verification results to further improve the performance of pose estimation.

[0087] Integrate the optimized deep learning model into the real-time monitoring system to achieve real-time human posture estimation and provide corresponding feedback or interaction according to application requirements.

[0088] Therefore, the present invention adopts the above-mentioned fine-grained human posture estimation method based on wireless radio frequency signals, which can accurately estimate human posture using existing Wi-Fi infrastructure without infringing privacy, and is suitable for various scenarios such as health monitoring and motion analysis.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A fine-grained human posture estimation method based on wireless radio frequency signals, characterized in that: The following steps are involved: S1. Establishment and synchronization system of Wi-Fi field, using commercial Wi-Fi router as wireless transmitter, deploying camera next to the wireless transmitter for synchronous picture shooting, and finally synchronizing the video frame with the Wi-Fi signal for time stamp; S2, data collection and preprocessing, the wireless receiving end receives the penetrating or reflected Wi-Fi signal, and calculates the channel state information CSI related to the human body posture in real time. Then, the video screen captured by the synchronous camera is recorded synchronously with the CSI value to complete the data collection. After the collected CSI value is noise filtered and standardized, the key frame is extracted from the video and the time stamp is matched with the CSI value to finally form a synchronous data set S3, build a deep learning network; S4, training and optimization, using the collected data to establish a training dataset, including synchronized CSI values ​​and video frames, using a teacher-student network architecture and a loss function to optimize the prediction matrix; S5. Posture estimation: Use the trained deep learning network to decode the real-time CSI value to obtain the estimated result of the human body posture.

2. The method for fine-grained human posture estimation based on wireless radio frequency signals according to claim 1, characterized in that: The Wi-Fi field is established using a wireless transmitter and a wireless receiver with three antennas.

3. The method for fine-grained human posture estimation based on wireless radio frequency signals according to claim 1, characterized in that: The method for obtaining the channel state CSI value in step S2 is specifically as follows: CSI=m×a×b×c Among them, m is the number of WiFi packets received, a is the number of OFDM carriers, b and c are the number of antennas at the wireless transmitter and receiver, respectively.

4. A fine-grained human posture estimation method based on wireless radio frequency signals according to any one of claim 1, characterized in that: The deep learning network in step S3 includes: Encoding module: upsamples and compresses the original data of CSI values; Feature extraction module: uses deep learning algorithms to extract key features from CSI values; Decoding module: Upsample the compressed feature map and convert it into a prediction matrix of human posture.

5. The method for fine-grained human posture estimation based on wireless radio frequency signals according to claim 1, characterized in that: The loss function used is the weighted sum of the mean square error loss MSE of a single point and the mean square error loss of the overall position; the loss function is designed as follows: MSE loss of a single point: used to measure the error between the position of each predicted key point and the actual key point position. For n key points, the expression is: Among them, p i represents the position of the key point of the i-th prediction, g i Represents the jth actual key point position; MSE loss of the overall position: used to evaluate the overall error between the set of all predicted key points and the actual key point set in spatial layout, by calculating the error between the geometric center of the predicted key point set and the geometric center of the actual key point set. Assume that C P To predict the geometric center of the key point, C G is the geometric center of the actual key point, and the loss expression is: L global =||C P -C G || 2 in, Combined loss function: The final loss function is the weighted sum of the MSE loss of a single point and the MSE loss of the overall position: L=L point +L global Among them, α and β are weight coefficients used to adjust the relative importance of single point loss and overall position loss in the total loss.

6. The method for fine-grained human posture estimation based on wireless radio frequency signals according to claim 1, characterized in that: The prediction matrix includes a multi-dimensional data combination predicted for each keypoint position; Spatial coordinates: The prediction matrix first includes the three-dimensional spatial coordinates of each key point, and the coordinates are expressed in x i ,y i , z i Indicates that i represents the index of the key point, and the mathematical formula is expressed as: P i =(x i ,y i ,z i ); Timestamp: The prediction of each keypoint position includes the spatial coordinates t i And the corresponding timestamp, the timestamp is expressed as follows: T i =t i ; Confidence score: The prediction matrix also contains the confidence score c i , which is used to evaluate the model's confidence in the accuracy of each predicted key point position, indicating the reliability of the model's prediction of each key point position, and is mathematically expressed as: C i =c i ; Keypoint identification: Each keypoint has an identifier k i , used to distinguish different body parts, such as head, arms, and legs, and the mathematical expression is: K i =k i ; The prediction matrix is ​​expressed as a multidimensional matrix, where each row represents a key point and each column represents an attribute of the point, spatial coordinates, timestamp, confidence, and identifier, which is mathematically expressed as: Where n is the total number of key points.

Images