Wi-Fi human pose estimation system and method based on physical constraint attention mechanism

Abstract

A Wi-Fi human posture estimation system and method based on a physical constraint attention mechanism, comprising a physical guided multipath perception feature extraction unit, a hierarchical semantic cross fusion unit and a physical feature decoupling domain adaptive processing unit, the present application constructs a unified framework integrating electromagnetic propagation physical mechanism and hierarchical semantic information and introduces a physical guided multipath perception attention mechanism, uses antenna geometry, wavelength dependence and multipath constraint to enhance the feature extraction of weak signals, adopts a hierarchical cross attention fusion strategy, uses high-level action semantics to gradually constrain the solution space of posture estimation, eliminates the ambiguity of signal mapping, and separates static environmental factors and dynamic human signals through a physical feature decoupling domain adaptive method.

Description

Technical Field

[0001] This invention relates to a technology in the field of wireless communication, specifically a Wi-Fi human posture estimation system and method based on a physical constraint attention mechanism. It is not only applicable to Wi-Fi signals, but can also be extended to millimeter-wave radar, 5G / 6G integrated sensing and communication, and ultra-wideband systems. Background Technology

[0002] Existing human pose estimation methods based on Wi-Fi channel state information have three shortcomings: the signal changes caused by human movement are weak and easily affected by noise; the mapping between channel state information and pose is not unique; and static multipath changes in the environment can easily lead to a decline in cross-environment performance. Summary of the Invention

[0003] To address the aforementioned shortcomings of existing technologies, this invention proposes a Wi-Fi human pose estimation system and method based on a physically constrained attention mechanism. It constructs a unified framework integrating electromagnetic propagation physical mechanisms and hierarchical semantic information, and introduces a physically guided multipath sensing attention mechanism. This mechanism utilizes antenna geometry, wavelength dependence, and multipath constraints to enhance feature extraction of weak signals. A hierarchical cross-attention fusion strategy is employed, using high-level action semantics to progressively constrain the solution space of pose estimation. This eliminates the ambiguity of signal mapping while using a domain adaptive method that decouples physical features to separate static environmental factors from dynamic human signals. This addresses the robustness issue of the model when migrating between different environments, thereby achieving high-precision, interference-resistant 3D human pose estimation with environmental generalization capabilities.

[0004] This invention is achieved through the following technical solution:

[0005] This invention relates to a Wi-Fi human pose estimation system based on a physically constrained attention mechanism, comprising: a physically guided multipath sensing feature extraction unit, a hierarchical semantic cross-fusion unit, and a physical feature decoupling domain adaptive processing unit. Specifically: the physically guided multipath sensing feature extraction unit performs CSI physical decomposition, physical location encoding, multipath feasibility mask modulation, and frequency-band attention processing based on collected Wi-Fi channel state information, antenna pair spacing, subcarrier wavelength, and subcarrier frequency information to obtain static CSI components, dynamic CSI components, and multipath sensing features; the hierarchical semantic cross-fusion unit performs action classification, action embedding, dynamic convolutional coarse-grained modulation, and action-pose cross-attention and gating fusion processing based on the multipath sensing features to obtain action embedding vectors and refined pose features; the physical feature decoupling domain adaptive processing unit performs dynamic feature domain invariant learning, static environment feature encoding, environment-specific calibration, and joint-specific decoding processing based on the static CSI components, dynamic CSI components, action embedding vectors, and refined pose features to obtain the three-dimensional coordinates of the human skeletal joints.

[0006] The physical guidance multipath sensing feature extraction unit includes: a CSI physical decomposition module, a physical location coding module, a multipath mask attention module, and a frequency band decomposition coding module. Specifically: the CSI physical decomposition module performs physical decomposition processing on static environmental components and dynamic human body components based on the collected Wi-Fi channel state information to obtain static CSI components and dynamic CSI components; the physical location coding module performs sine and cosine position coding processing based on the dynamic CSI components, antenna pair spacing, subcarrier wavelength, and subcarrier frequency information to obtain enhanced dynamic CSI features; the multipath mask attention module performs multipath feasibility mask initialization, learning, and attention fraction modulation processing based on the enhanced dynamic CSI features, antenna distance, and subcarrier frequency difference to obtain mask modulation attention features; and the frequency band decomposition coding module performs frequency band attention, adaptive weighting, residual connection, and layer normalization processing based on the mask modulation attention features to obtain multipath sensing features.

[0007] The hierarchical semantic cross-fusion unit includes: an action semantic embedding module, a dynamic convolutional coarse modulation module, and a cross-attention gating fusion module. Specifically: the action semantic embedding module performs global pooling, action classification, action probability mapping, and hypernetwork parameter generation based on multipath perception features to obtain action embedding vectors and dynamic convolutional kernels; the dynamic convolutional coarse modulation module performs dynamic convolution, residual network feature extraction, and global pooling based on multipath perception features and dynamic convolutional kernels to obtain coarse-grained pose features; and the cross-attention gating fusion module performs action-pose cross-attention and adaptive gating fusion based on the action embedding vectors and coarse-grained pose features to obtain refined pose features.

[0008] The physical feature decoupling domain adaptive processing unit includes: a dynamic feature invariance learning module, a static environment encoding module, an environment adaptation module, and a joint-specific decoding module. Specifically: the dynamic feature invariance learning module constructs a positive sample set and calculates domain contrast loss based on dynamic CSI components belonging to the same action in different environments, used to constrain the environment invariance of dynamic human features during offline training; the static environment encoding module performs static environment feature encoding based on static CSI components to obtain static environment encoded features; the environment adaptation module generates environment calibration deviations and performs feature calibration based on static environment encoded features and refined posture features to obtain environment-calibrated posture features; and the joint-specific decoding module performs gated weighting of each skeletal joint and two-layer multilayer perceptron decoding based on the environment-calibrated posture features and action embedding vectors to obtain the three-dimensional coordinates of the human skeletal joints.

[0009] This invention relates to a Wi-Fi human pose estimation method based on the above-mentioned system, which is based on physical guided multipath perception and hierarchical semantic fusion. In the offline stage, a deep neural network is constructed and trained to perform action semantic extraction, hierarchical cross-attention fusion, physical decoupling domain adaptation and joint optimization, and in the online stage, real-time human pose estimation is performed.

[0010] The deep neural network includes: a Physically Guided Multipath Aware Attention Mechanism (PGMA) encoder, a hierarchical cross-attention module, an environment encoder, an adapter, an action branch classification module, an action embedding module, a residual network (ResNet), a gated fusion refinement module, and a joint-specific decoder. Specifically: the PGMA encoder explicitly combines antenna array geometry, wavelength-related phase relationships, and multipath constraints to extract multipath awareness features through frequency decomposition; the hierarchical cross-attention module performs action-attention cross-attention processing based on the action embedding vector and coarse-grained pose features to obtain cross-attention features; the environment encoder performs static environment feature encoding processing based on static CSI components to obtain static environment encoded features; the adapter performs environment calibration deviation generation and feature calibration processing based on the static environment encoded features and refined pose features to obtain environmentally calibrated pose features; the action branch classification module... Based on multipath perception features, global pooling and action classification are performed to obtain action probability distributions. The action embedding module performs learnable matrix mapping based on the action probability distributions to obtain action embedding vectors. The ResNet residual network extracts residual features and performs global pooling based on the spatial features after dynamic convolution to obtain coarse-grained pose features. The gated fusion refinement module performs adaptive weighting based on the action embedding vectors, cross-attention features, and coarse-grained pose features to obtain refined pose features. The physical feature decoupling domain adaptive processing unit performs domain contrastive learning, static environment encoding, and environment-specific calibration based on static CSI components, dynamic CSI components, action embedding vectors, and refined pose features to obtain environment-calibrated pose features. The joint-specific decoder performs joint-specific decoding based on the environment-calibrated pose features and action embedding vectors, outputting predicted 3D human skeleton keypoint coordinates.

[0011] The aforementioned CSI decomposition refers to the mathematical decomposition of the received total CSI signal into time-invariant static components caused by reflections from walls and furniture, and dynamic components caused by Doppler frequency shift and path changes resulting from human movement, based on the OFDM channel model and through time-domain and frequency-domain characteristic analysis. This serves as the input basis for physical feature decoupling.

[0012] Technical effect

[0013] This invention uses antenna geometry, subcarrier wavelength, and multipath propagation feasibility as attention constraints, and obtains multipath sensing features through frequency band division processing; it drives dynamic convolution with action embedding and participates in cross-attention, gated fusion, and joint-specific weighting to narrow the candidate solution space from CSI to pose; and it calibrates for different environments through static and dynamic CSI decomposition, dynamic feature domain contrastive learning, environmental encoders, and adapters. These techniques work together in the 3D joint coordinate decoding process. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the system modules of the present invention;

[0015] Figure 2 This is a schematic diagram of the method of the present invention;

[0016] Figure 3 This is a schematic diagram of a scenario for an example embodiment;

[0017] Figure 4 This is a schematic diagram illustrating the effect of an example. Detailed Implementation

[0018] like Figure 1 As shown, this embodiment relates to a Wi-Fi human pose estimation system based on a physical constraint attention mechanism, which includes: a physical-guided multipath sensing feature extraction unit, a hierarchical semantic cross-fusion unit, and a physical feature decoupling domain adaptive processing unit.

[0019] like Figure 2 As shown, this embodiment utilizes the aforementioned system, constructing deep neural networks corresponding to the three units in the system to perform Wi-Fi human pose estimation based on a physical constraint attention mechanism. In the figure, static CSI and dynamic CSI are the outputs of the CSI physical decomposition module; the PGMA encoder corresponds to the physical position encoding module, multipath mask attention module, and frequency band decomposition encoding module in the physical guided multipath perception feature extraction unit, used to extract multipath perception features; action branch classification and action embedding together correspond to the action semantic embedding module in the hierarchical semantic cross-fusion unit, the residual network corresponds to the dynamic convolution coarse modulation module, and hierarchical cross-attention and gated fusion refinement together correspond to the cross-attention gated fusion module; the environment encoder, adapter, and joint-specific decoder correspond to the static environment encoding module, environment adaptation module, and joint-specific decoding module in the physical feature decoupling domain adaptive processing unit, respectively. Kinect V2 is used to acquire the three-dimensional coordinates of human skeletal joints, and the acquired pose labels are used for pose regression supervision during offline training.

[0020] The applications described in this embodiment specifically include:

[0021] Step 1, Signal Modeling and CSI Physical Decomposition: Acquire Channel State Information (CSI) using commercial Wi-Fi devices: For transmitters and receivers at carrier frequency... Channel response under Based on the Orthogonal Frequency Division Multiplexing (OFDM) system model, the system is decomposed into static environment components and dynamic human motion components using the multipath propagation principle. Then, based on the decomposed system model, the static CSI components are extracted. and dynamic CSI components Construct the input feature matrix Specifically: ,in: In time and frequency Channel frequency response under these conditions; Phase rotation caused by carrier frequency offset due to transceiver oscillator This is the frequency offset. Static multipath environmental components generated by reflections from stationary objects such as walls and furniture; The total number of dynamic paths; The first caused by human movement The amplitude change of the dynamic path; No. The length of the dynamic path; The wavelength of electromagnetic waves.

[0022] Step 2: Based on the dynamic CSI components, antenna pair spacing, subcarrier wavelength, and subcarrier frequency information obtained in Step 1, perform multipath sensing feature extraction (PGMA) based on a physically guided multipath sensing attention mechanism to obtain multipath sensing feature Z, specifically including:

[0023] 2.1 Explicitly coded antenna geometry and wavelength dependence as physical priors ,in: Location In frequency The physical location encoding vector below; Antenna pair Spatial distance between them; No. The wavelength of each subcarrier; No. The frequency of each subcarrier; The system's maximum frequency. The physical codes at each location are stacked into a matrix P and mapped onto the feature space to obtain... ; Query vector, key vector, and value vector are generated from enhanced features: , , , and All are learnable projection matrices.

[0024] 2.2 Through Learnable Multipath Feasibility Masks Modulate the attention score, specifically: ,in: Location Position Attention weights; Query vector; Key vector; Feature dimension; Location and There is a prior probability of significant multipath correlation, which is initialized based on Gaussian attenuation of antenna distance and frequency difference and optimized during training; the multipath feasibility mask is initialized as follows. and during training Update, in which and These are the attenuation scales for antenna distance and frequency difference, respectively. This is a learnable correction value.

[0025] 2.3 The spectrum is divided into B frequency bands, each processed by an independent attention head, and the results of each frequency band are adaptively weighted, residually connected, and layer normalized: ,in: , b is the frequency band number This is the encoder serial number. In this embodiment, four layers of PGMA encoders are stacked, and the multipath sensing feature Z is output.

[0026] Step 3: Based on the multipath sensing features Z obtained in Step 2, perform action semantic branching and embedding generation to obtain the action probability distribution, action embedding vector, and dynamic convolution kernel, specifically including:

[0027] 3.1 Features obtained in step 2.3 Global pooling is performed to obtain Predict action probability using the action branch classification module Specifically: , , Where: L is the number of feature locations, and W1, W2, b1, and b2 are the parameters of the action branch classification module.

[0028] 3.2 Map the predicted action probabilities to continuous action embedding vectors And use hypernetworks to generate dynamic convolution kernels. : , ,in: High-dimensional embedding vectors that integrate action semantics; Learnable mapping matrix; The probability distribution vector output by the action branch classification module; The generated dynamic convolution parameters; The hypernetwork parameter matrix is ​​used to generate the convolution kernel weights. The action embedding vector is also used to generate dynamic convolution kernels, construct cross-attention query vectors, and calculate joint-specific gating weights.

[0029] Step 4: Based on the multipath sensing feature Z obtained in Step 2 and the action embedding vector and dynamic convolution kernel obtained in Step 3, perform hierarchical cross-attention fusion from coarse-grained to fine-grained to obtain refined pose features, specifically including:

[0030] 4.1 Coarse-grained modulation: Utilizing dynamic convolution kernels Convolve the spatial features obtained in step 2.3 to obtain the features. Specifically: ,in: The spatial features are obtained by Z-rearrangement of multipath sensing features.

[0031] 4.2 Refined Coupling: The association between action semantics and pose features is calculated through cross-attention, specifically as follows: ,in: Cross-attention features enhanced with action semantics; Query vectors generated by action embeddings; These are the key vector and value vector generated from coarse-grained pose features, respectively. This is the scaling factor. Further calculation of the gating weights is needed. and obtain ,in: This is the gated projection matrix.

[0032] 4.3 Refine the pose feature output: Use the gated fusion features obtained in step 4.2 as the refined pose features. and the refined pose features and the action embedding vector obtained in step 3 Proceed to step 5 for environmental calibration and joint-specific decoding.

[0033] Step 5: Using the dynamic CSI components and static CSI components obtained in Step 1, perform domain-invariant learning and environment-specific adaptation respectively to obtain domain-invariant dynamic features and environment-calibrated attitude features, specifically including:

[0034] 5.1 Dynamic Feature Invariance Learning: Utilizing Contrastive Learning Loss To ensure that the dynamic characteristics of the same action remain consistent across different environments, specifically: ,in: Domain-invariant contrast loss, For batch size, No. Dynamic CSI features of each sample With sample A set of positive samples that belong to the same action but in different environments. The cosine similarity function, where τ is the temperature coefficient. This is an indicator function. In this embodiment, the temperature coefficient τ is taken as 0.07.

[0035] 5.2 Environment-Specific Adaptation and Joint-Specific Decoding: For a specific environment d, the static CSI components are... Input the static environment encoding module to obtain the static environment encoding features. ; Based on static environment coding characteristics through a lightweight environment adapter Generation environment calibration deviation And the environmental calibration deviation and refined attitude characteristics were combined. Add them together to obtain the attitude characteristics after environmental calibration. For different skeletal joints, joint gating weights are calculated to determine their dependence on action semantics. Joint features The joint features f_j are decoded into three-dimensional coordinates of the human skeletal joints by two multilayer perceptrons corresponding to each skeletal joint.

[0036] Step 6: Based on the action probability distribution obtained in Step 3, the three-dimensional coordinates of the human skeletal joints obtained in Step 5, and the domain contrast loss calculated in Step 5, construct a joint optimization objective for pose regression loss, action classification loss, and domain contrast loss to obtain the trained deep neural network, specifically including:

[0037] 6.1 Construct an optimization objective that jointly minimizes the pose regression loss, action classification loss, and domain contrast loss, specifically: ,in: Total loss function, Mean Joint Position Regression Loss (MPJPE) for Pose Estimation Cross-entropy loss for action classification, Domain-adaptive contrast loss, These are the balancing hyperparameters for action loss and domain loss, respectively.

[0038] In this embodiment .

[0039] 6.2 During training, the pose regression loss is calculated using 3D pose labels, the action classification loss is calculated using action labels, and a positive sample set is constructed for samples of the same action in different environments to calculate the domain contrast loss. Backpropagation is performed based on the total loss to update the learnable parameters participating in training in the physical guided multipath perception feature extraction unit, the hierarchical semantic cross-fusion unit, and the physical feature decoupling domain adaptive processing unit, thus obtaining the trained deep neural network.

[0040] Step 7, Online Estimation: Collect CSI data from the target environment and perform physical decomposition as in Step 1 to obtain static and dynamic CSI components. Input the dynamic CSI components into the trained physical-guided multipath perception feature extraction unit and hierarchical semantic cross-fusion unit to obtain action embedding vectors and refined pose features. Input the static CSI components into the static environment encoding module and environment adaptation module to generate environment calibration bias. Add the environment calibration bias to the refined pose features to obtain the environment-calibrated pose features. Input the environment-calibrated pose features and action embedding vectors into the joint-specific decoding module to output the three-dimensional coordinates of the human skeletal joints.

[0041] This embodiment uses a computing device configured with an Intel 5300 network interface card, and is equipped with one transmit antenna and three receive antennas. The CSI sampling frequency is 600 Hz, and the ground truth 3D pose data is synchronously acquired by a camera at 30 fps. Every 20 consecutive CSI data packets are grouped together to align with the video frame rate. The self-built dataset covers two indoor scenes and eight human actions, obtaining a total of 28,406 aligned samples. Validation was also performed using the Person-in-WiFi-3D dataset and the MM-Fi dataset: the former contains approximately 90,000 training samples and 7,800 test samples from seven volunteers performing eight actions at three locations, providing 3D annotations for 14 keypoints; the latter contains over 320,000 frames of data from 40 subjects performing 27 actions in four environments, providing annotations for 17 keypoints.

[0042] This experiment uses the collected and processed CSI data as input. First, it separates the data into static environment components and dynamic human body components through physical CSI decomposition. Then, it inputs the deep neural network of this invention for feature extraction and pose estimation, such as... Figure 1 As shown.

[0043] To verify the beneficial effects of this embodiment, a comprehensive comparative experiment was conducted with state-of-the-art methods (MetaFi++, HPE-Li, DT-Pose). This invention achieves superior performance in most experimental settings and evaluation metrics across self-built real-world scenarios, Person-in-WiFi-3D, and large-scale MM-Fi datasets, particularly demonstrating a significant advantage in the PA-MPJPE metric: its performance on real-world prototype datasets is shown in Table 1, where this method exhibits a significant advantage in real-world scenarios. Regarding error metrics, the mean joint position error (MPJPE) is significantly reduced to 142.1 mm, and the aligned mean joint position error (PA-MPJPE) is reduced to 103.5 mm. Compared to the competitive DT-Pose (MPJPE 162.3 mm), this invention significantly improves positioning accuracy. Simultaneously, the PCK@50 accuracy reaches 94.1%, demonstrating the reliability of the system in practical deployments.

[0044] Table 1. Performance comparison of our method with existing techniques on real-world prototype datasets.

[0045] Performance on the Person-in-Wi-Fi-3D Dataset: Person-in-Wi-Fi-3D is the first Wi-Fi-based end-to-end multi-person 3D pose estimation dataset. As shown in Table 2, our method achieves an MPJPE error of 82.5 mm on this dataset, which is 8.3% lower than the previous best-performing DT-Pose (90.0 mm), and the PA-MPJPE is only 53.8 mm. This indicates that even with multi-person interference and different environmental layouts, our model can still accurately capture human poses.

[0046] Table 2. Performance comparison of our method with existing techniques on the Person-in-Wi-Fi-3D dataset.

[0047] Performance on the MM-Fi dataset: MM-Fi is currently the largest and most challenging multimodal wireless sensing benchmark. As shown in Table 3, the present invention outperforms existing technologies in most experimental settings and metrics, especially demonstrating a significant advantage in the PA-MPJPE metric.

[0048] Table 3. Performance comparison of our method with existing techniques on the MM-Fi dataset (overall results)

[0049] In random partitioning (S1): Under Protocol 3 (P3) which includes all activities, the present invention achieves 155.5 mm for MPJPE and up to 93.0% for PCK@50, which is significantly better than HPE-Li and DT-Pose.

[0050] In cross-subject (S2): Even when facing unseen users, the MPJPE of this invention under the P3 protocol was still controlled at 188.1 mm, demonstrating the model's generalization ability to people of different body types.

[0051] In cross-environment (S3): In the most challenging cross-environment test, although the absolute error (MPJPE) of all methods increases, the PA-MPJPE (alignment error) of this invention under the P3 protocol is only 78.5 mm, far lower than DT-Pose's 106.2 mm. This demonstrates that the invention successfully decouples environmental features while preserving the correct human skeletal structure.

[0052] To address the most challenging domain offset problem in wireless sensing, this invention introduces a domain adaptation module. By explicitly separating the static environment component and dynamic human component in the CSI signal and combining it with domain contrast loss, the model is forced to learn essential pose features independent of the environment. As shown in the data in section S3 (cross-environment) of the table, in a completely unseen test environment, the PA-MPJPE error of this method (78.5 mm) is reduced by approximately 26% compared to the state-of-the-art method DT-Pose (106.2 mm). This means that even in a new room, the skeleton structure generated by this invention remains accurate and does not exhibit severe deformation.

[0053] This invention innovatively proposes a Physically Guided Multipath Awareness Attention (PGMA) mechanism. This mechanism utilizes the physical constraints of wireless signal propagation (such as antenna array geometry and wavelength phase relationships) to guide the attention distribution of the neural network, focusing it on the actual human reflection path rather than environmental multipath noise. This physical prior effectively avoids the limb distortion and biomechanically incompatible predictions common in purely data-driven methods, making pose estimation more robust and smooth in low signal-to-noise ratio environments. The visualization results are shown below. Figure 4 As shown.

[0054] Compared with existing technologies, this method reduces MPJPE to 142.1 mm and PA-MPJPE to 103.5 mm on a self-built real-world scene dataset through the collaborative processing of a physically guided multipath sensing feature extraction unit, a hierarchical semantic cross-fusion unit, and a physical feature decoupling domain adaptive processing unit; reduces MPJPE to 82.5 mm and PA-MPJPE to 53.8 mm on a Person-in-WiFi-3D dataset; and reduces PA-MPJPE from 104.8 mm in DT-Pose to 69.5 mm in MM-Fi random partitioning of S1 protocol 3.

[0055] The above-described specific implementations can be partially adjusted by those skilled in the art in different ways without departing from the principles and purpose of the present invention. The scope of protection of the present invention is defined by the claims and is not limited to the above-described specific implementations. All implementation schemes within the scope of the claims are bound by the present invention.

Claims

1. A Wi-Fi human pose estimation system based on a physically constrained attention mechanism, characterized in that, include: The system comprises a physically guided multipath sensing feature extraction unit, a hierarchical semantic cross-fusion unit, and a physical feature decoupling domain adaptive processing unit. Specifically: the physically guided multipath sensing feature extraction unit performs CSI physical decomposition, physical location encoding, multipath feasibility mask modulation, and frequency-band attention processing based on collected Wi-Fi channel state information, antenna pair spacing, subcarrier wavelength, and subcarrier frequency information to obtain static CSI components, dynamic CSI components, and multipath sensing features; the hierarchical semantic cross-fusion unit performs action classification, action embedding, dynamic convolutional coarse-grained modulation, and action-pose cross-attention and gating fusion processing based on the multipath sensing features to obtain action embedding vectors and refined pose features; and the physical feature decoupling domain adaptive processing unit performs dynamic feature domain invariant learning, static environment feature encoding, environment-specific calibration, and joint-specific decoding processing based on the static CSI components, dynamic CSI components, action embedding vectors, and refined pose features to obtain the three-dimensional coordinates of the human skeletal joints.

2. The Wi-Fi human pose estimation system based on a physically constrained attention mechanism according to claim 1, characterized in that, The physical guidance multipath sensing feature extraction unit includes: a CSI physical decomposition module, a physical location coding module, a multipath mask attention module, and a frequency band decomposition coding module. Specifically: the CSI physical decomposition module performs physical decomposition processing on static environmental components and dynamic human body components based on the collected Wi-Fi channel state information to obtain static CSI components and dynamic CSI components; the physical location coding module performs sine and cosine position coding processing based on the dynamic CSI components, antenna pair spacing, subcarrier wavelength, and subcarrier frequency information to obtain enhanced dynamic CSI features; the multipath mask attention module performs multipath feasibility mask initialization, learning, and attention fraction modulation processing based on the enhanced dynamic CSI features, antenna distance, and subcarrier frequency difference to obtain mask modulation attention features; and the frequency band decomposition coding module performs frequency band attention, adaptive weighting, residual connection, and layer normalization processing based on the mask modulation attention features to obtain multipath sensing features.

3. The Wi-Fi human pose estimation system based on a physically constrained attention mechanism according to claim 1, characterized in that, The hierarchical semantic cross-fusion unit includes: an action semantic embedding module, a dynamic convolutional coarse modulation module, and a cross-attention gating fusion module. Specifically: the action semantic embedding module performs global pooling, action classification, action probability mapping, and hypernetwork parameter generation based on multipath perception features to obtain action embedding vectors and dynamic convolutional kernels; the dynamic convolutional coarse modulation module performs dynamic convolution, residual network feature extraction, and global pooling based on multipath perception features and dynamic convolutional kernels to obtain coarse-grained pose features; and the cross-attention gating fusion module performs action-pose cross-attention and adaptive gating fusion based on the action embedding vectors and coarse-grained pose features to obtain refined pose features.

4. The Wi-Fi human pose estimation system based on a physically constrained attention mechanism according to claim 1, characterized in that, The physical feature decoupling domain adaptive processing unit includes: a dynamic feature invariance learning module, a static environment encoding module, an environment adaptation module, and a joint-specific decoding module. Specifically: the dynamic feature invariance learning module constructs a positive sample set and calculates domain contrast loss based on dynamic CSI components belonging to the same action in different environments, used to constrain the environment invariance of dynamic human features during offline training; the static environment encoding module performs static environment feature encoding based on static CSI components to obtain static environment encoded features; the environment adaptation module generates environment calibration deviations and performs feature calibration based on static environment encoded features and refined posture features to obtain environment-calibrated posture features; and the joint-specific decoding module performs gated weighting of each skeletal joint and two-layer multilayer perceptron decoding based on the environment-calibrated posture features and action embedding vectors to obtain the three-dimensional coordinates of the human skeletal joints.

5. A Wi-Fi human pose estimation method based on a physically constrained attention mechanism according to any one of claims 1-4, characterized in that, In the offline phase, a deep neural network is constructed and trained to perform action semantic extraction, hierarchical cross-attention fusion, physical decoupling domain adaptation, and joint optimization. In the online phase, real-time human pose estimation is performed. The deep neural network includes: a Physically Guided Multipath Aware Attention Mechanism (PGMA) encoder, a hierarchical cross-attention module, an environment encoder, an adapter, an action branch classification module, an action embedding module, a residual network (ResNet), a gated fusion refinement module, and a joint-specific decoder. Specifically: the PGMA encoder explicitly combines antenna array geometry, wavelength-related phase relationships, and multipath constraints to extract multipath awareness features through frequency decomposition; the hierarchical cross-attention module performs action-attention cross-attention processing based on the action embedding vector and coarse-grained pose features to obtain cross-attention features; the environment encoder performs static environment feature encoding processing based on static CSI components to obtain static environment encoded features; the adapter performs environment calibration deviation generation and feature calibration processing based on the static environment encoded features and refined pose features to obtain environmentally calibrated pose features; the action branch classification module... Based on multipath perception features, global pooling and action classification are performed to obtain action probability distributions. The action embedding module performs learnable matrix mapping based on the action probability distributions to obtain action embedding vectors. The ResNet residual network extracts residual features and performs global pooling based on the spatial features after dynamic convolution to obtain coarse-grained pose features. The gated fusion refinement module performs adaptive weighting based on the action embedding vectors, cross-attention features, and coarse-grained pose features to obtain refined pose features. The physical feature decoupling domain adaptive processing unit performs domain contrastive learning, static environment encoding, and environment-specific calibration based on static CSI components, dynamic CSI components, action embedding vectors, and refined pose features to obtain environment-calibrated pose features. The joint-specific decoder performs joint-specific decoding based on the environment-calibrated pose features and action embedding vectors, outputting predicted 3D human skeleton keypoint coordinates.

6. The Wi-Fi human pose estimation method based on a physically constrained attention mechanism according to claim 5, characterized in that, specifically... include: Step 1, Signal Modeling and CSI Physical Decomposition: Acquire Channel State Information (CSI) using commercial Wi-Fi devices: For transmitters and receivers at carrier frequency... Channel response under Based on the Orthogonal Frequency Division Multiplexing (OFDM) system model, the system is decomposed into static environment components and dynamic human motion components using the multipath propagation principle. Then, based on the decomposed system model, the static CSI components are extracted. and dynamic CSI components Construct the input feature matrix Specifically: ,in: In time and frequency Channel frequency response under these conditions; Phase rotation caused by carrier frequency offset due to transceiver oscillator This is the frequency offset; Static multipath environmental components generated by reflections from stationary objects such as walls and furniture; The total number of dynamic paths; The first caused by human movement The amplitude change of the dynamic path; No. The length of the dynamic path; The wavelength of electromagnetic waves; Step 2: Extract multipath perception features based on a physical-guided multipath perception attention mechanism; Step 3: Generate through action semantic branching and embedding; Step 4: Employ a hierarchical cross-attention fusion approach, from coarse-grained to fine-grained. Step 5: Use the dynamic CSI components and static CSI components obtained from Step 1 to perform domain-invariant learning and environment-specific adaptation respectively. Step 6: Joint optimization objective, i.e., constructing an optimization objective that jointly minimizes the pose regression loss, action classification loss, and domain contrast loss. The pose regression loss is calculated using 3D pose labels, the action classification loss is calculated using action labels, and a positive sample set is constructed for samples of the same action in different environments to calculate the domain contrast loss. Backpropagation is performed based on the total loss to update the learnable parameters of the physical position projection matrix, multipath feasibility mask, PGMA encoder, action branch, fusion module, environment adapter, and joint-specific decoder, resulting in the trained pose estimation network. ,in: Total loss function, Mean joint position regression loss (MPJPE) for pose estimation Cross-entropy loss for action classification, Domain-adaptive contrast loss, These are the balancing hyperparameters for action loss and domain loss, respectively; Step 7, Online Estimation: Acquire CSI in the target environment, obtaining static and dynamic CSI components as in Step 1; input the dynamic CSI components into the trained PGMA encoder and perform action semantic extraction, dynamic convolution, cross attention, and gating fusion; input the static CSI components into the static environment encoding module to obtain static environment encoded features, and generate environment calibration bias through the environment adapter; add the environment calibration bias to the refined pose features to obtain the environment-calibrated pose features; input the environment-calibrated pose features and action embedding vector into the joint-specific decoder to output the three-dimensional coordinates of the human skeletal joints.

7. The Wi-Fi human pose estimation method based on a physically constrained attention mechanism according to claim 6, characterized in that, Step 2 specifically includes: 2.1 Explicitly coded antenna geometry and wavelength dependence as physical priors ,in: Location In frequency The physical location encoding vector below; Antenna pair Spatial distance between them; No. The wavelength of each subcarrier; No. The frequency of each subcarrier; The system's maximum frequency is obtained by stacking the physical codes at each location into a matrix P and mapping it to the feature space. ; Query vector, key vector, and value vector are generated from enhanced features: , , , and All are learnable projection matrices; 2.2 Through Learnable Multipath Feasibility Masks Modulate the attention score, specifically: ,in: Location Position Attention weights; Query vector; Key vector; Feature dimension; Location and There is a prior probability of significant multipath correlation, which is initialized based on Gaussian attenuation of antenna distance and frequency difference and optimized during training; the multipath feasibility mask is initialized as follows. and during training Update, including: and These are the attenuation scales for antenna distance and frequency difference, respectively. This is a learnable correction value; 2.3 The spectrum is divided into B frequency bands, each processed by an independent attention head, and the results of each frequency band are adaptively weighted, residually connected, and layer normalized: ,in: b is the frequency band number. This is the coding layer number.

8. The Wi-Fi human pose estimation method based on a physically constrained attention mechanism according to claim 6, characterized in that, Step 3 specifically includes: 3.1 Features obtained in step 2.3 Global pooling is performed to obtain Predict action probability using the action branch classification module Specifically: , , Where: L is the number of feature locations, and W1, W2, b1, and b2 are the parameters of the action branch classification module; 3.2 Map the predicted action probabilities to continuous action embedding vectors And use hypernetworks to generate dynamic convolution kernels. : , ,in: High-dimensional embedding vectors that integrate action semantics; Learnable mapping matrix; The probability distribution vector output by the action branch classification module; The generated dynamic convolution parameters; The hypernetwork parameter matrix is ​​used to generate convolution kernel weights, and the action embedding vector is also used to generate dynamic convolution kernels, construct cross-attention query vectors, and calculate joint-specific gating weights.

9. The Wi-Fi human pose estimation method based on a physically constrained attention mechanism according to claim 6, characterized in that, Step 4 specifically includes: 4.1 Coarse-grained modulation: Utilizing dynamic convolution kernels Convolve the spatial features obtained in step 2.3 to obtain the features. Specifically: ,in: Spatial features obtained by Z-rearrangement of multipath sensing features; 4.2 Refined Coupling: The association between action semantics and pose features is calculated through cross-attention, specifically as follows: ,in: Cross-attention features enhanced with action semantics; Query vectors generated by action embeddings; These are the key vector and value vector generated from coarse-grained pose features, respectively. The scaling factor is used to further calculate the gating weights. and obtain ,in: For gated projection matrix; 4.3 Refine the pose feature output: Use the gated fusion features obtained in step 4.2 as the refined pose features. Output, and the refined pose features and the action embedding vector obtained in step 3 Proceed to step 5 for environmental calibration and joint-specific decoding.

10. The Wi-Fi human pose estimation method based on a physically constrained attention mechanism according to claim 6, characterized in that, Step 5 specifically includes: 5.1 Dynamic Feature Invariance Learning: Utilizing Contrastive Learning Loss To ensure that the dynamic characteristics of the same action remain consistent across different environments, specifically: ,in: Domain-invariant contrast loss, For batch size, No. Dynamic CSI features of each sample With sample A set of positive samples that belong to the same action but in different environments. The cosine similarity function, where τ is the temperature coefficient. For indicator functions; 5.2 Environment-Specific Adaptation and Joint-Specific Decoding: For a specific environment d, the static CSI components are... Input the static environment encoding module to obtain the static environment encoding features. ; Based on static environment coding characteristics through a lightweight environment adapter Generation environment calibration deviation And the environmental calibration deviation and refined attitude characteristics were combined. Add them together to obtain the attitude characteristics after environmental calibration. For different skeletal joints, joint gating weights are calculated to determine their dependence on action semantics. Joint features The joint features are processed by two multilayer perceptrons corresponding to each skeletal joint. Decoded into three-dimensional coordinates of human skeletal joints.

Images