Efficient through-wall human pose estimation method and device based on mamba network

Abstract

The application discloses a kind of efficient wall-penetrating human posture estimation method and device based on Mamba network belongs to wall-penetrating radar posture estimation technical field.The method includes: collecting wall-penetrating radar multichannel complex echo, after real and imaginary part split, distance pruning and static clutter suppression, splice into network input;End-to-end posture estimation network is constructed, which contains structure alignment embedding module, generates weight along distance dimension and channel dimension using double-domain re-calibration mechanism to strengthen effective features;The enhanced features are down-sampled to a fixed length distance token sequence and added to the position encoding;The sequence is efficiently modeled for long-range dependencies using a Mamba encoder based on a selective state-space model;The encoded features are globally aggregated and output three-dimensional human keypoint coordinates through a posture regression head.The application significantly reduces computational overhead and inference latency while ensuring posture estimation accuracy, suitable for public safety, emergency rescue and other wall-penetrating detection scenarios.

Description

Technical Field

[0001] This invention belongs to the field of through-wall radar attitude estimation technology, specifically relating to an efficient through-wall human attitude estimation method and device based on Mamba network. Background Technology

[0002] Through-wall human pose estimation is a key technology in public safety, emergency rescue, and building security. It utilizes the penetrating properties of electromagnetic waves to obtain the echo of a person behind a wall and infer their pose, even under conditions of wall obstruction, low light, or smoke. However, traditional methods typically employ a two-stage approach: explicit imaging / reconstruction + pose estimation. First, a radar image is generated using algorithms such as back-projection, and then pose regression is performed in the image domain. This type of method has a complex processing flow, high computational intensity, and the error from the previous imaging stage is propagated to the next, making it difficult to meet the requirements for real-time or low-power deployment.

[0003] In recent years, end-to-end methods based on deep learning have received widespread attention. For example, existing literature has proposed a Transformer-based method for 3D human pose estimation using through-wall radar, which globally models the radar signal through a self-attention mechanism and directly outputs the pose result. However, through-wall radar echoes typically exhibit a coupled structure of "virtual channel dimension × range dimension × time frame." When the range resolution increases or multiple frames are introduced, the sequence length increases significantly. The computational complexity of Transformer's self-attention increases quadratically with the sequence length, leading to a sharp increase in computational and storage overhead, which limits its application in resource-constrained scenarios.

[0004] Furthermore, regarding data annotation, since radar signals are difficult to annotate manually for key points directly, existing research generally adopts a cross-modal supervision approach. This involves simultaneously acquiring visual data and generating supervision signals using a visual pose model. After training, the inference phase only retains the radar input to output the pose. This paradigm provides a feasible data construction foundation for through-wall radar pose estimation, but how to design an efficient network structure to adapt to the characteristics of radar signals and reduce computational complexity remains a pressing technical problem that needs to be solved.

[0005] In summary, existing methods suffer from low computational efficiency and high resource consumption when processing long sequences of radar signals. There is an urgent need for an efficient attitude estimation scheme that can maintain global modeling capabilities while achieving linear complexity. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides an efficient through-wall human pose estimation method and apparatus based on Mamba networks. Starting directly from the complex echoes of through-wall radar, it constructs a sequence of range dimensions and introduces it into Mamba sequence modeling based on a selective state space model. This enables long-range dependency acquisition that expands linearly with the sequence length, thereby significantly improving inference efficiency and speed while ensuring the global information modeling capability required for pose estimation, and reducing overall latency and resource consumption.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] An efficient method for human pose estimation through walls based on Mamba networks, the method comprising:

[0009] Step 1: Establish a method for constructing data acquisition and supervision information from through-wall radar. Collect multi-channel complex echo data of the human body behind the wall using through-wall radar, and simultaneously acquire attitude supervision information corresponding to the echo data.

[0010] Step 2: Preprocess the complex echo data to obtain the network input tensor;

[0011] Step 3: Construct an end-to-end pose estimation network, which includes a structure alignment embedding module, a sequence encoding module, and a pose regression module. The structure alignment embedding module performs channel information fusion and dual-domain recalibration on the network input tensor to obtain enhanced features. The enhanced features are downsampled along the distance dimension into a fixed-length distance token sequence, and after adding position encoding, they are input into the sequence encoding module. The distance token sequence is modeled for long-range dependencies using a Mamba encoder based on a selective state-space model to obtain the encoded sequence features.

[0012] Step 4: Globally aggregate the encoded sequence features and output the coordinates of the three-dimensional key points of the human body through the pose regression module.

[0013] Furthermore, step 1 includes: the radar adopts a MIMO system to form a virtual channel array to obtain spatial diversity, and collects continuous multi-frame sequences to cover the echo changes of the human body under different postures; simultaneously collects external sensors or external modal sequences, and generates human body key point coordinates or key point heat maps as training supervision signals through a preset posture estimation model.

[0014] Furthermore, step 2 includes: splitting the complex echo data into real and imaginary parts; cropping the distance dimension to focus on the distance range related to human activity; suppressing static clutter by background subtraction, moving average, or high-pass filtering; and splicing the processed real and imaginary parts along the channel dimension to form a network input tensor, and performing amplitude normalization or standardization processing.

[0015] Furthermore, in step 3, the network input tensor is fused with channel information and recalibrated in a dual-domain manner through the structure alignment embedding module to obtain enhanced features. This includes: using 1×1 convolution to fuse channel information and reduce the channel dimension to a preset dimension; the dual-domain recalibration includes: the first branch aggregates along the channel dimension to obtain a distance descriptor, which is then converted into distance weights after local one-dimensional convolution; the second branch aggregates along the distance dimension to obtain a channel descriptor, which is then converted into channel weights after local cross-channel interaction; the recalibration results of the two branches are then fused in a learnable manner and normalized at each layer to output enhanced features.

[0016] Furthermore, in step 3, the enhanced features are downsampled along the distance dimension into a fixed-length distance token sequence, and after adding position encoding, they are input into the sequence encoding module. This includes: compressing the distance dimension to a fixed length L using adaptive average pooling or adaptive max pooling to obtain the distance token sequence; mapping each token to the latent space dimension D through linear projection; and superimposing sine / cosine position encoding or learnable position encoding on the distance token sequence to obtain a position-aware sequence representation.

[0017] Furthermore, in step 3, the distance token sequence is modeled for long-range dependency using a Mamba encoder based on a selective state-space model to obtain encoded sequence features. This includes: adopting a bidirectional structure, with the forward branch encoding in the original distance order and the reverse branch reversing the distance token sequence along the length dimension before encoding and then reversing it back to the original order; and performing learnable fusion and layer normalization on the outputs of the forward and reverse branches to obtain encoded sequence features.

[0018] Furthermore, step 4 includes: using mean pooling or weighted pooling of the token dimension to form a global representation; the pose regression module includes a pose regression head and a skeleton topology refinement module, which first outputs the initial three-dimensional key point coordinates through a fully connected layer, then constructs the human skeleton topology with joints as nodes and bone connections as edges, and performs propagation and residual refinement through a graph convolutional network to output the refined key point coordinates.

[0019] On the other hand, the present invention provides a high-efficiency through-wall human pose estimation device based on a Mamba network, comprising:

[0020] The acquisition module is used to establish a method for acquiring and supervising data from through-wall radar. It acquires multi-channel complex echo data of the human body behind the wall through through-wall radar and simultaneously obtains attitude supervision information corresponding to the echo data.

[0021] The preprocessing module is used to preprocess the complex echo data to obtain the network input tensor;

[0022] The processing module is used to construct an end-to-end pose estimation network, which includes a structure alignment embedding module, a sequence encoding module, and a pose regression module. The structure alignment embedding module performs channel information fusion and dual-domain recalibration on the network input tensor to obtain enhanced features. The enhanced features are downsampled along the distance dimension into a fixed-length distance token sequence, and after adding position encoding, they are input into the sequence encoding module. The distance token sequence is then modeled for long-range dependencies using a Mamba encoder based on a selective state-space model to obtain the encoded sequence features.

[0023] The output module is used to globally aggregate the encoded sequence features and output the coordinates of the three-dimensional key points of the human body through the pose regression module.

[0024] Thirdly, the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned efficient through-wall human pose estimation method based on Mamba network.

[0025] Fourthly, the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enable the processor to implement the aforementioned efficient wall-penetrating human pose estimation method based on a Mamba network.

[0026] The beneficial effects of this invention are as follows:

[0027] 1. This invention adopts an end-to-end design approach, directly using the complex echo of through-wall radar as input, without the need for explicit imaging or reconstruction stages. This avoids the problem of the previous stage imaging error being passed to the subsequent stage attitude regression in the traditional two-stage method, while simplifying the processing flow and reducing system complexity.

[0028] 2. This invention addresses the coupling characteristics of "range dimension - virtual channel dimension" in through-wall radar echoes by designing a structure alignment embedding and dual-domain recalibration module. By generating range weights and channel weights early in feature extraction, it adaptively strengthens the effective range interval and key channel combination, effectively suppressing interference from strong wall echoes and multipath clutter, and providing a more discriminative input representation for subsequent sequence coding.

[0029] 3. This invention introduces a Mamba sequence encoder based on a selective state-space model, which constructs the distance dimension as a fixed-length token sequence for efficient long-range dependency modeling. This makes the computational complexity expand nearly linearly with the sequence length, significantly reducing computational and storage overhead compared to the quadratic complexity of the traditional Transformer, improving inference speed, and making it more suitable for real-time or resource-constrained deployment scenarios.

[0030] 4. This invention can optionally introduce a bidirectional branch with distance axis reversal to enhance distance context awareness, and use a skeleton topology refinement module to perform graph convolution propagation and residual refinement on the output key points, thereby improving the geometric consistency and skeleton coherence of the attitude output, and achieving high-precision attitude estimation while maintaining a low number of parameters. Attached Figure Description

[0031] Figure 1 This is a flowchart of an efficient through-wall human pose estimation method based on Mamba network according to the present invention.

[0032] Figure 2 This is a network architecture diagram for structural alignment feature extraction and fixed-length sequence construction in this invention;

[0033] Figure 3 This is a diagram of the pose regression network architecture combining Mamba and GCN in this invention;

[0034] Figure 4 This is a schematic diagram of a data acquisition experiment scenario in an embodiment of the present invention;

[0035] Figure 5 This is a schematic diagram of the through-wall radar system used in an embodiment of the present invention;

[0036] Figure 6 This is a multi-channel radar echo signal amplitude diagram in an embodiment of the present invention;

[0037] Figure 7 This is a comparison chart of the test set model output results and the actual labels in an embodiment of the present invention. Detailed Implementation

[0038] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0039] like Figure 1 As shown, this invention provides an efficient through-wall human pose estimation method based on Mamba networks. It employs an end-to-end approach, directly using complex echoes from through-wall radar as input. Through structure-aligned embedding and dual-domain recalibration, it highlights effective range intervals and effective channel combinations in the early stages of feature extraction, reducing interference from strong wall echoes, multipath propagation, and clutter on pose features. Simultaneously, it constructs a fixed-length token sequence for the range dimension and introduces Mamba sequence modeling based on a selective state-space model to achieve long-range dependency capture. This makes the sequence modeling overhead expand nearly linearly with length, thereby significantly reducing computational and storage pressure and improving inference speed while maintaining global modeling capabilities. Furthermore, cross-modal supervision can be optionally used during the training phase to reduce the difficulty of pose supervision acquisition. After training, only radar input is needed to output the key points of the human 3D pose during the inference phase. Optional skeleton topology refinement further improves geometric consistency and stability. The method mainly includes the following steps:

[0040] Step 1: Establish a method for constructing data acquisition and supervision information from through-wall radar. Collect multi-channel complex echo data of the human body behind the wall using through-wall radar, and simultaneously acquire attitude supervision information corresponding to the echo data.

[0041] Step 2: Preprocess the complex echo data to obtain the network input tensor;

[0042] Step 3: Construct an end-to-end pose estimation network, which includes a structure alignment embedding module, a sequence encoding module, and a pose regression module. The structure alignment embedding module performs channel information fusion and dual-domain recalibration on the network input tensor to obtain enhanced features. The dual-domain recalibration includes generating distance weights along the distance dimension and generating channel weights along the channel dimension. The enhanced features are downsampled along the distance dimension into a fixed-length distance token sequence, and after adding position encoding, are input into the sequence encoding module. A Mamba encoder based on a selective state-space model is used to model the long-range dependencies of the distance token sequence to obtain the encoded sequence features.

[0043] Step 4: Globally aggregate the encoded sequence features and output the coordinates of the three-dimensional key points of the human body through the pose regression module.

[0044] Further, step 1 includes:

[0045] The data acquisition scenario was set within a longitudinal distance of 2-6 meters from the radar and camera, with a left and right field of view of -3 to 3 meters. Subjects of different body types and heights were selected. The experimental scenario is as follows: Figure 4 As shown, subjects performed various actions within the scene, including basic behaviors such as standing, raising both arms, squatting, and walking. Through-wall radar collected human echo data from behind walls; the radar could employ a MIMO system to form a virtual channel array to achieve spatial diversity, such as... Figure 5 As shown, each acquisition involves multiple consecutive frames to cover echo changes in different postures and motion states of the human body. To construct posture supervision information, external sensors or external modal sequences can be simultaneously acquired to generate human body keypoint coordinates or keypoint heatmaps as training supervision signals. These supervision signals are used to constrain the network output during the training phase, enabling the network to learn the mapping relationship between radar echoes and human posture. The source, specific generation method, and synchronization means of the external supervision information are not limited and can be selected according to experimental conditions. This experiment uses the Orbbec Mega depth camera and Microsoft's accompanying posture estimation model to obtain high-precision skeletal point labels. After training, when deploying inference, only the through-wall radar input is retained to output the posture estimation results.

[0046] Further, step 2 includes:

[0047] The raw echo from through-wall radar is in complex form. To facilitate deep network processing, the complex echo is split into real and imaginary parts; let the number of virtual channels be C, and the number of original range sampling points be... The original input of a single frame can then be represented as a real matrix and an imaginary matrix. To reduce redundancy and focus on the range interval relevant to human activity, the range dimension can be truncated to obtain a range sub-interval of length R. To suppress static clutter from walls and the environment, background subtraction is performed when a reference background frame exists to obtain the clutter-suppressed input. The amplitude of the multi-channel echo signal of the radar data after this step is as follows: Figure 6 As shown; then the real and imaginary parts are concatenated along the channel dimension to form the network input tensor, enabling the network to utilize both amplitude and phase-related information simultaneously. The data preprocessing is specifically represented as follows:

[0048] ,

[0049] ,

[0050] ,

[0051] ,

[0052] in, The matrix representing the real part of the original echo data. The matrix representing the imaginary part of the original echo data; Indicates the number of virtual channels. This represents the number of original distance sampling points, therefore and All dimensions are . and Let these represent the real and imaginary parts of the matrix after distance-dimensional clipping, respectively; where the symbol " "This means retaining all channels and intercepting the previous ones." Each distance sampling point, This indicates the length of the distance dimension after clipping. and They represent the times at time 1 and 2 respectively. Real and imaginary inputs after background suppression; and These represent the real and imaginary matrices of the reference background frame, respectively. This represents the tensor that is ultimately input into the network. This represents a concatenation operation along the channel dimension, therefore the concatenated input dimension is... Among them, the former Each channel corresponds to the real part information, then... Each channel corresponds to the imaginary part information. Through the above processing, the network input retains the distance dimension structural information while jointly utilizing the amplitude and phase correlation features in the complex echo, providing a foundation for subsequent feature extraction and sequence modeling. To improve training stability, amplitude normalization or standardization of the input is required; alternatively, moving average background modeling, high-pass filtering, or robust statistical background estimation can be used to replace background subtraction. The above preprocessing methods can be used individually or in combination without changing the core idea of ​​"constructing a learnable complex echo input representation and suppressing static clutter".

[0053] Further, step 3 includes:

[0054] To adapt to the "distance dimension-virtual channel dimension" coupled structure of through-wall echoes, the end-to-end attitude estimation network first extracts structural alignment features from the input tensor. For example... Figure 2 As shown, the structure-aligned feature extraction uses 1×1 convolutions to fuse channel information. The channel dimension can be defined by the user. After multiple experiments, dimensionality reduction to 64 yielded the best results. This step aims to complete the initial feature transformation while maintaining information integrity. A dual-domain recalibration mechanism can be introduced: one branch aggregates along the channel dimension to obtain distance descriptors and generates distance weights to highlight pose-sensitive distance intervals; the other branch aggregates along the distance dimension to obtain channel descriptors and generates channel weights to highlight effective channel combinations. Finally, the recalibration results of the two branches are fused and normalized to enhance the output features. This structure enables the network to strengthen key distance intervals and key channel combinations early in feature extraction, providing more discriminative input representations for subsequent sequence encoding. The process formula is expressed as:

[0055] ,

[0056] ,

[0057] ,

[0058] in, Indicates aggregation along the channel dimension. Indicates aggregation along the distance dimension; Distance weights Channel weight; The learnable fusion coefficient; This represents the first layer of 1×1 convolution. Represents a one-dimensional convolution mapping. and Represents a non-linear activation function. For normalization operators; To enable element-wise multiplication and support broadcasting, the distance weight branch can employ local one-dimensional convolution to model the correlation between neighboring distance intervals, and the channel weight branch can employ local cross-channel interaction to reduce additional parameters; all of the above implementations can be replaced without changing the overall structure of the two-domain recalibration.

[0059] After extracting structural alignment features, to reduce the computational and storage overhead of subsequent sequence modeling, the distance dimension of the enhanced features is downsampled to a fixed length L, resulting in a fixed-length distance token sequence. Subsequently, the tensor is transposed to the master-order form of the tokens, and each token is mapped to the latent space dimension D through linear projection, yielding a position-aware sequence representation. To preserve the positional information of the distance tokens, fixed sine / cosine positional codes or learnable positional codes can be superimposed on the sequence. This step compresses the long distance sequence into a token sequence of controllable length while preserving local distance structure information as much as possible, providing input conditions for subsequent efficient sequence encoding. The process can be expressed as:

[0060] ,

[0061] ,

[0062] ,

[0063] in, It can be either adaptive average pooling or adaptive max pooling; To enhance the feature tensor; This represents the number of channels after embedding. This is the fixed sequence length after downsampling; The feature tensor after downsampling; for Transpose of; For linear projection mapping; This is the projected sequence representation; For implicit space dimension; This is the position encoding matrix; The location index of the token; Encode the dimension index for the location; symbol " The "" indicates that position encoding is added to the sequence representation. Furthermore, downsampling can also use learnable stride convolution or segmented pooling; position encoding can also use learnable position vectors or relative position encoding; the above substitutions do not change the core idea of ​​"fixed-length tokenization + position-aware sequence construction".

[0064] After obtaining the location-aware distance token sequence, a Mamba sequence encoder based on a selective state-space model is used to efficiently model long-range dependencies in the token sequence. The Mamba encoder, based on state-space recursion, organizes the sequence modeling process into a computational form that expands nearly linearly with the sequence length. Through a selective mechanism, the model parameters can vary with the input, thereby achieving selective propagation and forgetting of information. In engineering implementation, a hardware-oriented parallel recursion strategy can be adopted to improve the efficiency of long sequence inference. Furthermore, considering that the discrimination clues for certain distance intervals in through-wall echoes may simultaneously depend on their preceding and following distance contexts, a bidirectional structure can be selected: the forward branch encodes in the original distance order, and the reverse branch reverses the distance token sequence along its length dimension before encoding, and then reverses it back to the original order. Finally, the outputs of the two branches are learnedably fused and normalized to obtain the encoded sequence features. The process formula is expressed as:

[0065] ,

[0066] ,

[0067] in, Given the input sequence, The length of the token. For implicit space dimension; For the reason In this embodiment, a sequence encoding operator is formed by stacking Mamba encoded blocks. Choosing 3, the Mamba coding block implements long sequence modeling based on the selective state-space model. This is a positive branch output. Output as a reverse branch; This represents a reversal operation along the length dimension of the token. Indicates a reverse operation; The learnable fusion coefficient; For normalization operators; This represents the final sequence encoding features after fusion. Furthermore, if the deployment scenario requires extreme lightweighting, only the positive branches can be retained to form a unidirectional structure; or the fusion method of bidirectional branches can be replaced with linear mapping after splicing, gated fusion, etc.; the above substitutions do not change the core idea of ​​"efficient sequence encoding and bidirectional context modeling based on Mamba".

[0068] Furthermore, step 4 includes the following:

[0069] like Figure 3As shown, the architecture diagram of the Mamba+GCN-based pose regression network is presented. The sequence features output from step 3 are globally aggregated to form the global representation required for pose regression. Mean pooling or weighted pooling of the token dimension can be used. Subsequently, the pose regression head outputs the coordinates of the three-dimensional key points of the human body. To improve the consistency of the skeleton structure and the stability of the key points, a skeleton topology refinement module can be optionally set: treating joints as nodes and bone connections as edges, graph convolution propagation and residual refinement are performed on a fixed human skeleton topology, outputting the refined key point coordinates. This module introduces structural priors for pose regression, making the output results more consistent in geometric structure. The process formula is expressed as:

[0070] ,

[0071] ,

[0072] ,

[0073] ,

[0074] in, Indicates sequence features The global representation obtained after aggregation along the token dimension. This indicates a token-based pooling operator, which can use either mean pooling or weighted pooling. The feature dimension representing the global representation; This indicates a key point reverting to the beginning. This represents the predicted coordinates of the three-dimensional key points of the human body. Indicates the number of joints, therefore ; This represents an adjacency matrix with self-loops. This represents the corresponding degree matrix. Indicates the first Layered graph convolution input features, Indicates the first Learnable parameters of layer graph convolution. Represents a non-linear activation function; This represents the keypoint correction amount output by the skeleton topology refinement module. This represents the graph convolution thinning operator. This means that the correction is superimposed on the initial predicted keypoints to obtain the refined pose result. Furthermore, the skeleton topology refinement module can also be implemented using a renormalized graph convolution, or a learnable adjacency matrix can be used to enhance expressive power; geometric constraint losses such as bone length consistency and joint angle range can also be added to the regression head as alternatives or supplements.

[0075] During model training, the pose supervision information constructed in step 1 is used to constrain the output keypoints. The pose supervision information can be 3D keypoint coordinates, 2D / 3D keypoint heatmaps, or a combination thereof. The loss function can include keypoint regression loss and structural consistency regularization. When using a skeleton topology refinement module, supervision can be applied to both the regression output and the refinement output simultaneously to achieve stable convergence. After training, the trained network parameters are fixed. During the inference phase, for any input through-wall radar complex echo, steps 2 to 4 are executed sequentially to output the 3D pose keypoints of the human body. Cross-modal supervision, as an optional training method, can be used to provide stable pose constraints during the training phase. After training, only the radar input is retained during the inference phase to output the pose results. The loss function design formula is expressed as:

[0076] ,

[0077]

[0078] ,

[0079] in, Indicates the coordinates of key monitoring points. Indicates the first Predicted coordinates of each joint, Indicates the first The actual coordinates of each joint Indicates the total number of joints. express Norm, therefore This represents the keypoint regression loss obtained by averaging the prediction errors of all joints. Represents the total loss function. and These are weighting coefficients, used to balance the attitude regression loss. With regularization loss Contributions; Represents a structural regularization term. This can be a bone length consistency constraint term, used to maintain reasonable connection length relationships in the predicted skeleton. This can be a time-smoothing constraint term used to constrain the continuity of attitude changes between adjacent frames. This time-smoothing constraint term is enabled when multiple frames are input. Furthermore, if the supervision signal is in the form of a heatmap, the regression loss can be replaced with a distribution alignment loss or KL divergence; consistency constraints can also be added to intermediate features to enhance cross-modal alignment.

[0080] According to one embodiment of the present invention, to verify the effectiveness of the invention, an experiment was conducted on autonomously acquired through-wall radar attitude data. In the experiment, the through-wall radar acquired complex echoes and constructed training supervision information. The network adopted an end-to-end framework of "radar structure alignment embedding + dual-domain recalibration + range tokenization + Mamba sequence encoding + keypoint regression / refinement." During the inference phase, only the radar echoes were input to output the key points of the human body's 3D pose. Examples of the main parameters of the model network structure and training configuration are shown in Table 1.

[0081] Table 1

[0082]

[0083] To quantitatively evaluate the attitude estimation accuracy, the Mean Per Joint Position Error (MPJPE) is used as the primary evaluation metric, and the proportion of correct keypoints at a 50mm threshold (PCK@50mm) is used as an auxiliary metric, defined as follows:

[0084] ,

[0085] ,

[0086] in, Indicates the number of test samples (or frames). Indicates the number of joints. and They represent the first The first sample Predicted and supervised 3D coordinates of each joint. This is an indicator function.

[0087] For the collected through-wall radar attitude data, the model is trained on the training set and evaluated on the test set. The method of this invention achieves MPJPE=38.62mm and PCK@50mm=78.66% on the test set. Regarding model size and computational overhead, the total number of parameters in the example network of this invention is approximately 3.55M, and the estimated computation cost per forward inference is approximately 9-13 GFLOPs (this is related to implementation optimization; latency / throughput statistics can be supplemented based on the actual deployment platform).

[0088] For ease of comparison, representative methods related to "RF / radar 3D human pose estimation (including conditions such as occlusion / non-line-of-sight / wall separation)" from several publicly available documents were selected, and their accuracy and complexity indices in the publicly reported data are summarized in Table 2. It should be noted that different methods may use different signal systems, datasets, and error definitions; the values ​​in Table 2 are recorded according to the original definitions in the publicly available documents and are for comparative reference only.

[0089] Table 2

[0090]

[0091] This scheme is based on UWB MIMO radar. First, a 3D radar image is constructed and converted into discrete point data. Then, a lightweight network completes attitude coordinate regression. Related error and complexity statistics are provided in publicly available reports. The MPJPE / PA-MPJPE, parameter count, and GFLOPs of mmDiff and P4Transformer are all taken from the same publicly available comparative experimental results table and model efficiency analysis table, facilitating order-of-magnitude comparisons from a consistent source. Unlike the methods mentioned above, this invention uses the complex echo of through-wall radar (or a lightweight pre-processed tensor) as direct input. Within the network, radar structure alignment embedding, dual-domain recalibration, range tokenization, and Mamba sequence encoding based on a selective state-space model are completed. Optional bidirectional range axis inversion branches and skeleton topology refinement modules can be introduced, thus forming a computational caliber and module closed loop that more closely resembles end-to-end deployment. It should be noted that the number of parameters in this invention is slightly higher than that of UWB-Pose, mainly due to the integrated capabilities of "structural alignment + sequence modeling + (optional) refinement": this integration can make fuller use of the coupled structure of "virtual channel × distance × time / frame" and enhance robustness under complex wall / multipath conditions; at the same time, the overall number of parameters in this invention remains in the millions, and allows for on-demand trade-offs between accuracy and resource overhead by disabling optional branches. Especially when the distance resolution is improved or the sequence length increases due to the introduction of multiple frames of input, the Mamba structure has the modeling characteristic of linearly scaling with the sequence length, which can maintain more controllable computational and storage overhead under long sequence conditions.

[0092] Figure 7 The images, displayed from top to bottom, show the visualizations of three different poses on the test set: a frontal view with the right hand extended horizontally while holding an object, a sideways view with the right hand extended horizontally while holding an object, and a frontal view with both hands holding an object. The Predicted Skeleton on the left represents the model output, and the GT Skeleton on the right represents the ground truth label. This invention fully learns from the original radar echo data and can output relatively good 3D human pose estimation points.

[0093] On the other hand, this invention provides a high-efficiency through-wall human pose estimation device based on a Mamba network, which includes modules capable of implementing the steps of the aforementioned method, specifically including:

[0094] The acquisition module is used to establish a method for acquiring and supervising data from through-wall radar. It acquires multi-channel complex echo data of the human body behind the wall through through-wall radar and simultaneously obtains attitude supervision information corresponding to the echo data.

[0095] The preprocessing module is used to preprocess the complex echo data to obtain the network input tensor;

[0096] The processing module is used to construct an end-to-end pose estimation network, which includes a structure alignment embedding module, a sequence encoding module, and a pose regression module. The structure alignment embedding module performs channel information fusion and dual-domain recalibration on the network input tensor to obtain enhanced features. The enhanced features are downsampled along the distance dimension into a fixed-length distance token sequence, and after adding position encoding, they are input into the sequence encoding module. The distance token sequence is then modeled for long-range dependencies using a Mamba encoder based on a selective state-space model to obtain the encoded sequence features.

[0097] The output module is used to globally aggregate the encoded sequence features and output the coordinates of the three-dimensional key points of the human body through the pose regression module.

[0098] Thirdly, the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned efficient through-wall human pose estimation method based on Mamba network.

[0099] Fourthly, the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enable the processor to implement the aforementioned efficient wall-penetrating human pose estimation method based on a Mamba network.

[0100] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An efficient method for through-wall human pose estimation based on Mamba networks, characterized in that, The method includes: Step 1: Establish a method for constructing data acquisition and supervision information from through-wall radar. Collect multi-channel complex echo data of the human body behind the wall using through-wall radar, and simultaneously acquire attitude supervision information corresponding to the echo data. Step 2: Preprocess the complex echo data to obtain the network input tensor; Step 3: Construct an end-to-end pose estimation network, which includes a structure alignment embedding module, a sequence encoding module, and a pose regression module. The structure alignment embedding module performs channel information fusion and dual-domain recalibration on the network input tensor to obtain enhanced features. The enhanced features are downsampled along the distance dimension into a fixed-length distance token sequence, and after adding position encoding, they are input into the sequence encoding module. The distance token sequence is modeled for long-range dependencies using a Mamba encoder based on a selective state-space model to obtain the encoded sequence features. Step 4: Globally aggregate the encoded sequence features and output the coordinates of the three-dimensional key points of the human body through the pose regression module.

2. The efficient through-wall human pose estimation method based on Mamba network according to claim 1, characterized in that, Step 1 includes: the radar uses a MIMO system to form a virtual channel array to obtain spatial diversity, and collects continuous multi-frame sequences to cover the echo changes of the human body under different postures; it simultaneously collects external sensors or external modal sequences, and generates human body key point coordinates or key point heat maps as training supervision signals through a preset posture estimation model.

3. The efficient through-wall human pose estimation method based on Mamba network according to claim 1, characterized in that, Step 2 includes: splitting the complex echo data into real and imaginary parts; cropping the distance dimension to focus on the distance range related to human activity; suppressing static clutter by background subtraction, moving average, or high-pass filtering; and splicing the processed real and imaginary parts along the channel dimension to form the network input tensor, and performing amplitude normalization or standardization.

4. The efficient through-wall human pose estimation method based on Mamba network according to claim 1, characterized in that, In step 3, the network input tensor is fused with channel information and recalibrated in a dual-domain manner through the structure alignment embedding module to obtain enhanced features. This includes: using 1×1 convolution to fuse channel information and reduce the channel dimension to a preset dimension; the dual-domain recalibration includes: the first branch aggregates along the channel dimension to obtain a distance descriptor, which is then converted into distance weights after local one-dimensional convolution; the second branch aggregates along the distance dimension to obtain a channel descriptor, which is then converted into channel weights after local cross-channel interaction; the recalibration results of the two branches are then fused in a learnable manner and normalized at each layer to output enhanced features.

5. The efficient through-wall human pose estimation method based on Mamba network according to claim 1, characterized in that, In step 3, the enhanced features are downsampled along the distance dimension into a fixed-length distance token sequence, and then input into the sequence encoding module after adding position encoding. This includes: compressing the distance dimension to a fixed length L using adaptive average pooling or adaptive max pooling to obtain the distance token sequence; mapping each token to the latent space dimension D through linear projection; and superimposing sine / cosine position encoding or learnable position encoding on the distance token sequence to obtain a position-aware sequence representation.

6. The efficient through-wall human pose estimation method based on Mamba network according to claim 1, characterized in that, In step 3, the distance token sequence is modeled for long-range dependency using a Mamba encoder based on a selective state-space model to obtain encoded sequence features. This includes: adopting a bidirectional structure, with the forward branch encoding in the original distance order and the reverse branch reversing the distance token sequence along the length dimension before encoding and then reversing it back to the original order; and performing learnable fusion and layer normalization on the outputs of the forward and reverse branches to obtain encoded sequence features.

7. The efficient through-wall human pose estimation method based on Mamba network according to claim 1, characterized in that, Step 4 includes: using mean pooling or weighted pooling of the token dimension to form a global representation; the pose regression module includes a pose regression head and a skeleton topology refinement module. First, the initial three-dimensional key point coordinates are output through a fully connected layer. Then, the human skeleton topology is constructed with joints as nodes and bone connections as edges. The topology is propagated and refined through a graph convolutional network, and the refined key point coordinates are output.

8. A high-efficiency through-wall human pose estimation device based on Mamba network, characterized in that, include: The acquisition module is used to establish a method for acquiring and supervising data from through-wall radar. It acquires multi-channel complex echo data of the human body behind the wall through through-wall radar and simultaneously obtains attitude supervision information corresponding to the echo data. The preprocessing module is used to preprocess the complex echo data to obtain the network input tensor; The processing module is used to construct an end-to-end pose estimation network, which includes a structure alignment embedding module, a sequence encoding module, and a pose regression module; the structure alignment embedding module performs channel information fusion and dual-domain recalibration on the network input tensor to obtain enhanced features; The enhanced features are downsampled along the distance dimension into a fixed-length distance token sequence, and after adding position encoding, they are input into the sequence encoding module. The distance token sequence is then modeled for long-range dependencies using a Mamba encoder based on a selective state space model to obtain the encoded sequence features. The output module is used to globally aggregate the encoded sequence features and output the coordinates of the three-dimensional key points of the human body through the pose regression module.

9. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When one or more programs are executed by the one or more processors, the one or more processors implement the efficient through-wall human pose estimation method based on Mamba network as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores executable instructions that, when executed by a processor, enable the processor to implement the efficient through-wall human pose estimation method based on a Mamba network as described in any one of claims 1-7.

Images