Explicit denoising-driven space-time fusion 3D human body posture estimation method and medium

Abstract

The invention discloses an explicit denoising-driven space-time fusion 3D human body posture estimation method and a medium. The method comprises the following steps: firstly, constructing a network model containing an explicit denoising attention module, a space-time fusion module and a regression layer; the explicit denoising attention module performs parallel modeling through signal and noise double branches, feature denoising is realized through feature alignment, dynamic gating and explicit difference, and the space-time fusion module performs decoupling modeling through'space-time 'and'time-space' double paths and performs adaptive feature fusion. During training, performing end-to-end training on the model by using a human body posture data set containing different figures, postures and random shielding; in the testing process, a to-be-estimated human body video is preprocessed and then input into the trained model, explicit denoising and space-time fusion processing are conducted, and a 3D human body posture result is output through the regression layer. According to the method, long-sequence noise and shielding interference are effectively suppressed, single-path modeling error accumulation is relieved, and the precision and robustness of attitude estimation in a complex scene are improved.

Description

Technical Field

[0001] This invention relates to the field of 3D human pose estimation technology, specifically to an explicit denoising-driven spatiotemporal fusion 3D human pose estimation method. This method uses monocular RGB video as input and achieves high-precision recovery of the 3D spatial coordinates of human key points through spatiotemporal feature joint modeling and noise suppression mechanisms. Background Technology

[0002] Monocular 3D human pose estimation aims to estimate the 3D coordinates of human joints from images, typically with reference to the root joint or pelvis. It is a fundamental computer vision task with a wide range of applications, including human motion prediction, human action recognition, and human-centered generation. High-precision 3D pose estimation requires the system to accurately locate each joint and maintain spatiotemporal consistency in pose estimation, which remains a significant challenge in complex action scenarios.

[0003] Recovering 3D human pose from monocular RGB video is a classic research problem. Existing methods can be divided into two categories: The first category is end-to-end regression methods. Early regression-based CNN methods directly mapped images to 3D joint coordinates, enabling end-to-end prediction of human pose. Although this type of method made some progress in early research, its generalization ability in complex action scenarios and its adaptability to different pose changes are still limited. The second category is two-stage methods. This method first uses a mature 2D human pose estimation model to detect the positions of 2D joints in the image, and then further regresses the 3D human pose based on the obtained 2D joint sequence. Compared with end-to-end methods, two-stage methods have significant advantages: the 2D pose estimation model can usually be trained on a larger and more diverse 2D human pose dataset, thereby obtaining stronger visual representation capabilities and better generalization performance. In addition, through multi-view data acquisition, a large number of 2D-3D pose pairs can be constructed for supervised training, thereby further improving the accuracy of 3D pose regression. Currently, two-stage methods typically employ network architectures such as multilayer perceptron (MLP), temporal convolutional network (TCN), graph convolutional network (GCN), or Transformer to model 2D keypoint sequences in order to achieve 3D pose prediction.

[0004] In recent years, the Transformer network architecture has been widely used in natural language processing and computer vision, and has gradually been introduced into 3D human pose estimation tasks. For example, PoseFormer was the first to introduce the Transformer architecture in this field. By treating each joint as a token, it globally models the joint information in the complete time series, realizing the transformation from 2D joint point sequences to 3D human poses, significantly improving the estimation accuracy compared to traditional convolutional neural network-based methods. Subsequently, MixSTE further pointed out that different human joints have significant differences in motion patterns and temporal changes, and traditional methods are unable to effectively capture the dynamic correspondence between joints between frames, resulting in insufficient modeling of spatiotemporal dependencies.

[0005] Despite the significant progress made by the aforementioned methods in 3D human pose estimation tasks, existing technologies still face several key technical challenges in high-precision and high-robust applications, mainly in the following aspects: 1. Noise accumulation in long sequence data leads to decreased estimation accuracy: When processing long video sequences, a large amount of redundant or unstable inter-frame information can interfere with joint position prediction, causing the mean joint position error (MPJPE) to gradually increase. Existing methods (such as PoseFormer) can model long temporal dependencies, but they lack explicit suppression mechanisms for long sequence noise.

[0006] 2. Insufficient robustness in occlusion scenarios: When some joints of the human body are occluded or key point detection is incomplete, the two-stage method is prone to large errors in the 3D pose regression process. Existing methods lack adaptive compensation or optimization mechanisms for occlusion situations.

[0007] 3. Single-path spatiotemporal modeling structures suffer from error propagation problems: Most existing methods typically employ a single-path sequential modeling approach, either "space-time" or "time-space". In this structure, errors generated in the previous stage may continue to propagate and be amplified in subsequent modeling processes. Furthermore, single-path structures lack effective reverse correction mechanisms, making it difficult to achieve cross-dimensional information complementarity and error correction.

[0008] In summary, existing technologies still have significant shortcomings in areas such as long-sequence noise suppression, robustness to occluded scenes, and cross-dimensional error propagation control. Therefore, there is an urgent need to propose a high-precision 3D human pose estimation method that can simultaneously enhance spatiotemporal dependency modeling capabilities and explicitly suppress noise interference, in order to improve the accuracy and stability of human pose estimation in complex scenes. Summary of the Invention

[0009] To address the aforementioned issues, this invention provides an explicit denoising-driven spatiotemporal fusion 3D human pose estimation method, which can effectively improve the accuracy of 3D joint prediction and the ability to model spatiotemporal features.

[0010] The method of the present invention includes the following steps.

[0011] S1: Construct a high-precision human pose estimation network model based on spatiotemporal dimension explicit denoising attention, the model including explicit denoising attention module, spatiotemporal fusion module and regression layer; Explicit Denoising Attention Module: This module performs linear projection on the preprocessed 2D joint coordinates and adds positional and group coding to preserve spatiotemporal information and guide the attention mechanism to focus on anatomically relevant joints. It also extracts features from the processed 2D joint feature vectors to generate a signal-to-noise aware attention-weighted feature representation. Preprocessing includes video slicing and extracting joint coordinates for each frame using a 2D pose estimation network.

[0012] Spatiotemporal fusion module: Dual-path modeling is achieved by combining explicit denoising attention modules in different orders. One path follows a "space-time" order, while the other follows a "time-space" order. Finally, the features of the two paths are weighted and fused to achieve full integration of spatiotemporal information.

[0013] Regression layer: Receives the fused features and regresses the final pose data of 3D joint coordinates through a multilayer perceptron (MLP) to predict human pose.

[0014] S2. During the training phase, the network model is trained end-to-end using a human pose dataset containing different people and poses with random occlusion, and the model parameters are optimized to improve robustness to long-sequence noise and occlusion scenes. S3. In the testing phase, the human video to be estimated is input into the trained network model, which then passes through the preprocessing, explicit denoising attention module, and spatiotemporal fusion module in sequence. Finally, the predicted 3D human pose is output through the regression layer.

[0015] Furthermore, the explicit denoising attention module is used to receive a 2D joint feature vector containing learnable positional encoding and human anatomy grouping encoding, and generate structural semantic representation and unstructured residual representation through parallel signal self-attention branch and constrained noise self-attention branch, respectively. After completing the feature subspace alignment, a learnable scalar is introduced. The noise component is dynamically gated, and the signal branch output is corrected by explicit differential method, so as to remove redundant information at the feature level and explicitly suppress occlusion noise and inter-frame jitter in long sequence data.

[0016] Furthermore, in step S1, the core idea of ​​the dual-path spatiotemporal fusion module is as follows: the 2D coordinate feature vectors of different joints in each frame are regarded as spatial dimension features, and the changes in the 2D coordinate feature vectors of the same joint in the video sequence are regarded as temporal dimension features. After applying "temporal explicit denoising attention module + spatial explicit denoising attention module" and "spatial explicit denoising attention module + temporal explicit denoising attention module" respectively, the feature representations obtained from the two paths are fused to form a feature expression with fully coupled spatiotemporal information.

[0017] Furthermore, during the training phase, the image encoding module performs the following steps: (1) The input video data is sliced ​​and 2D human pose estimation is performed on each frame to obtain the corresponding 2D joint feature vectors. This operation can effectively reduce the dimensionality of the input data and reduce the information content (W) of a single frame image. H 3) Reduced to "number of key points" "Feature dimension" retains key information related to human posture.

[0018] (2) The obtained 2D joint feature vectors are mapped to a high-dimensional feature space through linear projection so as to convert the low-dimensional information into a feature representation that is more suitable for subsequent modeling, thereby enhancing the expressive power of the network.

[0019] (3) Add positional encoding and group encoding to each 2D joint feature vector. By fusing positional encoding, group encoding and projection features, the spatial positional information, temporal positional information and related information of other joints are preserved. Positional encoding enables the model to understand the relative position and structure of each part in the image, thereby improving the ability to perceive the spatial relationship of human posture.

[0020] Furthermore, the explicit denoising attention module described in step S1 achieves signal-to-noise decoupling of human motion features through parallel collaborative modeling of the signal branch and the noise branch. The specific steps include: Signal self-attention modeling: a multi-head self-attention mechanism is used to construct signal branches and capture stable semantic structure information reflecting the laws of human movement in parallel through multiple independent subspaces, including the spatial association of the topological structure of the human skeleton and the coherent motion trajectory in long sequences. Constrained noise self-attention modeling: Construct a noise branch parallel to the signal branch and introduce a dual constraint mechanism to limit its modeling capacity so that it can only capture unstructured residual components. Feature Space Alignment and Dynamic Gating: A learnable alignment matrix is ​​introduced to map the features of the noisy branch to the attention subspaces of the signal branch, and a learnable scalar is introduced. As a dynamic gating coefficient, the noise reduction intensity is adaptively adjusted according to the sensitivity of different feature dimensions; Explicit differential correction: Residual correction is performed by the signal branch output and the noise component after dynamic gating. Redundant information is explicitly stripped at the feature level, thereby obtaining a high signal-to-noise ratio feature representation that conforms to human physiological constraints and suppresses inter-frame jitter and occlusion noise.

[0021] Furthermore, the steps of the dual-path spatiotemporal fusion module described in step S1 are as follows: Spatial-temporal modeling path: Following the processing logic of "space first, time later", firstly, the explicit denoising attention unit in the spatial dimension is used to capture the topological relationship between joints in a single frame and extract spatial enhancement features that meet human anatomical constraints; then, the cross-frame feature sequences of each joint are modeled to capture long-term temporal motion dependence and suppress trajectory jitter, generating spatial-temporal feature representations. Temporal-spatial modeling path: Following the reverse logic of "time first, space later", the joint motion trajectory is first smoothed by using explicit denoising attention units in the temporal dimension, and then the joint structure of a single frame is refined in the spatial dimension to generate a temporal-spatial feature representation that complements the first path. Adaptive spatiotemporal feature fusion: Based on the sensitivity of the attention mechanism to the processing order, a learnable balance factor is used to dynamically aggregate the complementary features generated by the two paths. By integrating heterogeneous spatiotemporal information, the problem of error accumulation and propagation in single-path sequential modeling is alleviated, and a robust final spatiotemporal feature representation is obtained.

[0022] Furthermore, step S3 includes the following steps: The system takes human video data to be detected as input. Before inputting into the explicit denoising attention module, the video is sliced ​​and 2D pose estimation is performed to obtain the corresponding 2D joint position vectors. Then, the 2D joint position vectors are projected onto a high-dimensional feature space through a linear layer, and position encoding and group encoding are added to each joint feature. The 2D joint feature vectors with position encoding and group encoding are input into the explicit denoising attention module and the spatiotemporal fusion module to extract high-quality spatiotemporal feature representations related to human pose. Finally, the feature representations output by the spatiotemporal fusion module are input into the regression layer to obtain the predicted 3D human pose estimation results.

[0023] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0024] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0025] In summary, this invention presents an explicit denoising-driven spatiotemporal fusion 3D human pose estimation method. This method first preprocesses the input video to obtain a series of frames, then uses a 2D pose estimation model to obtain 2D joint coordinates, which are then mapped to a high-dimensional space to obtain high-dimensional features. Subsequently, learnable spatiotemporal position codes and human anatomical group codes are superimposed to enhance the model's semantic perception of the human skeleton's topological structure. The core module consists of an explicit denoising attention module and a spatiotemporal fusion module: the explicit denoising attention module captures stable physiological constraint semantics through the signal branch, captures unstructured residuals through a dual constraint mechanism in the noise branch, and uses an explicit signal-to-noise differential strategy to remove redundant information, effectively suppressing inter-frame jitter and occlusion noise in long sequences. The spatiotemporal fusion module adopts a dual-path decoupled modeling architecture of "space-time" and "time-space," fully capturing the asymmetric dependencies of human motion in spatial topology and temporal evolution through an adaptive fusion mechanism. Finally, the regression layer uses high-quality spatiotemporal features to map high-precision 3D joint coordinates. The method of this invention is simple and efficient, and significantly improves the accuracy and robustness of pose estimation in occluded scenes and long video sequences.

[0026] Specifically, the beneficial effects of this invention are as follows: The method of this invention is simple and efficient. Compared with models that directly use image data, it reduces the amount of input and improves computational efficiency; through an explicit denoising attention mechanism, it effectively suppresses long sequence noise and significantly improves the accuracy of 3D position prediction of human joints; through a group coding mechanism and an occlusion data enhancement training strategy, it enhances the model's structural perception ability and robustness in scenarios where human joints are occluded or missing detections; through a dual-path spatiotemporal fusion structure, it achieves full interaction and fusion of spatial and temporal information, effectively mitigating the error accumulation problem that may occur in the single-path modeling process, thereby further improving the stability and accuracy of human pose estimation. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating the method of the present invention; Figure 2 This is a schematic diagram of the structure of the explicit denoising-driven spatiotemporal fusion high-precision human pose estimation network model constructed in this invention; Figure 3 It is a group coding method. Detailed Implementation

[0028] The technical solution of the present invention will be described in detail below through embodiments, but the scope of protection of the present invention is not limited to the embodiments described.

[0029] like Figure 1 As shown, this invention provides an explicit denoising-driven spatiotemporal fusion 3D human pose estimation method. Based on the traditional two-stage estimation process, this method significantly improves pose estimation accuracy in complex occlusion and long sequence scenes by introducing an explicit denoising attention mechanism and dual-path spatiotemporal decoupling modeling. Specifically, it includes the following steps: S1. Construct a spatiotemporal fusion high-precision human pose estimation network model driven by explicit denoising; S2. During the training phase, the network model is trained end-to-end using a human pose dataset containing different people and poses with random occlusion, and the model parameters are optimized to improve robustness to long-sequence noise and occlusion scenes. S3. In the testing phase, the human video to be estimated is input into the trained network model, which then passes through the preprocessing, explicit denoising attention module, and spatiotemporal fusion module in sequence. Finally, the predicted 3D human pose is output through the regression layer.

[0030] The above steps can be specifically explained as follows: A 3D human pose estimation network is constructed, which includes an explicit denoising attention module, a dual-path spatiotemporal fusion module, and a regression layer. First, the input video sequence is processed by time slicing, and the human joint coordinates of each frame are extracted using a 2D pose estimation method to form a 2D joint sequence. Then, the 2D joint sequence is mapped to a high-dimensional feature space through linear projection, and learnable spatiotemporal position codes and human anatomy group codes are superimposed. The encoded high-dimensional features are input into the explicit denoising attention module, and modeling is performed in parallel through the signal branch and the noise branch. The signal branch is based on a multi-head self-attention mechanism to capture stable semantic structure features of human posture, while the noise branch models and suppresses unstructured residuals by introducing a dual constraint mechanism. Subsequently, the features of the two branches are fused and optimized by combining feature alignment strategy, dynamic gating mechanism and explicit differential correction operation to obtain a denoised feature representation with high signal-to-noise ratio. For the denoised features, bidirectional spatiotemporal modeling is performed along two paths: "space → time" and "time → space". The complementary integration of features from the two paths is achieved through an adaptive weight fusion mechanism. Finally, the corresponding 3D human joint coordinates are output through regression layer mapping to realize human posture reconstruction.

[0031] The explicit denoising-driven spatiotemporal fusion high-precision human pose estimation network model constructed in this invention, such as... Figure 2 As shown, it includes an explicit denoising attention module, a spatiotemporal fusion module, and a regression layer.

[0032] In this invention, when training the network model, the loss function used in S2 for training the network model is... Loss function. The operations of each module during training of the network model are as follows: 1) When training the network model, the preprocessing steps are as follows: S2.1.1 First, preprocess the input image data, including the input video. Sliced ​​into a set of frames in chronological order This is so that subsequent feature extraction can be performed, among which... , The height and width of the image; This represents the number of channels in the image (usually 3). This represents the number of video frames.

[0033] S2.1.2 Subsequently, 2D human pose estimation is performed on each of the above frames to obtain the corresponding 2D coordinate information. ,in The number of joints can be represented as:

[0034] in, This involves performing 2D pose estimation on the input image to obtain the 2D coordinate information of each joint in each frame. The number of joints is typically 17, and the dimension is typically 3 (the number of joints is 17). coordinate, (Coordinates and confidence values).

[0035] 2) During the training of the network model, the explicit denoising attention module performs the following steps: S2.2.1 The explicit denoising attention module proposed in this invention achieves feature decoupling and noise reduction processing of complex human motion sequences through collaborative modeling of signal and noise branches. The signal branch is mainly used to capture stable semantic structural information reflecting the laws of human motion, including the inherent spatial relationships formed by the topological structure of the human skeleton and the coherent motion trajectory in long-term sequences; while the noise branch is used to model random noise components introduced by observation errors, joint occlusion, or inter-frame redundancy, i.e., unstructured residual components lacking temporal continuity and unrelated to human anatomical structure. Before being input into this module, the given 2D skeleton sequence x first needs to be... Mapping to a higher-dimensional space, as shown in the following formula:

[0036] in, Indicates the projection layer. The feature dimension of each joint is To preserve the spatial structure information of the joints and the temporal order information in the video sequence, this invention introduces learnable spatial position coding into the feature representation. and time location encoding Furthermore, to enhance the model's semantic perception of the topological structure of the human skeleton, this invention adds group coding. Grouping information such as Figure 3 As shown. Figure 3 The numbers in the diagram are explained as follows: 0 represents the pelvis; 1 represents the right hip joint; 2 represents the right knee joint; 3 represents the right ankle joint; 4 represents the left hip joint; 5 represents the left knee joint; 6 represents the left ankle joint; 7 represents the mid-spine; 8 represents the rib cage; 9 represents the nose; 10 represents the center of the head; 11 represents the left shoulder joint; 12 represents the left elbow joint; 13 represents the left wrist joint; 14 represents the right shoulder joint; 15 represents the right elbow joint; and 16 represents the right wrist joint.

[0037] (1) Signal self-attention branch The signal self-attention branch employs a multi-head self-attention mechanism, extracting stable semantic structure patterns and their contextual dependencies from complex input sequences through parallel learning in multiple independent subspaces. For the first... One attention point ( , (It is the number of heads), , and The following formula is given ( (Indicates the signal self-attention branch)

[0038]

[0039]

[0040] in , , It is a learnable projection matrix, and , , , For feature dimension, It is the dimension of a single head.

[0041] Within each attention head, scaling click attention can be described as a mapping function that queries the matrix. Key matrix Sum matrix Mapping to the output attention matrix generates an attention weight map that reflects the inter-joint dependence intensity; this process is achieved through... The function normalizes the weights, ensuring that the model can accurately extract high signal-to-noise ratio semantic features from the global spatiotemporal sequence. The self-attention output of the multi-head signal branch is:

[0042] in, This is a scaling factor used to prevent gradient vanishing and maintain training stability. Through a multi-head mechanism, the model can jointly model the dependencies between different joints in multiple representation subspaces. In 3D human pose estimation tasks, there are significant spatial structural dependencies between joints. Therefore, in the spatial dimension, this mechanism can learn the correlations between different joints; while in the temporal dimension, it can learn which historical frames should be referenced in the current frame, thus capturing the temporal continuity features of human motion.

[0043] (2) Noise self-attention branch To guide the noisy branches to focus on capturing unstructured residual components rather than overfitting long-term semantic dependencies, this invention designs a collaborative strategy of dual constraint mechanism and single-head attention architecture: ① Dual constraint mechanism 1) Shallow network structure constraint: By limiting the number of very shallow layers, the model capacity is restricted, making it unable to model complex non-directly connected skeleton dependencies and long-term motion patterns in terms of structure, and can only capture local and instantaneous feature changes.

[0044] 2) DropPath: DropPath is a regularization technique commonly used in training deep networks. Its core operation is to randomly "turn off" the functional branches in a residual block during training, retaining only the identity mapping. By physically cutting off the branch connections, it creates "information discontinuity". In deep learning, the deeper the model and the more parameters it has, the easier it is to overfit. DropPath is designed to break the excessive dependence between network layers, forcing each layer or branch to still learn useful features even when some components are missing. Since structural signals depend on the propagation of complete paths, the random path dropping in DropPath fragments these signals in the time dimension, forcing the branch to be unable to capture long-term motion trajectories and instead only capture instantaneous, discontinuous noise that still exists even when the path is broken, thus disrupting temporal continuity. The layers turned off by DropPath do not work in the current forward propagation and do not update gradients during backward propagation. This reduces the risk of the model over-optimizing in a single direction, leading to optimization difficulties. For high-frequency noise that is inherently random, its gradient direction is relatively less affected by path dropping, which makes the branch naturally evolve into a filter focused on "unstructured residuals," strengthening its noise filtering properties.

[0045] ② Feature transformation and projection In the noise branch, the input features are first subjected to a perturbation transformation:

[0046] Among them, the output features of MLP and The feature dimensions are consistent. Subsequently, a learnable weight decay coefficient is introduced. To further prevent its long-term learning dependence, it exhibits the characteristics of a high-pass filter, in the noise branch. , and The following formula is given ( (Representing the noise self-attention branch)

[0047]

[0048]

[0049] in, , , , Weight decay forced The weights are kept extremely small, making it difficult to capture high-dimensional, complex nonlinear relationships. In this case, the noisy branch cannot learn complex poses and is forced to fit only local random noise. Furthermore, the noisy branch uses single-head attention to constrain its representational ability, preventing it from learning multiple stable semantic relationship patterns in parallel. Under the "unstable" mapping caused by DropPath, the attention map generated by this branch will focus more on outliers and unstructured noise components, and its attention output is:

[0050] in N It is the attention output feature of the noise branch, representing the feature representation that the model separates from the input skeleton sequence and contains only random noise components.

[0051] (3) Explicit noise reduction attention Based on the construction of the aforementioned signal self-attention branch and noise self-attention branch, this invention views the input feature representation as a superposition of structural semantic components and unstructured residual components. Since the signal branch, under a multi-head parallel mechanism, can stably model the anatomical constraints and long-term motion patterns between joints, while the noise branch, under a dual-constraint mechanism, can only capture local, discontinuous random perturbations, the two form a complementary decomposition relationship in the feature space. Building upon this, this invention introduces an explicit differential strategy to actively separate signal and noise at the feature level, thereby achieving structural suppression of random interference.

[0052] Since the signal branch uses a multi-head mechanism while the noise branch uses a single-head representation, noise features need to be mapped to each attention subspace to achieve residual matching across subspaces. For the , Each attention head is used to introduce a learnable alignment matrix:

[0053] in This mapping ensures that the noise components and their corresponding semantic subspaces maintain consistency in dimension and scale, achieving fine-grained matching. After obtaining the alignment noise term, the module introduces a learnable suppression coefficient. As a dynamic gating mechanism, the noise reduction intensity is automatically adjusted. This coefficient employs a non-shared weight allocation based on the feature sensitivity of different attention heads, thereby applying refined residual correction to the signal representation and obtaining the noise-reduced attention features.

[0054] in, The adaptive optimization enables the model to suppress occlusion noise and inter-frame jitter noise while preserving high-frequency motion details. Since the noise branch can only learn unstructured residuals under structural constraints, its output statistically approximates the high-frequency perturbation components in the input. Therefore, explicit differencing can be viewed as performing a structure-aware high-pass suppression process in the attention feature space, thereby improving robustness while preserving anatomical consistency. Through this explicit differencing computation, the module effectively reduces attention noise, a common phenomenon in long sequence processing, and improves the model's accuracy in modeling key joint relationships.

[0055] 3) During the training of the network model, the spatiotemporal fusion module performs the following steps: The spatiotemporal fusion module aims to fully capture the asymmetric dependencies of human pose in spatial topology and temporal evolution through a dual-path decoupled modeling architecture. This module receives high-dimensional feature tensors from the preprocessing layer. Two explicit denoising attention paths, "space-time" and "time-space", are constructed to achieve deep coupling and complementary expression of spatiotemporal features.

[0056] S3.1, Spatial-Temporal Explicit Noise Reduction Attention Path This approach follows the logic of "spatial modeling first, then temporal aggregation," aiming to first establish a stable single-frame anatomical structure representation, and then further model the motion law of joints changing over time.

[0057] (1) Frame-by-frame spatial representation enhancement For each frame in the sequence Extract the joint features of this frame. The input is fed into the spatial explicit denoising attention unit ( This unit extracts spatial topological features that satisfy human anatomical constraints by explicitly suppressing detection noise within a single frame, thus obtaining spatial enhancement features. The spatial augmentation results of all frames are stacked to obtain intermediate feature representations: .

[0058] (2) Joint-by-joint time evolution modeling To model the continuity of the motion trajectory of each joint in the time dimension, the feature tensor is... Perform dimensional permutation to obtain a feature matrix indexed by joints. For each joint Take the sequence of the joint in the time dimension. The input is fed into the temporally explicit denoising attention unit ( This allows for the modeling of long-term temporal motion dependencies and the suppression of trajectory jitter noise, thereby obtaining temporal augmentation features. After performing temporal modeling on all joints, the feature tensors are reduced in dimension to obtain space-time augmented features: .

[0059] S3.2, Temporal-Spatial Explicit Noise Reduction Attention Path This approach, through a reverse sequence of "time modeling first, space correction later," provides the model with contextual information that complements the space-time path, thereby enhancing the overall spatiotemporal modeling capability.

[0060] (1) Joint-by-joint trajectory noise reduction First, the input features are transposed along the time and joint dimensions to obtain the feature representation. For each joint Take the time series of this joint. And input to the temporally explicit denoising attention unit ( Thus, smoothed temporal features are obtained. Stack the temporal augmentation features of all joints and restore the original feature order to obtain the temporal augmentation feature representation: .

[0061] (2) Frame-by-frame spatial topology refinement For each frame Extract the joint features of this frame. And input to the spatial explicit denoising attention unit ( Further refinement of the spatial structure model yields spatial enhancement features. After stacking all the frame results, the temporal-spatial augmentation features are obtained: .

[0062] S3.3 Adaptive Spatiotemporal Feature Fusion Mechanism To integrate heterogeneous features captured by different modeling sequences, this module introduces a dynamic weight fusion strategy. This is achieved through a learnable balancing factor. Feature-level residual aggregation is performed on the outputs of the two paths to obtain the final robust spatiotemporal representation: .

[0063] in, These are weight parameters that are adaptively optimized during the training phase, used to balance the contributions of space-priority and time-priority features to the final 3D coordinate regression.

[0064] 4) When training the network model, the regression layer performs the following steps: The regression layer, as the output of the network, aims to map the deeply coupled features output by the spatiotemporal fusion module to 3D physical space. Its execution steps are as follows: The feature vectors capturing human anatomical structure and dynamic evolution are input into the regression layer, and the final 3D pose estimation result is reconstructed through nonlinear transformation. The mathematical expression is:

[0065] in ， MLP is a feedforward neural network composed of multiple layers of linear mappings and nonlinear activation functions. Its basic structure includes an input layer, one or more hidden layers, and an output layer, with fully connected layers to achieve feature transformation.

[0066] In the testing phase of S3, step S3 includes the following steps: inputting the human video to be detected; slicing the video and estimating 2D pose to obtain 2D joint coordinate information, and projecting the 2D joint coordinate vectors into a high-dimensional feature space through a linear layer; subsequently, adding position encoding to each 2D joint feature vector to preserve spatial position information, temporal position information, and grouping information; inputting the feature vectors with position and group encodings into the spatiotemporal fusion module to extract features related to human pose; and inputting the obtained high-quality human feature representation into the regression layer to obtain the predicted human pose estimation result.

[0067] Although the method of this invention has been simplified, it still maintains excellent performance in pose estimation accuracy. By introducing an effective feature extraction module and a reasonable training strategy, the method of this invention demonstrates high robustness under different backgrounds and pose changes. As can be seen from the comparison between the present invention and existing methods in Table 1, the present invention outperforms other methods in both MPJPE (Mean Joint Position Error) and P-MPJPE (Mean Joint Error after Procrustes Alignment).

[0068] Table 1 Comparison results between the present invention and existing methods

[0069] In summary, this invention achieves high-precision human pose estimation by designing an explicit denoising attention architecture in the spatiotemporal dimension. The method first constructs a spatiotemporally fused human pose estimation network, effectively removing irrelevant redundant information from long-term data using a signal-to-noise sensing mechanism, thus improving the feature selection efficiency of the attention mechanism. During the training phase, the network model is trained using human pose datasets with different poses and individuals exhibiting random occlusion. In the testing phase, high-quality 3D human pose estimation is achieved using the trained network model. Compared to traditional methods, this invention features a simpler network model design and significantly improves the accuracy of pose estimation.

[0070] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0071] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0072] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the mobile source emission prediction methods based on time-series feature migration described in the above embodiments.

[0073] It is understood that the systems, devices, and storage media provided in the embodiments of the present invention correspond to the methods provided in the embodiments of the present invention, and the explanations, examples, and beneficial effects of the relevant content can be referred to the corresponding parts of the above methods.

[0074] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0075] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0076] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0077] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A spatiotemporal fusion method for estimating human pose using explicit denoising-driven approaches, characterized in that, Includes the following steps: A 3D human pose estimation network is constructed, which includes an explicit denoising attention module, a dual-path spatiotemporal fusion module, and a regression layer. First, the input video sequence is processed by time slicing, and the human joint coordinates of each frame are extracted using a 2D pose estimation method to form a 2D joint sequence. Then, the 2D joint sequence is mapped to a high-dimensional feature space through linear projection, and learnable spatiotemporal position codes and human anatomy group codes are superimposed. The encoded high-dimensional features are input into the explicit denoising attention module, and modeling is performed in parallel through the signal branch and the noise branch. The signal branch is based on a multi-head self-attention mechanism to capture stable semantic structure features of human posture, while the noise branch models and suppresses unstructured residuals by introducing a dual constraint mechanism. Subsequently, the features of the two branches are fused and optimized by combining feature alignment strategy, dynamic gating mechanism and explicit differential correction operation to obtain a denoised feature representation with high signal-to-noise ratio. For the denoised features, bidirectional spatiotemporal modeling is performed along two paths, "space→time" and "time→space", respectively. The complementary integration of features from the two paths is achieved through an adaptive weight fusion mechanism. Finally, the corresponding 3D human joint coordinates are output through regression layer mapping to realize human pose reconstruction.

2. The explicit denoising-driven spatiotemporal fusion 3D human pose estimation method according to claim 1, characterized in that, The explicit denoising attention module is used to receive a 2D joint feature vector containing learnable positional codes and human anatomy group codes, and generate structural semantic representations and unstructured residual representations through parallel signal self-attention branches and constrained noise self-attention branches, respectively. After completing the feature subspace alignment, a learnable scalar is introduced. The noise component is dynamically gated, and the signal branch output is corrected by explicit differential method, so as to remove redundant information at the feature level and explicitly suppress occlusion noise and inter-frame jitter in long sequence data.

3. The explicit denoising-driven spatiotemporal fusion 3D human pose estimation method according to claim 1, characterized in that, The dual-path spatiotemporal fusion module uses a dual-path decoupling modeling architecture to extract spatiotemporal dependencies of denoised features. The first path follows the order of "space first, then time", and models the joint topological association within a single frame and the temporal dynamic relationship across frames by passing explicit denoising attention units in the spatial and temporal dimensions in turn. The second path follows the "time first, space second" order, and forms complementary feature representations by inversely modeling long-term temporal motion dependencies and joint structure constraints; Adaptive weight fusion of the two path outputs is performed to integrate heterogeneous spatiotemporal information and alleviate the error accumulation problem of single-path modeling.

4. The explicit denoising-driven spatiotemporal fusion 3D human pose estimation method according to claim 1, characterized in that, The regression layer is used to receive the final feature representation output by the spatiotemporal fusion module, and regress the final pose data in the form of 3D joint coordinates through the multilayer perceptron to predict the human pose.

5. The explicit denoising-driven spatiotemporal fusion 3D human pose estimation method according to claim 1, characterized in that, Also includes During the training phase, the 3D human pose estimation network is trained end-to-end using a human pose dataset containing different people, different poses and random occlusions, to optimize model parameters and improve robustness to long-sequence noise and occlusion scenes. During the testing phase, the human video to be estimated is input into the trained network model, which then passes through preprocessing, an explicit denoising attention module, and a spatiotemporal fusion module. Finally, the predicted 3D human pose is output through a regression layer.

6. The explicit denoising-driven spatiotemporal fusion 3D human pose estimation method according to claim 1, characterized in that, Step S1 also includes data preprocessing and feature encoding of the input video, as follows: The input video data is sliced ​​according to a preset number of frames, and a 2D pose estimation network is used to extract the 2D coordinates and confidence information of each joint in each frame to construct a 2D joint coordinate sequence. The 2D joint coordinate sequence is mapped to a high-dimensional feature space by linear projection to obtain the initial joint feature vector. Then, a learnable spatial position code, a temporal position code, and a human anatomy group code are superimposed on each joint feature vector to enable the model to obtain the relative spatial topology information between joints, the temporal sequence information between video frames, and the constraint information that conforms to the human physiological structure.

7. The explicit denoising-driven spatiotemporal fusion 3D human pose estimation method according to claim 1, characterized in that, The explicit denoising attention module described in step S1 achieves signal-to-noise decoupling of human motion features through parallel collaborative modeling of the signal and noise branches. The specific steps include: Signal self-attention modeling: a multi-head self-attention mechanism is used to construct signal branches and capture stable semantic structure information reflecting the laws of human movement in parallel through multiple independent subspaces, including the spatial association of the topological structure of the human skeleton and the coherent motion trajectory in long sequences. Constrained noise self-attention modeling: Construct a noise branch parallel to the signal branch and introduce a dual constraint mechanism to limit the modeling capacity so that only unstructured residual components can be captured; Feature Space Alignment and Dynamic Gating: A learnable alignment matrix is ​​introduced to map the features of the noisy branch to the attention subspaces of the signal branch, and a learnable scalar is introduced. As a dynamic gating coefficient, the noise reduction intensity is adaptively adjusted according to the sensitivity of different feature dimensions; Explicit differential correction: Residual correction is performed by the signal branch output and the noise component after dynamic gating. Redundant information is explicitly stripped at the feature level, thereby obtaining a high signal-to-noise ratio feature representation that conforms to human physiological constraints and suppresses inter-frame jitter and occlusion noise.

8. The explicit denoising-driven spatiotemporal fusion 3D human pose estimation method according to claim 1, characterized in that, The steps of the dual-path spatiotemporal fusion module in step S1 are as follows: Spatial-temporal modeling path: Following the processing logic of "space first, time later", we first use explicit denoising attention units in the spatial dimension to capture the topological relationships between joints in a single frame and extract spatial enhancement features that satisfy human anatomical constraints. Subsequently, the cross-frame feature sequences of each joint are modeled to capture long-term temporal motion dependencies and suppress trajectory jitter, generating spatial-temporal feature representations; Temporal-spatial modeling path: Following the reverse logic of "time first, space later", the joint motion trajectory is first smoothed by using explicit denoising attention units in the temporal dimension, and then the single-frame joint structure is refined in the spatial dimension to generate a temporal-spatial feature representation that complements the first path. Adaptive spatiotemporal feature fusion: Based on the sensitivity of the attention mechanism to the processing order, a learnable balance factor is used to dynamically aggregate the complementary features generated by the two paths. By integrating heterogeneous spatiotemporal information, the problem of error accumulation and propagation in single-path sequential modeling is alleviated, and a robust final spatiotemporal feature representation is obtained.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it causes the processor to perform the steps of the method as described in any one of claims 1 to 8.

Images