3D human body posture estimation method and system based on space time sequence information fusion

Abstract

The invention belongs to the technical field of computer vision and robot collaborative perception, and relates to a 3D human body posture estimation method and system based on space time sequence information fusion. The method comprises the steps that a multi-view image sequence is collected, and 2D joint coordinates of all view angles are extracted from the multi-view image sequence through a posture detector; position embedding, global embedding and edge embedding are carried out based on the 2D joint coordinates, and spatial features, including position information, global association information and skeleton edge information, of each joint point are obtained; fusing the features of each visual angle in the spatial features through a cross-visual-angle attention mechanism to obtain global feature representation; performing spatio-temporal feature enhancement and time sequence mixing on the global feature representation to obtain features integrating time sequence information, space information and channel information; and performing 3D posture regression on the features of the integrated time sequence information, the space information and the channel information to obtain a 3D human body posture. According to the invention, accurate and robust multi-view 3D human body posture estimation can be realized.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and robot collaborative perception technology, specifically involving a method and system for solving multi-view 3D human pose estimation in multi-robot collaborative systems using a graph contrast learning Transformer (STCGformer) based on spatial-temporal information fusion. Background Technology

[0002] Robotics is increasingly integrating into all aspects of human life and work, from industrial manufacturing to home services, from medical rehabilitation to security monitoring. Multi-robot collaborative systems have become a research hotspot due to their higher efficiency, robustness, and task execution range. In these systems, robots need to possess accurate perception of their surroundings, especially human activities, to achieve safe, efficient, and natural interaction and collaboration. 3D Human Pose Estimation (HPE), as one of the core tasks of computer vision, aims to recover the 3D coordinates of human joints from images or videos, thereby providing robots with crucial 3D skeletal information for understanding human behavioral intentions. This is of paramount importance for improving the autonomy and intelligence level of multi-robot systems.

[0003] While significant progress has been made in 2D human pose estimation using monocular vision, the transition from 2D to 3D has been a long-standing and challenging problem. A single 2D image may correspond to multiple physically plausible 3D poses, leading to severe depth ambiguity. To overcome this challenge, multi-view 3D HPE methods utilizing multi-view visual information have emerged. These methods simultaneously observe the target from different spatial locations and use geometric constraints to resolve depth information, theoretically providing more reliable 3D reconstruction results. However, existing methods still face a series of severe limitations when applied to dynamic and complex multi-robot collaborative scenarios.

[0004] First, in terms of spatial feature modeling, many current state-of-the-art models are based on Graph Convolutional Networks (GCNs) or Transformer architectures. While GCN methods can explicitly model the connections between human joints, most works focus on local adjacency relationships or a single global context, failing to fully exploit the multi-layered topological structure inherent in the human skeleton, such as physical connections (edge ​​information) between joints and higher-order global joint associations, resulting in insufficient utilization of the spatial geometric constraints of pose. On the other hand, some Transformer models treat human joints as independent tokens. Although this can capture long-range dependencies, it ignores the inherent structural constraints between joints, making the learned spatial representation lack physical plausibility and limiting the accuracy of the final pose estimation.

[0005] Secondly, in multi-view fusion strategies, traditional methods often heavily rely on precise camera parameters (such as intrinsic and extrinsic parameters) for geometric alignment or triangulation between views. In multi-robot systems, the robot platform is frequently in motion, and the camera's viewpoint changes dynamically. Furthermore, the system may use low-cost sensors, making it difficult to accurately calibrate camera parameters in real time or resulting in significant errors. This sensitivity to camera parameters makes many existing algorithms insufficiently adaptable and robust in real-world, decentralized robotic applications, struggling to maintain stable performance when camera parameters are unknown or inaccurate.

[0006] Furthermore, occlusion in complex scenarios presents another major challenge. In collaborative work environments or robot-surrounded scenarios, self-occlusion by humans, object occlusion, and mutual occlusion between robots occur frequently. Most existing methods only utilize information within a very short time sequence (such as a few adjacent frames), lacking the ability to effectively model long-term temporal dependencies. When severe occlusion occurs, the estimation results are prone to significant deviations or jitter because it is impossible to effectively infer the reasonable position and motion trend of the occluded joints from historical pose sequences, failing to meet the requirements of robot systems for continuous, smooth, and reliable pose tracking.

[0007] Therefore, there is an urgent need for a comprehensive solution that can take into account powerful spatial feature extraction capabilities, adaptability to uncalibrated or dynamic camera perspectives, and long-term temporal modeling to combat occlusion. Summary of the Invention

[0008] This invention addresses the core pain points in the aforementioned multi-robot application scenarios by proposing a novel end-to-end framework. It aims to achieve accurate and robust multi-view 3D human pose estimation by deeply fusing spatial and temporal information without relying on camera parameters, providing key technical support for improving the environmental perception and interaction capabilities of multi-robot systems.

[0009] The purpose of this invention is to overcome the core challenges faced by existing 3D human pose estimation technologies in multi-robot collaborative perception scenarios, including insufficient spatial feature modeling, over-reliance on camera calibration parameters, and poor robustness in dynamic occlusion environments. To achieve this goal, this invention proposes an innovative framework called STCGformer, which integrates spatial and temporal information processing through an end-to-end deep learning model, ultimately outputting accurate and smooth 3D human pose sequences.

[0010] The technical solution adopted in this invention is as follows: A 3D human pose estimation method based on spatial-temporal information fusion includes the following steps: Acquire multi-view image sequences and extract 2D joint coordinates from each viewpoint using a pose detector; Based on 2D joint coordinates, position embedding, global embedding and edge embedding are performed to obtain the spatial features of each joint point containing position information, global association information and bone edge information; By fusing features from various perspectives in the spatial features through a cross-view attention mechanism, a global feature representation is obtained; The global feature representation is enhanced with spatiotemporal features and mixed with temporal sequence to obtain features that integrate temporal, spatial and channel information. 3D pose regression is performed on the features of the integrated temporal, spatial and channel information to obtain 3D human pose.

[0011] Furthermore, the spatial features of each joint point, including position embedding, global embedding, and edge embedding based on 2D joint coordinates, containing position information, global association information, and bone edge information, are obtained, including: Mapping 2D joint coordinates to high-dimensional features yields the position embedding; Calculating global embeddings based on human skeleton diagrams: , ;in, Represents the cascade of features of a K-order graph. Represents global embedded features. Represents the ReLU activation function. This indicates the embedding of joint coordinates. This represents a learnable weight matrix. The first part representing the human skeleton diagram adjacency matrix of order, This represents the normalized adjacency matrix. Degree matrix; Edge features are constructed based on the skeleton connection relationships to obtain edge embeddings; The location embedding, global embedding, and edge embedding are fused through multi-head semantic attention to output viewpoint-independent spatial features.

[0012] Furthermore, the edge feature is an edge matrix constructed based on the Euclidean distance vector between the coordinates of two connected joints. Each element , and This represents the coordinates of two connected joints.

[0013] Furthermore, the step of fusing features from various perspectives in the spatial features through a cross-view attention mechanism to obtain a global feature representation includes: Separate the various perspectives of spatial features, and analyze the perspectives. By adding cross-view embeddings and temporal embeddings to the features, enhanced features are obtained; For each target perspective Using its enhanced features as the query, and taking the enhanced features of all perspectives as keys and values ​​respectively, a cross-attention mechanism is used to... Actively retrieve and aggregate complementary information from all perspectives to obtain an enhanced representation for each perspective. ; The viewpoint is assigned using the Sigmoid function. weight By aggregating features from various perspectives, a global feature representation that integrates complementary information from multiple perspectives is obtained: .

[0014] Furthermore, the spatiotemporal feature enhancement and temporal mixing of the global feature representation yields features integrating temporal, spatial, and channel information, including: The global feature representation and global embedding are subjected to a dot product operation, and then transformed by batch normalization layers and convolutional layers to enhance the features; A deep network based on causal delay convolution is used to capture historical frame information. For a given first frame... Embedding at frame time After residual connection and layer normalization operations, features integrating temporal, spatial, and channel information are obtained. .

[0015] Furthermore, the step of performing 3D pose regression on the features integrating temporal, spatial, and channel information to obtain 3D human pose includes: Based on characteristics Obtain spatiotemporal characteristics ),in This represents a multilayer perceptron, which is then regressed to a 3D pose through a fully connected layer. The loss function used Includes mean squared error loss and contrastive learning loss .

[0016] Furthermore, the loss function The combined calculation formula is as follows: in, Represents a true 3D pose label; , Represents hyperparameters; N represents the key points of the human body; features extracted from different viewpoints within the same time frame. and Treat them as positive sample pairs; contrast loss It shortens the feature distance between different viewpoints in the same frame, improving viewpoint consistency; sim is a cosine similarity function; τ It is the temperature coefficient.

[0017] A 3D human pose estimation system based on spatial-temporal information fusion, comprising: The multi-view 2D pose extraction module is used to acquire multi-view image sequences and extract 2D joint coordinates of each view from the multi-view image sequence through a pose detector. The spatial feature extraction module is used to perform position embedding, global embedding and edge embedding based on 2D joint coordinates to obtain the spatial features of each joint point, which include position information, global association information and bone edge information. The multi-view fusion module is used to fuse features from various perspectives in the spatial features through a cross-view attention mechanism to obtain a global feature representation. The spatiotemporal modeling module is used to perform spatiotemporal feature enhancement and temporal mixing on the global feature representation to obtain features that integrate temporal, spatial and channel information. The 3D pose regression module is used to perform 3D pose regression on the features that integrate the three types of information: time, space and channel, to obtain the 3D human pose.

[0018] This invention presents a complete solution through the cascading and collaborative operation of three modules: CGT, CFT, and STM. This method is the first to introduce a contrastive learning strategy into 3D human skeleton estimation, achieving efficient multi-view fusion and long-term temporal modeling without relying on camera parameters. Compared to traditional methods, this invention can more effectively handle complex perception tasks in multi-robot systems. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall process of the method of the present invention, showing the complete processing flow from multi-view image input to 3D pose output.

[0020] Figure 2 This is a diagram of the STM module architecture, designed to model long-term temporal dependencies in fused features of consecutive frames. Its backbone includes layer normalization and a series of causal delayed convolutional network (DTCN) units. The DTCN units employ causal delayed convolution to ensure temporal causality and exponentially increase the receptive field, thereby efficiently capturing long-term motion context. This module also integrates hybrid operations of spatiotemporal and channel dimensions to comprehensively extract spatiotemporal correlation information from the sequence. This design enables STM to effectively utilize historical frame information to infer the joint positions that may be occluded in the current frame, significantly improving the accuracy and temporal smoothness of pose estimation results under occlusion conditions. Detailed Implementation

[0021] The present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0022] The present invention provides a 3D human pose estimation method based on spatial-temporal information fusion, which mainly includes the following: 1) Spatial feature enhancement module: Comparison graph Transformer (CGT).

[0023] This module aims to fundamentally improve the quality of spatial feature representations of human pose from a single viewpoint. Unlike traditional methods that simply treat human joints as graph nodes and perform simple neighborhood aggregation, the CGT module employs a richer embedding strategy. It first constructs a comprehensive embedding representation for each joint, incorporating positional information, global association information, and skeletal edge information. Specifically, positional embedding captures the absolute coordinates of the joint; global embedding, by calculating a multi-order adjacency matrix, characterizes the deep topological relationships between joints that go beyond direct connections; and edge embedding precisely describes the physical properties and orientation of the bones connecting two joints. All these embeddings are fed into a Transformer-based graph encoder. This encoder utilizes its self-attention mechanism to dynamically learn the complex interdependencies between joints and uses edge embedding information to semantically constrain and guide the attention weights, thereby learning a more discriminative and viewpoint-independent representation of the essential features of human pose. This step lays a solid foundation for subsequent multi-view fusion.

[0024] 2) Multi-view adaptive fusion module: Complementary view fusion Transformer (CFT).

[0025] After extracting high-quality spatial features from each single viewpoint, this invention designs a CFT module to intelligently fuse information from multiple robot perspectives. The core advantage of this module lies in its complete elimination of dependence on precise camera calibration parameters, making it particularly suitable for multi-robot systems where camera positions may dynamically change. The CFT module works by simulating complementary observations between multiple perspectives. Through a cross-attention mechanism, features from one perspective (as a query) actively "inquire" about features from all other perspectives (as keys and values), thereby automatically discovering and aggregating complementary information that can compensate for its own perspective deficiencies or ambiguities. This process is performed independently for each perspective, ensuring that the features of each perspective are enhanced. Finally, the module adaptively fuses all enhanced perspective features into a unified global feature representation containing omnidirectional spatial information through a learnable weighting mechanism. This method effectively solves the problem of missing local information caused by perspective differences and occlusion.

[0026] 3) Spatiotemporal mixing module: Spatiotemporal mixer (STM).

[0027] To address the unavoidable occlusion problem in dynamic scenes and generate temporally coherent pose sequences, this invention introduces an STM module specifically designed for modeling long-term spatiotemporal dependencies. Human motion exhibits high continuity and smoothness over short periods, meaning that occluded joint information in the current frame is likely implicit in the motion trajectories of historical frames. The STM module receives a continuous frame sequence output by the CFT module, which has undergone spatial augmentation and multi-view fusion. It employs a deep network structure incorporating causal-delayed convolutions, which efficiently mixes and refines historical frame information over long time spans (e.g., 81 frames). In this way, the model can not only perceive subtle local motion changes but also grasp global motion trends. Therefore, when faced with occlusion, it can make reasonable predictions based on historical motion patterns, significantly improving the robustness and temporal smoothness of the estimation results.

[0028] In one embodiment of the present invention, a 3D human pose estimation method based on spatial-temporal information fusion is provided, the overall process of which is as follows: Figure 1 As shown in the figure, the following steps are included: Step 1: Multi-view 2D pose extraction.

[0029] In a multi-robot system, each robot is equipped with a camera to capture multi-view image sequences. ,in V For the number of viewpoints, t This is a time frame. 2D joint coordinates for each viewpoint are extracted using a pre-trained 2D pose detector (such as CPN). ,in T For sequence length, J The number of joints, after being extracted by CPN, is used to obtain the coordinate encoding of the two-dimensional joint coordinates, which is then converted to a higher dimension as follows: , of which table Encoder.

[0030] Step 2: Spatial feature extraction (CGT module).

[0031] The CGT module enhances the structured representation of the human skeleton by embedding location, global, and edge features.

[0032] 1) Location embedding: The coordinates obtained in step 1 are encoded using a learnable function. Mapping to high-dimensional features This embedding captures the absolute position information of the joint.

[0033] 2) Global Embedding: To capture higher-order topological relationships between joints that go beyond direct physical connections, a global embedding is based on the human skeleton graph (its first-order...). The adjacency matrix of order is The formula for calculating global embedding is: , in, Represents the cascade of features of a K-order graph. Represents global embedded features. Represents the ReLU activation function. This indicates the embedding of joint coordinates. This represents a learnable weight matrix. This represents the normalized adjacency matrix. Degree matrix, This represents a k-order adjacency relationship (i.e., joints are connected by k edges). This embedding enables the model to be aware of global dependencies between joints.

[0034] 3) Edge embedding: To explicitly model the physical properties of the skeleton, edge features are constructed based on the skeleton's connection relationships. For each edge present in the graph (connecting joints)... and Its edge features are edge matrices constructed based on the Euclidean distance vectors of the coordinates of two connected joints. Each element .in, and This represents the coordinates of two connected joints.

[0035] The above embeddings, after being fused by multi-head semantic attention (MHSA), output viewpoint-independent spatial features. Its update rules are as follows: Among them, MHSA is the multi-head self-attention mechanism; LN represents the layer normalization operation; and , It refers to different feature transformation functions in the feedforward network, specifically different linear layers, which transform the features to the same higher dimension as after passing through the multi-head self-attention mechanism; Indicates element-wise multiplication; The depth of the stacked multi-head attention layer; This represents a feature transformation function, specifically a linear layer, designed to align dimensions.

[0036] Step 3: Multi-view fusion (CFT module).

[0037] The CFT module fuses features from multiple robot perspectives through a cross-view attention mechanism: 1) Viewpoint-specific encoding: The spatial features obtained in step 2 are encoded... The various perspectives (e.g., four perspectives) are separated, and the perspectives are... Features Add cross-view embedding With temporal embedding Enhanced features are obtained Cross-perspective embedding Used to label different viewpoints, temporal embedding It can be used to mark different timing sequences and can be implemented using existing technologies.

[0038] 2) Cross-view attention: For each target viewpoint Enhance its features As a query (Q), and all perspectives (including) Enhancement features of itself ( j =1,..., V These are respectively used as keys (K) and values ​​(V). Through a cross-attention mechanism, the perspective... Actively retrieve and aggregate complementary information from all perspectives: in, This represents the query, key, and value projection (obtained by linear transformation of features from various perspectives). Indicates the dimension of the key vector, used to scale the dot product result; Represents the ReLU activation function; These are the global embedding features generated in step 2, used to incorporate spatial structure priors. This operation is performed independently for each viewpoint, resulting in an enhanced representation for each viewpoint. .

[0039] 3) Adaptive weighted fusion: Viewpoints are obtained through Sigmoid function allocation. weight By aggregating features from various perspectives, we obtain: This step outputs a global feature representation that incorporates complementary information from multiple perspectives. Then, it is passed to step 4 for time series modeling.

[0040] Step 4: Spatiotemporal modeling (STM module).

[0041] The STM module obtains the result from step 3. Perform long-term temporal modeling to address occlusion and motion blur: 1) Spatiotemporal feature enhancement: This enhances the output of step 3. With global embedding Perform a dot product operation, then transform the data through a batch normalization (BN) layer and a 1×1 convolutional layer to enhance the features: in, This represents the dot product operation. This represents the convolution operation. This represents the ReLU activation function.

[0042] 2) Temporal Hybrid: STM is a deep network based on causal-delayed convolution, such as... Figure 2 As shown, historical frame information can be captured for a given frame. Embedding at frame time After residual connection and layer normalization operations, features integrating temporal, spatial, and channel information can be obtained. : Step 5: 3D pose regression and loss function.

[0043] According to the output of step 4 Obtain spatiotemporal characteristics ),in This represents a multilayer perceptron, which is then regressed to a 3D pose through a fully connected layer. Loss function It combines mean squared error loss and contrastive learning loss. : in, Represents a true 3D pose label. , The hyperparameter is used to balance the importance of the two losses, and N represents the key points of the human body (17). To enhance viewpoint invariance, an InfoNCE-based contrast loss is introduced, which integrates features extracted by the CGT module from different viewpoints at the same time frame. and Treat them as positive sample pairs. Comparative loss. This reduces the feature distance between different viewpoints within the same frame, improving viewpoint consistency. `sim` is a cosine similarity function. τ It is the temperature coefficient.

[0044] Using the proposed solution (STCGformer), the average MPJPE reaches 27.4 mm when using CPN to detect 2D poses on the Human3.6M dataset, a significant improvement over the baseline method; when using real 2D poses, it further reduces the MPJPE to 11.6 mm. In cross-scene tests on the 3DHP dataset, the PCK metric reaches 99.8%, demonstrating its strong generalization ability.

[0045] Another embodiment of the present invention provides a 3D human pose estimation system based on spatial-temporal information fusion, comprising: The multi-view 2D pose extraction module is used to acquire multi-view image sequences and extract 2D joint coordinates of each view from the multi-view image sequence through a pose detector. The spatial feature extraction module is used to perform position embedding, global embedding and edge embedding based on 2D joint coordinates to obtain the spatial features of each joint point, which include position information, global association information and bone edge information. The multi-view fusion module is used to fuse features from various perspectives in the spatial features through a cross-view attention mechanism to obtain a global feature representation. The spatiotemporal modeling module is used to perform spatiotemporal feature enhancement and temporal mixing on the global feature representation to obtain features that integrate temporal, spatial and channel information. The 3D pose regression module is used to perform 3D pose regression on the features that integrate the three types of information: time, space and channel, to obtain the 3D human pose.

[0046] The above division of modules is merely illustrative. In practical applications, the functions described above can be assigned to different functional modules as needed to complete all or part of the functions described in the aforementioned method. The specific working process of each module can be found in the corresponding processes in the aforementioned method embodiments.

[0047] Another embodiment of the present invention provides a computer device (computer, server, smartphone, etc.) including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing steps of the method of the present invention.

[0048] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk) that stores a computer program, which, when executed by a computer, implements the steps of the method of the present invention.

[0049] Another embodiment of the present invention provides a computer program product, the computer program product including a computer program, which, when executed by a computer, implements the steps of the method of the present invention.

[0050] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Those skilled in the art can modify or make equivalent substitutions to the technical solutions of the present invention without departing from the spirit and scope of the present invention. The scope of protection of the present invention should be determined by the claims.

Claims

1. A 3D human pose estimation method based on spatial-temporal information fusion, characterized in that, Includes the following steps: Acquire multi-view image sequences and extract 2D joint coordinates from each viewpoint using a pose detector; Based on 2D joint coordinates, position embedding, global embedding and edge embedding are performed to obtain the spatial features of each joint point containing position information, global association information and bone edge information; By fusing features from various perspectives in the spatial features through a cross-view attention mechanism, a global feature representation is obtained; The global feature representation is enhanced with spatiotemporal features and mixed with temporal sequence to obtain features that integrate temporal, spatial and channel information. 3D pose regression is performed on the features of the integrated temporal, spatial and channel information to obtain 3D human pose.

2. The method according to claim 1, characterized in that, The method involves position embedding, global embedding, and edge embedding based on 2D joint coordinates to obtain spatial features for each joint point that include position information, global association information, and bone edge information, including: Mapping 2D joint coordinates to high-dimensional features yields the position embedding; Calculating global embeddings based on human skeleton diagrams: , ;in, Represents the cascade of features of a K-order graph. Represents global embedded features. Represents the ReLU activation function. This indicates the embedding of joint coordinates. This represents a learnable weight matrix. The first part representing the human skeleton diagram adjacency matrix of order, This represents the normalized adjacency matrix. Degree matrix; Edge features are constructed based on the skeleton connection relationships to obtain edge embeddings; The location embedding, global embedding, and edge embedding are fused through multi-head semantic attention to output viewpoint-independent spatial features.

3. The method according to claim 1, characterized in that, The edge features are edge matrices constructed based on the Euclidean distance vectors of the coordinates of two connected joints. Each element , and This represents the coordinates of two connected joints.

4. The method according to claim 1, characterized in that, The process of fusing features from various perspectives within the spatial features through a cross-view attention mechanism to obtain a global feature representation includes: Separate the various perspectives of spatial features, and analyze the perspectives. By adding cross-view embeddings and temporal embeddings to the features, enhanced features are obtained; For each target perspective Using its enhanced features as the query, and taking the enhanced features of all perspectives as keys and values ​​respectively, a cross-attention mechanism is used to... Actively retrieve and aggregate complementary information from all perspectives to obtain an enhanced representation for each perspective. ; The viewpoint is assigned using the Sigmoid function. weight By aggregating features from various perspectives, a global feature representation that integrates complementary information from multiple perspectives is obtained: .

5. The method according to claim 4, characterized in that, The process of enhancing and temporally blending the global feature representation to obtain features that integrate temporal, spatial, and channel information includes: The global feature representation and global embedding are subjected to a dot product operation, and then transformed by batch normalization layers and convolutional layers to enhance the features; A deep network based on causal delay convolution is used to capture historical frame information. For a given first frame... Embedding at frame time After residual connection and layer normalization operations, features integrating temporal, spatial, and channel information are obtained. .

6. The method according to claim 5, characterized in that, The process of performing 3D pose regression on the features of the integrated temporal, spatial, and channel information to obtain 3D human pose includes: Based on characteristics Obtain spatiotemporal characteristics ),in This represents a multilayer perceptron, which is then regressed to a 3D pose through a fully connected layer. The loss function used Includes mean squared error loss and contrastive learning loss .

7. The method according to claim 6, characterized in that, The loss function The combined calculation formula is as follows: in, Represents a true 3D pose label; , Represents hyperparameters; N represents the key points of the human body; features extracted from different viewpoints within the same time frame. and Treat them as positive sample pairs; contrast loss It shortens the feature distance between different viewpoints in the same frame, improving viewpoint consistency; sim is a cosine similarity function; τ It is the temperature coefficient.

8. A 3D human pose estimation system based on spatial temporal information fusion, characterized in that, include: The multi-view 2D pose extraction module is used to acquire multi-view image sequences and extract 2D joint coordinates of each view from the multi-view image sequence through a pose detector. The spatial feature extraction module is used to perform position embedding, global embedding and edge embedding based on 2D joint coordinates to obtain the spatial features of each joint point, which include position information, global association information and bone edge information. The multi-view fusion module is used to fuse features from various perspectives in the spatial features through a cross-view attention mechanism to obtain a global feature representation. The spatiotemporal modeling module is used to perform spatiotemporal feature enhancement and temporal mixing on the global feature representation to obtain features that integrate temporal, spatial and channel information. The 3D pose regression module is used to perform 3D pose regression on the features that integrate the three types of information: time, space and channel, to obtain the 3D human pose.

9. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer, implements the method according to any one of claims 1 to 7.

Images