Point cloud 3D human body posture estimation method based on geometric adaptive local coordinate system

Abstract

The invention discloses a point cloud 3D human body posture estimation method based on a geometric adaptive local coordinate system, and relates to the technical field of 3D human body posture estimation. The method comprises the following steps: based on a geometric adaptive local coordinate system module, dynamically constructing a local coordinate system taking a key point as a center, enhancing the capturing capability of fine-grained local features in a point cloud, and performing local feature extraction; based on a double-branch global feature extraction module, dynamically selecting an architecture according to a point cloud type and a task demand, keeping the consistency of local features and global features, and fusing the local features to generate global attitude representation; and based on the refining module, the refining precision of attitude estimation is improved through a self-attention mechanism and attitude priori knowledge. Through the innovative GAL module and the high-efficiency GFE module, high-precision, robust and high-real-time 3D human body posture estimation is realized, and the limitations of the prior art in the aspects of insufficient local fine-grained feature extraction, high calculation complexity and the like are remarkably overcome.

Description

Technical Field

[0001] This invention belongs to the field of 3D human pose estimation technology, and particularly relates to a point cloud 3D human pose estimation method based on a geometrically adaptive local coordinate system. Background Technology

[0002] 3D human pose estimation is an important technology in the field of computer vision. It aims to accurately identify and locate the positions of key points or joints of the human skeleton from 3D input data (such as point clouds, RGB images or depth maps). It is widely used in virtual reality (VR), augmented reality (AR), motion analysis, health monitoring, human-computer interaction, autonomous driving and robot navigation. The algorithm is required to have high accuracy, robustness and real-time performance in complex scenarios such as occlusion, sparse data or diverse poses.

[0003] Existing methods mainly fall into four categories: 1) 2D-to-3D upscaling methods based on RGB images, which use convolutional neural networks (CNNs) or Transformers to extract key point features from 2D images and infer 3D positions through regression or geometric optimization. These methods are simple in structure but are susceptible to occlusion and lighting effects due to a lack of depth information. 2) Feature extraction methods based on point clouds, which extract local features through Farthest Point Sampling (FPS) and K-Nearest Neighbors (KNN) grouping, and then generate a global pose representation through max pooling, attention mechanisms, or state space modeling. This overcomes the lack of depth information but insufficient extraction of fine-grained local features in the global coordinate system. 3) End-to-end methods based on LiDAR point clouds, which combine high-precision 3D coordinates from LiDAR point clouds with feature extraction and regression to estimate pose. These methods are suitable for outdoor scenes but neglect local geometric details. 4) Methods based on local-global feature fusion balance consistency and detail through global modeling (using Transformer or SSM to capture long-distance dependencies) and local refinement (combining kinematic constraints). However, local feature extraction is mostly performed in the global coordinate system, making it difficult to adapt to geometric changes. The principle is based on geometric and spatiotemporal modeling: local features are generated through the covariance matrix or distance relationships of neighboring points, while global features capture long-distance dependencies through attention mechanisms or dynamic state matrices. The optimization objective is to minimize the Euclidean distance of keypoints (e.g., MPJPE). Some methods introduce geometric priors or environmental cues to enhance robustness. However, existing technologies still have limitations in fine-grained feature extraction, computational efficiency, and adaptability to complex scenes in sparse point cloud regions, providing room for further innovation.

[0004] This invention will solve the problems existing in the prior art from the following two aspects:

[0005] 1) Insufficient extraction of fine-grained local features. Existing methods struggle to effectively capture local geometric details when processing sparse point cloud regions (such as hands, knees, and feet), resulting in low keypoint localization accuracy, especially in complex poses or occluded scenes. 2D-to-3D upscaling methods based on RGB images lack depth information and cannot accurately represent 3D geometric structures; point cloud-based and LiDAR point cloud-based methods rely on a global coordinate system, extracting features through FPS sampling and KNN grouping, ignoring local geometric changes and making it difficult to capture fine-grained features; while local-global fusion methods combine local refinement, they extract features in a global coordinate system and lack adaptation mechanisms for local geometric structures.

[0006] 2) Insufficient geometric adaptability to complex scenes. Existing methods struggle to dynamically adapt to diverse point cloud geometries (such as different human poses or environmental changes), exhibiting weak generalization ability. RGB image methods are limited by a lack of 3D information; point cloud and LiDAR methods, based on fixed global coordinate systems or static priors, struggle to adjust feature extraction strategies according to local geometric characteristics; while local-global methods introduce kinematic constraints, these constraints are mostly statically designed and lack adaptive mechanisms. These problems stem from the insufficient adaptation of existing methods to local geometry and computational efficiency in structural design, providing room for improvement in developing more efficient and robust 3D human pose estimation techniques.

[0007] This invention proposes a point cloud 3D human pose estimation method based on a geometrically adaptive local coordinate system. By introducing a learnable geometric bias matrix and a dynamically adjusted local coordinate system, the method enhances the adaptability to complex point cloud geometry and improves the generalization ability in diverse scenarios. Summary of the Invention

[0008] The purpose of this invention is to provide a point cloud 3D human pose estimation method based on a geometrically adaptive local coordinate system, so as to solve the problems of insufficient local fine-grained feature extraction and insufficient geometric adaptability to complex scenes in the existing methods proposed in the background art.

[0009] To achieve the above objectives, the present invention employs the following technical solution:

[0010] In its first aspect, this invention proposes a point cloud 3D human pose estimation method based on a geometrically adaptive local coordinate system, comprising the following steps:

[0011] S1. Local Feature Extraction: Based on the geometrically adaptive local coordinate system module, a dynamic local coordinate system centered on key points is constructed to enhance the ability to capture fine-grained local features in the point cloud and perform local feature extraction.

[0012] S2. Global Feature Fusion: Based on the dual-branch global feature extraction module, the architecture is dynamically selected according to the point cloud type and task requirements to maintain the consistency between local and global features; local features are fused to generate a global pose representation.

[0013] S3, Pose Regression: Based on the refinement module, it improves the refinement accuracy of pose estimation by integrating prior knowledge through a self-attention mechanism and utilizing prior knowledge of pose.

[0014] Preferably, the geometrically adaptive local coordinate system module in S1 is specifically as follows:

[0015] Key points are selected from the point cloud using farthest point sampling. Neighboring points are grouped using the K-nearest neighbor algorithm, and the spatial distribution of the neighboring points is analyzed to generate the normal vector, tangent vector, and second tangent vector of the local coordinate system. A learnable bias matrix is ​​introduced to optimize the geometric adaptability of the coordinate system. After the neighboring points are mapped to the local coordinate system, local features are extracted through a geometric affine module and a one-dimensional convolutional layer. At the same time, relative position loss is used to ensure the geometric consistency of the key point neighborhood.

[0016] Preferably, S1 is specifically as follows:

[0017] Given a set of points Where N represents a point in the (x,y,z) Cartesian coordinate system, all points are fed into a feature extraction layer consisting of a geometrically adaptive local coordinate system module and a Mamba module; in each layer, the farthest point sampling is used to resample the N points, and then the K nearest neighbor algorithm is used to find the K neighbors of the N points;

[0018] The sampled point and its neighbor points are fed together into the eigenvector decomposition module. The product of the differences between the sampled point and its neighbor is summed to obtain the covariance matrix. The covariance matrix is ​​then subjected to eigenvector decomposition to obtain the normal vector, tangent vector, and second tangent vector, as shown below:

[0019]

[0020] Among them, C i Represents the covariance matrix; Represents the eigenvector matrix; This represents a diagonal matrix containing the eigenvalues ​​of the covariance matrix;

[0021] At the same time, a learnable paranoia matrix is ​​introduced. The bias matrix is ​​applied to the three vectors and concatenated into a GAL rotation matrix. Finally, the neighborhood points are mapped to the local coordinate system using the rotation matrix, thus transforming each neighborhood point from the global coordinate system to its corresponding local coordinate system, as shown below:

[0022]

[0023] n i =V i [:,0]

[0024] t1 i =V i [:,1]

[0025] t2 i =V i [:,0]×V i [:,1]

[0026] The points in the obtained local coordinate system are fed into the geometric affine module and the convolutional layer to extract local features, as shown below:

[0027] F final =Conv1d(GAM({p i,j}))

[0028] Among them, {p i,j |j=1,...,k} represents the k neighboring points of pi, GAM is a geometric affine module used to enhance local features, and Conv1d is a one-dimensional convolution;

[0029] Relative position loss is used to ensure geometric consistency of the neighborhood of key points.

[0030] Furthermore, the eigenvector corresponding to the smallest eigenvalue of the covariance matrix is ​​used as the normal vector, the eigenvector corresponding to the second smallest eigenvalue is used as the tangent vector, and the vector obtained by calculating the cross product of the normal vector and the tangent vector is used as the second tangent vector describing the local neighborhood.

[0031] Preferably, the relative position loss Loss position The details are as follows:

[0032]

[0033] The neighboring points are represented as {(k,k′)|k,k′=1,...,K}.

[0034] Preferably, the dual-branch global feature extraction module in S2 is specifically as follows:

[0035] The dual-branch global feature extraction module supports both Mamba and PCT-v2 architectures.

[0036] When the dual-branch global feature extraction module adopts the Mamba architecture, the feature extraction process iteratively executes four stages according to the selected serialization strategy to gradually optimize the feature representation;

[0037] When the dual-branch global feature extraction module adopts the PCT-v2 architecture, it extracts deep, hierarchical features through four iterative cycles.

[0038] Preferably, the refining module in S3 is as follows:

[0039] The refining module is designed based on radar point cloud grid recovery technology, and the refining module includes two self-attention layers.

[0040] In a second aspect, this invention proposes a point cloud 3D human pose estimation system based on a geometrically adaptive local coordinate system, comprising:

[0041] The geometrically adaptive local coordinate system module is used for local feature extraction.

[0042] The dual-branch global feature extraction module adopts both the Mamba and PCT-v2 architectures, dynamically selecting the architecture for global feature fusion.

[0043] The refinement module is used to improve the refinement accuracy of attitude estimation by integrating prior knowledge.

[0044] Compared with the prior art, the beneficial effects of the present invention are:

[0045] (1) The method in this invention achieves high-precision, robust and real-time 3D human pose estimation through innovative GAL module and high-efficiency GFE module, which significantly overcomes the limitations of existing technologies in terms of insufficient local fine-grained feature extraction and high computational complexity.

[0046] (2) The method in this invention enhances the ability to capture fine-grained local features in point clouds by dynamically constructing a local coordinate system centered on key points. Compared with traditional global coordinate system methods (such as PointNet and PRN), the GAL module significantly improves the feature quality of sparse point cloud regions (such as hands and knees), especially showing higher accuracy in complex poses or occluded scenes.

[0047] (3) In this invention, the global feature extraction module can dynamically select the most suitable feature extraction method based on the point cloud type and specific task requirements. By combining the geometrically adaptive local feature extraction module with the global feature extraction module, a novel feature extraction framework is proposed, seamlessly integrating the advantages of both. This hybrid method enhances the ability to capture fine-grained local features of point clouds while ensuring the integrity of global contextual information.

[0048] (4) Experiments on the Sloper4D and HumanM3 datasets show that the present invention significantly outperforms existing technologies in terms of mean joint position error (MPJPE, 4.85 cm vs. PRN's 5.70 cm) and keypoint accuracy (PCK-5, 74.31% vs. 67.08%), especially demonstrating stronger robustness in sparse point cloud regions. GALNet's modular design facilitates integration, has low power consumption, and is suitable for embedded devices. Attached Figure Description

[0049] Figure 1 This is a flowchart of the point cloud 3D human pose estimation method based on a geometrically adaptive local coordinate system in this invention; Figure 2 This is a comparison diagram showing the flow of the present invention and existing 3D human pose estimation methods. Figure 3 This is a visualization diagram of the point cloud 3D human pose estimation method in this invention;

[0050] Figure 4 This is a comparison diagram of the predicted key points and the actual key points in the attitude estimation in this invention;

[0051] Figure 5 This is a visualization diagram of the point cloud 3D human pose estimation method in this invention on two datasets, Sloper4D and HumanM3. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] Example 1:

[0054] This invention proposes a point cloud-based 3D human pose estimation method. Through an innovative Geometric Adaptive Local Coordinate System (GAL) module and a highly efficient Global Feature Extraction (GFE) module, it achieves high-precision, robust, and real-time 3D human pose estimation, significantly overcoming the limitations of existing technologies in areas such as insufficient local fine-grained feature extraction and high computational complexity. The dynamic local coordinate system construction of the GAL module and the efficient global modeling of the GFE module are the key technologies of this invention. Figure 1The GALNet architecture is shown, from top to bottom: the GAL module (local coordinate system construction and feature extraction), the GFE module (global feature fusion), and the overall network (point cloud input, four-layer encoder, and refinement module for regression joints). The method includes the following steps:

[0055] Step 1: Construction of the Geometrically-Adapted Local Coordinate System Network (GALNet), which is built through the Geometrically Adapted Local Coordinate System (GAL) module and used for local feature extraction.

[0056] The process and structure of the GAL module are as follows: Figure 1 As shown, its core function is to enhance the capture of fine-grained local features in point clouds by dynamically constructing a local coordinate system centered on keypoints. The specific process includes: selecting keypoints from the point cloud using farthest point sampling (FPS), grouping neighboring points using the K-nearest neighbor (KNN) algorithm, analyzing the spatial distribution of neighboring points, generating the normal vector, tangent vector, and second tangent vector of the local coordinate system, and introducing a learnable bias matrix to optimize the geometric adaptability of the coordinate system. After the neighboring points are mapped to the local coordinate system, local features are extracted through a geometric affine module (GAM, based on a multilayer perceptron) and a one-dimensional convolutional layer, while relative position loss is used to ensure the geometric consistency of the keypoint neighborhood. Specifically:

[0057] Given a set of points Here, N represents a point in the (x, y, z) Cartesian coordinate system. These points are fed into a feature extraction layer consisting of a GAL module and a Mamba module. In each layer, farthest point sampling (FPS) is used to resample the N points, and then KNN is used to find the K neighbors of the N points. The sampled points and their neighbors are then fed into the eigenvector decomposition module, where the products of the differences between the sampled points and their neighbors are summed to obtain the covariance matrix. The covariance matrix describes the distribution of neighboring points in three-dimensional space and their geometric relationships with the sampled points. The covariance matrix is ​​then subjected to eigenvector decomposition, as shown below.

[0058]

[0059] Where C i This represents the covariance matrix. This represents the eigenvector matrix. This represents a diagonal matrix containing the eigenvalues ​​of the covariance matrix. The covariance matrix essentially measures the degree of variation (variance) of data in various directions and the correlation between different directions. Its eigenvalues ​​represent the magnitude of variation in each direction, while the corresponding eigenvectors represent the direction of data variation. Therefore, the eigenvector corresponding to the smallest eigenvalue of the covariance matrix represents the direction of the smallest variation, i.e., the data has the smallest variance in this direction. This invention selects the eigenvector corresponding to the smallest eigenvalue as the normal vector and the eigenvector corresponding to the second smallest eigenvalue as the tangent vector. The vector obtained by calculating the cross product of the normal vector and the tangent vector is used as the vector describing the second tangent direction of the local neighborhood.

[0060] At the same time, a learnable paranoia matrix is ​​introduced. By training to adapt to geometric transformations in point cloud data, the adaptability and flexibility of the coordinate system are further optimized. Then, the bias matrix is ​​applied to the three vectors to form a GAL rotation matrix. Finally, the neighborhood points are mapped to the local reference coordinate system through the rotation matrix, thus completing the transformation of each neighborhood point from the global coordinate system to its corresponding local coordinate system, as shown below.

[0061]

[0062] n i =V i [:,0]

[0063] t1 i =V i [:,1]

[0064] t2 i =V i [:,0]×V i [:,1]

[0065] The points in the obtained local coordinate system are fed into the geometric affine module and the convolutional layer to extract local features, as shown below.

[0066] F final =Conv1d(GAM({p i,j}))

[0067] Among them, {p i,j |j=1,...,k} represents the k neighboring points of pi. GAM is a geometric affine module proposed by PointMLP to enhance local features. Conv1d is a one-dimensional convolution.

[0068] Point clouds based on a geometrically adaptive local coordinate system need to ensure that the relative positional relationships between neighboring points remain consistent with the original coordinate system after transformation. This is to avoid the local coordinate system distorting the original geometric relationships, which could lead to errors and omissions in the human spatial geometry. Therefore, this invention proposes a relative position loss, used to compare the Euclidean distances of point pairs in the original coordinate system and the geometrically adaptive local coordinate system, to quantify and optimize the geometric consistency of the local coordinate system. Ideally, the local coordinate system should preserve the relative positional relationships between neighboring points in the original point cloud. By calculating the difference in distances between neighboring point pairs in the original and local coordinate systems, we can evaluate whether the local coordinate system effectively maintains these relationships. The loss function is Loss. position This difference is quantified. If the local coordinate system preserves the original relative positions well, then D original and D GAL Ideally, the local coordinates should be very close, resulting in a small loss value. Conversely, if the local coordinate system distorts the relative positions, the loss value will increase. Minimizing this loss encourages the model to learn a better representation of the local coordinate system, thereby improving its ability to capture the local geometry of the point cloud.

[0069]

[0070] The neighboring points are represented as {(k,k′)|k,k′=1,...,K}.

[0071] Step 2: Construction of the feature extraction framework. The feature extraction framework is built based on the dual-branch global feature extraction (GFE) module, which is used for global feature fusion.

[0072] This invention proposes an innovative feature extraction framework that cleverly integrates the advantages of local and global feature extraction to achieve efficient capture of fine-grained local features of point clouds while ensuring the integrity of global contextual information.

[0073] The GFE module is responsible for efficiently fusing local features to generate a global pose representation. Its structure is as follows: Figure 1As shown, this module supports two architectures: the Mamba State Space Model (SSM) and PointCloudTransformer-v2 (PCT-v2). The Mamba architecture models long-sequence dependencies in point clouds through four-layer iterations, combining a dynamic state matrix and global convolution operations. It employs a bidirectional serialization strategy to enhance context capture, resulting in linearly increasing computational complexity, significantly reducing computational overhead compared to traditional Transformer methods (such as PRN). The PCT-v2 architecture extracts deep, hierarchical features through a bidirectional attention mechanism, adapting to complex point cloud structures. The GFE module dynamically selects the architecture based on the point cloud type and task requirements, ensuring a balance between local details and global consistency. It boasts fast inference speed and is suitable for real-time applications such as virtual reality and robot navigation.

[0074] Specifically, when the Global Feature Extraction (GFE) module adopts the Mamba architecture, the feature extraction process iteratively executes four stages according to the selected serialization strategy, progressively optimizing the feature representation. Similarly, when the GFE module uses the PointCloudTransformer (PCT) architecture, it extracts deep, hierarchical features through four iterative cycles, ensuring robust modeling of complex point cloud structures.

[0075] Mamba is a deep learning model based on a state-space model (SSM), balancing long sequence modeling capabilities with computational efficiency. Its core innovation lies in introducing time-varying parameters and optimizing the computation process through global convolution, resulting in excellent performance in both training and inference phases. For continuous state-space models, SSM combines multiple inputs and outputs through state variables, and controls the temporal variation of the state space through three different state matrices, as shown in Equation 7.

[0076] h′(t)=Ah(t)+Bx(t)

[0077] y(t)=Ch(t)

[0078] Here, h(t) represents the system state, x(t) and y(t) represent the system input and output, respectively, and A, B, and C represent the state transition matrix, input matrix, and output matrix, respectively. Continuous models are suitable for describing physical systems, but further transformation is needed for discrete data such as text.

[0079] Mamba introduces a feature on the traditional SSM that allows system parameters to change dynamically with input, as shown below.

[0080] Δ=f(x t )

[0081]

[0082] Where, Δ, All about x t In addition to the functions, to address the parallelization issue during training, Mamba introduces global convolution for computation. This transforms recursive computation into convolutional operations, leveraging the parallel computing capabilities of the GPU to improve efficiency. Furthermore, it allows the convolutional kernel to be influenced by all states of the entire sequence, ensuring the model can capture global dependencies, as shown below.

[0083]

[0084] in, is the convolution kernel, and M is the length of sequence x.

[0085] This invention introduces the sophisticated serialization strategy of the recently proposed PCM and the bidirectional processing of Bimamba-v2, taking into account both past and future information to capture contextual information in the sequence. The input to GFE is the result of concatenating point clouds and local features, and is fed into either Bimamba-v2 or PointCloudTransformer-v2 to capture global features.

[0086] Step 3: Refine the module construction for attitude regression.

[0087] This invention introduces a refinement module inspired by LiDAR-HMR (radar point cloud mesh recovery), which is based on a self-attention mechanism and makes full use of attitude prior knowledge. By effectively integrating prior information, it significantly improves the refinement accuracy of attitude estimation.

[0088] Due to the incomplete nature of point clouds, one or more keypoints of the human body may go unobserved, leading to incomplete estimations of the template pose. To address this issue, this invention introduces a refinement module based on a self-attention mechanism, comprising two self-attention layers. Notably, the refinement module does not receive any point cloud or corresponding feature input; instead, it primarily relies on pose priors learned from the data.

[0089] This invention utilizes a GAL module (local feature extraction), a GFE module (global feature fusion), and a refining module (pose regression). These modules together form an efficient and powerful point cloud processing framework, achieving an excellent balance between preserving local details and maintaining global feature consistency.

[0090] Step 4: Training and optimization of the point cloud processing framework, including network training and inference.

[0091] Training settings: Adam optimizer used, initial learning rate 0.001, batch size 32, training for 100 epochs. Data augmentation included random rotations (±30°) and scaling (±10%).

[0092] Inference process: Input test point cloud, which is then processed sequentially through the GAL module (local feature extraction), GFE module (global feature fusion), and refinement module (pose regression), outputting the 3D coordinates of 17 key points. The inference time is approximately 20ms / frame (RTX4090).

[0093] Output and Visualization: Outputs coordinates of 17 key points (e.g., head, shoulders, hands), which can be mapped to a human skeletal model for applications such as VR and motion analysis. Visualization Reference Figure 3 In the lower half, the estimated skeleton (red) highly overlaps with the actual skeleton (green), verifying high accuracy.

[0094] Performance results: On the Sloper4D test set, MPJPE is 4.85cm, PA-MPJPE is 3.93cm, PCK-3 is 49.90%, and PCK-5 is 74.31%, which is better than PRN (MPJPE 5.70cm, PCK-5 67.08%).

[0095] Experimental verification:

[0096] Based on the above-described invention, the following evaluation of its technical effects is proposed:

[0097] This invention significantly improves the accuracy of key point localization in 3D human pose estimation tasks, especially in sparse point cloud regions (such as hands, knees, and feet) and complex pose scenes. The mean position error per joint (MPJPE) and alignment error (PA-MPJPE) are superior to existing technologies. The comparison results are shown in Tables 1 and 2.

[0098] Table 1. Performance comparison results of the GAL method of this invention with existing best methods.

[0099]

[0100] As shown in Table 1, based on the experimental results, the MPJPE of this invention are 4.85cm and 7.43cm on the Sloper4D and HumanM3 datasets, respectively, which is better than the existing best methods (such as PRN's 5.70cm and 8.22cm).

[0101] Table 2 Comparison of mAP index between the GAL method and the PRN method of this invention

[0102]

[0103] As shown in Table 2, the experimental results demonstrate that in dense point cloud regions such as the torso, where the geometry is relatively simple and the point cloud distribution is relatively uniform, the GAL method of this invention achieves comparable mAP performance to PRN, and in some cases, even slightly lower. This may be because PRN can effectively utilize global features when processing dense point clouds. However, in regions farther from the torso and with sparser point cloud distribution, such as the hand, knee, and foot, the GAL method exhibits significant advantages, with a substantial improvement in mAP compared to PRN. This improvement is mainly attributed to the innovative design of the GAL model in local feature modeling, which can more efficiently capture fine-grained geometric information and joint details in sparse point cloud regions, thereby significantly improving the accuracy of joint localization.

[0104] Figure 2 A comparison between conventional methods and the method of this invention is presented. For example... Figure 2 As shown, the upper part is the traditional method flow: point cloud data is represented in a global coordinate system centered on the geometric origin, and the human skeleton (red) is estimated by the network and compared with the real skeleton (green); the lower part is the flow of the present invention: point cloud data is converted into a geometrically adaptive local coordinate system representation through the GAL module, and the skeleton is estimated by the network and compared with the real skeleton, showing higher accuracy. Figure 3 This is a visualization of the point cloud 3D human pose estimation method in this invention.

[0105] Figure 4 This demonstrates a comparison between the ground truth keypoints and the model-predicted keypoints in pose estimation, showcasing the model's accuracy in this task. Figure 4 As shown, to comprehensively demonstrate the results, this invention visualizes the point cloud and keypoints from three perspectives: 0°, 45°, and 90°. Two different point clouds, A and B, were selected as inputs. Point clouds A and D are relatively complete and dense, enabling both the baseline method and the method of this invention to accurately estimate keypoints. However, in sparser areas such as the knees and feet, the predicted keypoints of this invention are better aligned with the true keypoints. In contrast, point clouds B and C are sparser than point cloud A, with significant point cloud gaps in the foot region. Under these conditions, the method of this invention has a clear advantage. From the three perspectives, the predicted keypoint accuracy for the torso, abdomen, and knees surpasses that of the baseline method, especially showing significant alignment with true keypoints at the knees. However, in the foot region where point clouds are missing, both methods perform poorly. This indicates that addressing missing point cloud regions will be a focus of future work. Figure 5 The point cloud 3D human pose estimation method of this invention is visualized on two datasets, Sloper4D and HumanM3, with point clouds and key points visualized from three perspectives: 0°, 45° and 90°.

[0106] The above description is only for the purpose of helping to understand the method and core essence of the present invention, but the scope of protection of the present invention is not limited thereto. For those skilled in the art, any equivalent substitutions or modifications made to the technical solution and inventive concept disclosed in the present invention within the scope of the technology disclosed in the present invention should be covered within the scope of protection of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A point cloud 3D human pose estimation method based on a geometrically adaptive local coordinate system, characterized in that, Includes the following steps: S1. Local Feature Extraction: Based on the geometrically adaptive local coordinate system module, a dynamic local coordinate system centered on key points is constructed to enhance the ability to capture fine-grained local features in the point cloud and perform local feature extraction. S2. Global Feature Fusion: Based on the dual-branch global feature extraction module, the architecture is dynamically selected according to the point cloud type and task requirements to maintain the consistency between local and global features; local features are fused to generate a global pose representation. S3, Pose Regression: Based on the refinement module, it improves the refinement accuracy of pose estimation by integrating prior knowledge through a self-attention mechanism and utilizing prior knowledge of pose.

2. The point cloud 3D human pose estimation method based on a geometrically adaptive local coordinate system according to claim 1, characterized in that, The geometrically adaptive local coordinate system module in S1 is specifically as follows: Key points are selected from the point cloud by sampling from the farthest point. Neighboring points are grouped using the K-nearest neighbor algorithm. The spatial distribution of the neighboring points is analyzed to generate the normal vector, tangent vector and second tangent vector of the local coordinate system. A learnable bias matrix is ​​introduced to optimize the geometric adaptability of the coordinate system. After the neighborhood points are mapped to the local coordinate system, local features are extracted through a geometric affine module and a one-dimensional convolutional layer. At the same time, relative position loss is used to ensure the geometric consistency of the neighborhood of key points.

3. The point cloud 3D human pose estimation method based on a geometrically adaptive local coordinate system according to claim 1, characterized in that, S1 is specifically as follows: Given a set of points Where N represents a point in the (x,y,z) Cartesian coordinate system, and all points are fed into the feature extraction layer consisting of a geometrically adaptive local coordinate system module and a Mamba module; In each layer, the farthest point sampling is used to resample the N points, and then the K nearest neighbor algorithm is used to find the K neighbors of the N points; The sampled point and its neighbor points are fed together into the eigenvector decomposition module. The product of the differences between the sampled point and its neighbor is summed to obtain the covariance matrix. The covariance matrix is ​​then subjected to eigenvector decomposition to obtain the normal vector, tangent vector, and second tangent vector, as shown below: Among them, C i Represents the covariance matrix; Represents the eigenvector matrix; This represents a diagonal matrix containing the eigenvalues ​​of the covariance matrix; At the same time, a learnable paranoia matrix is ​​introduced. The bias matrix is ​​applied to the three vectors and concatenated into a GAL rotation matrix. Finally, the neighborhood points are mapped to the local coordinate system using the rotation matrix, thus transforming each neighborhood point from the global coordinate system to its corresponding local coordinate system, as shown below: n i =V i [:,0] t1 i =V i [:,1] t2 i =V i [:,0]×V i [:,1] The points in the obtained local coordinate system are fed into the geometric affine module and the convolutional layer to extract local features, as shown below: F final =Conv1d(GAM({p i,j })) Among them, {p i,j |j=1,...,k} represents the k neighboring points of pi, GAM is a geometric affine module used to enhance local features, and Conv1d is a one-dimensional convolution; Relative position loss is used to ensure geometric consistency of the neighborhood of key points.

4. The point cloud 3D human pose estimation method based on a geometrically adaptive local coordinate system according to claim 3, characterized in that, The eigenvector corresponding to the smallest eigenvalue of the covariance matrix is ​​used as the normal vector, and the eigenvector corresponding to the second smallest eigenvalue is used as the tangent vector. The vector obtained by calculating the cross product of the normal vector and the tangent vector is used as the second tangent vector describing the local neighborhood.

5. The point cloud 3D human pose estimation method based on a geometrically adaptive local coordinate system according to claim 2 or 4, characterized in that, The relative position loss Loss position The details are as follows: The neighboring points are represented as {(k,k′)|k,k′=1,...,K}.

6. The point cloud 3D human pose estimation method based on a geometrically adaptive local coordinate system according to claim 1, characterized in that, The dual-branch global feature extraction module in S2 is as follows: The dual-branch global feature extraction module supports both Mamba and PCT-v2 architectures. When the dual-branch global feature extraction module adopts the Mamba architecture, the feature extraction process iteratively executes four stages according to the selected serialization strategy to gradually optimize the feature representation; When the dual-branch global feature extraction module adopts the PCT-v2 architecture, it extracts deep, hierarchical features through four iterative cycles.

7. The point cloud 3D human pose estimation method based on a geometrically adaptive local coordinate system according to claim 1, characterized in that, The refining module in S3 is as follows: The refining module is designed based on radar point cloud grid recovery technology, and the refining module includes two self-attention layers.

8. A point cloud 3D human pose estimation system based on a geometrically adaptive local coordinate system applied to the method of claim 1, characterized in that, include: The geometrically adaptive local coordinate system module is used for local feature extraction. The dual-branch global feature extraction module adopts both the Mamba and PCT-v2 architectures, dynamically selecting the architecture for global feature fusion. The refinement module is used to improve the refinement accuracy of attitude estimation by integrating prior knowledge.

Images