A 3D human pose estimation method, system, device and medium based on successive over-relaxation graph convolutional neural network

Abstract

A 3D human pose estimation method, system, device and medium based on successive over-relaxation graph convolutional neural network, the method comprising: detecting an input image or video using a 2D pose detector to obtain 2D joint coordinates of a human body in the image or video; performing uniform standardization processing on the 2D joint coordinates; constructing a graph adjacency matrix to represent a graph structure model; taking the standardized 2D joint coordinates as initial input features of a graph convolutional neural network, providing initial 2D joint information for the graph convolutional neural network, updating the initial input features by using a successive over-relaxation iteration method combined with the constructed graph adjacency matrix, and finally outputting predicted 3D human poses including 3D coordinates of each joint of the human body by the graph convolutional neural network after SOR loop iteration and update; the system, device and medium are used to implement the method; the present application improves the calculation efficiency of the network, reduces the risk of overfitting, and achieves better robustness.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a 3D human pose estimation method, system, device, and medium based on successive hyperrelaxation graph convolutional neural networks. Background Technology

[0002] 3D human pose estimation technology aims to predict the position of body joints in three-dimensional space from images or video sequences, and is widely used in fields such as action recognition, human-computer interaction, pose-driven, and behavior analysis. With the development of deep learning technology, this field has made significant progress. However, 3D human pose estimation still faces two major challenges: (1) self-occlusion, where one part of the body is occluded by another, making it difficult for the model to accurately estimate the position of the occluded part; and (2) depth ambiguity, where multiple 3D poses may correspond to the same 2D projection in an image due to occlusion, self-occlusion, and changes in body shape. To address these challenges, recent 3D human pose estimation methods focus on improving accuracy and robustness. Traditional 3D human pose estimation methods based on convolutional neural networks typically locate each joint of the human body independently, ignoring the spatial relationships between joints. This can lead to incorrect pose estimation, especially when joints occlude each other.

[0003] To address the occlusion problem, researchers have utilized Graph Convolutional Networks (GCNs) for 3D human pose estimation. Compared to traditional convolutional neural network methods, GCNs model the human skeletal structure as a graph (nodes represent joints, and edges represent relationships between joints), capturing spatial relationships between joints and learning effective pose representations. A key advantage of using GCNs is their ability to capture dependencies between joints. The position of each joint depends on the positions of other joints in the body, and GCNs can explicitly model these dependencies, allowing the model to treat the body as a connected system and make accurate predictions even when some joints are occluded. Vanilla GCN is a basic GCN model that operates directly on the graph, utilizing its structural information to update node features through the product of the adjacency matrix and the feature matrix. While Vanilla GCN has achieved good results in 3D human pose estimation, it still has some drawbacks. First, some studies use shared weight matrices (i.e., all nodes in the graph use the same weight matrix) to determine the importance of adjacent joints. However, shared weight matrices may fail to capture the subtle nuances of complex relationships between human joints, limiting the model's expressive power and leading to suboptimal performance. Secondly, GCNs without skip connections are limited by the receptive field of their iterators, capturing information only from the local neighborhood of each joint in the graph. Skip connections allow information to flow around certain layers in the network, improving the ability to model long-range dependencies. Thirdly, GCNs that only use local information can lead to oversmoothing, where the learned node representations become too similar, especially as network depth increases.

[0004] Patent application CN117894070A discloses a 3D human pose recognition network and method based on feature pyramid fusion. It utilizes a feature pyramid fusion network with a ResNet50 network architecture at its core, while integrating an ECA (Efficient Channel Attention) mechanism to learn the weights of each feature channel. This technology improves the accuracy and efficiency of 3D human pose recognition through its innovative network structure and attention mechanism. However, because the convolutional layers of the ResNet50 network architecture have a fixed receptive field, this limits the network's ability to capture long-distance dependencies. In 3D human pose estimation, the spatial relationships between key points on the human body may span considerable distances. ResNet50 may not be able to effectively model these relationships, leading to insufficient understanding of the overall human pose and resulting in spatial discontinuities in the pose estimation results and less smooth motion trajectories between key points.

[0005] Patent application CN118038561A discloses a 3D human pose estimation method based on a spatiotemporal recurrent architecture. It designs three graph convolutional networks (GCNs) to process graph data, learning the spatial correlations between human joints and addressing the problem of joint self-occlusion. Furthermore, the method introduces a global-local alternation mechanism, cleverly combining self-attention and sliding window mechanisms. This allows the network to focus on local details and global information at different time scales, thus capturing temporal features more comprehensively and resolving the problem of joint depth blur. However, this method uses the same weight matrix for all nodes in the graph, which may cause the model to fail to capture the subtle differences in the complex relationships between human joints, limiting the model's expressive power and resulting in suboptimal performance. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, the present invention aims to provide a 3D human pose estimation method, system, device, and medium based on a successive super-relaxed graph convolutional neural network. Through the designed successive super-relaxed graph convolutional neural network architecture, the 3D human pose estimation can be accurately realized, and the occlusion and depth blur problems in 3D human pose estimation can be effectively solved. The present invention improves the computational efficiency of the network, reduces the risk of overfitting, and achieves better robustness.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A 3D human pose estimation method based on successive hyper-relaxation graph convolutional neural networks includes the following steps:

[0009] Step 1: Use a 2D pose detector to detect the input image or video and obtain the 2D joint coordinates of the human body in the image or video;

[0010] Step 2: Perform uniform standardization processing on the 2D joint coordinates detected in Step 1;

[0011] Step 3: Model the predefined human skeleton structure as a graph structure model. In the graph structure model, each joint of the human body is represented as a node in the graph. The edges connecting these nodes are used to simulate the connection method of the human skeleton. Construct a graph adjacency matrix to represent the graph structure model.

[0012] Step 4: The standardized 2D joint coordinates from Step 2 are used as the initial input features of the graph convolutional neural network to provide initial 2D joint information. The Successive OverRelaxation (SOR) iterative method is used, combined with the graph adjacency matrix constructed in Step 3, to update the initial input features. After SOR iterative updates, the graph convolutional neural network finally outputs the predicted 3D human pose, including the 3D coordinates of each joint of the human body.

[0013] In step 4, the graph convolutional neural network uses a successive super-relaxed graph convolutional neural network (SOR GCN). The SOR GCN includes an input layer, four residual blocks, and an output layer. The input layer consists of a graph convolutional layer (SOR GConv) and a GELU activation function. Each residual block consists of two graph convolutional layers (SOR GConv), where layer normalization is performed after the first SOR GConv, and a GELU activation function follows the second SOR GConv. The output layer includes a nonlocal layer and a graph convolutional layer (SOR GConv). The last graph convolutional layer (SOR GConv) of the successive super-relaxed graph convolutional neural network generates the 3D human pose. Skip connections are implemented between residual blocks. The nonlocal layer is used to capture global or long-distance dependencies between human joints. The SOR iterative update method is implemented within a single graph convolutional layer (SOR GConv) to update the feature representation of each node in the graph. The specific implementation steps are as follows:

[0014] Step 4.1, Feature Transformation: The standardized 2D keypoint coordinates from Step 2 are used as the initial input features and multiplied with the learnable weight matrix in the graph convolutional layer SOR GConv to obtain a new feature representation H. At the same time, a learnable modulation vector is assigned to the feature representation H, and the modulation vector is multiplied with the feature representation H to achieve the modulation operation of the feature representation H.

[0015] Step 4.2, Adjacency Matrix Weight Modulation: Add the graph adjacency matrix from Step 3 to a learnable weight matrix of the same dimension, and perform symmetry adjustment to make the graph adjacency matrix a positive definite matrix, thus realizing the weight modulation process and obtaining the weight-modulated graph adjacency matrix.

[0016] Step 4.3, Graph Adjacency Matrix Triangular decomposition: This involves adjusting the weights of the graph adjacency matrix obtained in step 4.2. Perform upper and lower triangular decomposition, that is D is a diagonal matrix, L is a lower triangular matrix, and U is an upper triangular matrix;

[0017] Step 4.4, SOR iterative algorithm propagation update: using the graph adjacency matrix after triangular decomposition in step 4.3. The feature representation H obtained in step 4.1 is propagated and updated using the SOR iterative algorithm. In the SOR iterative algorithm, given the feature representation H of the l-th layer... (l) The feature representation H of the next layer after the layer propagation rule (l+1) Represented as:

[0018] H (l+1) =B ω H (l) +F ω (4)

[0019] B ω =(D-ωL) -1 ((1-ω)D+ωU) (5)

[0020] F ω =ω(D-ωL) -1 X (6)

[0021] In the formula, B ω F is the SOR iteration matrix; ω The residual part; D, L, U are the adjacency matrices. The decomposition form of X; X is the input feature representation in the graph convolutional layer SOR GCN in the input layer, the input feature representation in the residual block of the first graph convolutional layer SOR GCN, and the output feature representation in the graph convolutional layer SOR GCN of the output layer.

[0022] Step 4.5, Output the updated result: For the updated feature representation H from step 4.4... (l+1) Add a bias term and output the iterative update result, which is the feature representation of human joints; if the current graph convolution is not the output layer SOR GConv, it is used as the input of the next graph convolution layer SOR GConv in the SOR iterative algorithm; if the current graph convolution is the output layer SOR GConv, the output feature representation is the human 3D joint.

[0023] This invention also provides a 3D human pose estimation system based on a successive super-relaxed graph convolutional neural network, comprising:

[0024] The Human Body 2D Joint Coordinate Acquisition Module is used to detect input images or videos using a 2D pose detector and acquire the 2D joint coordinates of the human body in the images or videos.

[0025] The 2D joint coordinate standardization processing module is used to perform uniform standardization processing on the detected 2D joint coordinates.

[0026] The graph adjacency matrix construction module is used to model the predefined human skeleton structure as a graph structure model. In the graph structure model, each joint of the human body is represented as a node in the graph. The edges connecting these nodes simulate the connection method of the human skeleton. The graph adjacency matrix is ​​constructed to represent the graph structure model.

[0027] The 3D human pose output module uses standardized 2D joint coordinates as the initial input features of the graph convolutional neural network (GCNN), providing initial 2D joint information. It employs a successive over relaxation (SOR) iterative method, combined with a constructed graph adjacency matrix, to update the initial input features. After iterative SOR updates, the GCNN finally outputs the predicted 3D human pose, including the 3D coordinates of each joint.

[0028] This invention also provides a 3D human pose estimation device based on a successive super-relaxation graph convolutional neural network, comprising:

[0029] Memory: A computer program that stores the above-mentioned 3D human pose estimation method based on successive hyperrelaxation graph convolutional neural network, and is a computer-readable device;

[0030] Processor: Used to implement the 3D human pose estimation method based on successive super-relaxation graph convolutional neural network when executing the computer program.

[0031] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the aforementioned 3D human pose estimation method based on a successive hyper-relaxation graph convolutional neural network.

[0032] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0033] 1. This invention is based on the Successive Over Relaxation (SOR) iterative method and designs a Successive Over Relaxation Graph Convolutional Neural Network (SOR GCN). The Successive Over Relaxation Graph Convolutional Neural Network (SOR GCN) can propagate information more effectively. Even if some joints are occluded, it can still make inferences through the update results of adjacent joints, thereby estimating the position of the occluded joints more accurately.

[0034] 2. This invention uses a graph adjacency matrix weight modulation strategy to capture the subtle differences in the complex relationships between human joints, thereby improving the model's expressive power and estimation performance.

[0035] 3. The present invention introduces a nonlocal layer, which enables the model to capture the global or long-distance dependencies between human joints, thereby better understanding the overall human posture and solving the depth ambiguity problem, that is, multiple 3D postures correspond to the same 2D projection in the image.

[0036] In summary, this invention, through the design of a Successively Relaxed Graph Convolutional Neural Network (SOR GCN), the introduction of a graph adjacency matrix weight modulation strategy, and nonlocal layers, achieves technical effects such as more effective information propagation, improved model expressiveness, and capture of global or long-distance dependencies between human joints. These innovations enable this invention to more accurately estimate the position of occluded joints, improve estimation results, and solve the depth ambiguity problem, thereby leading to a better understanding of the overall human posture. Attached Figure Description

[0037] Figure 1 This is a diagram of the structure of the Successive Over-Relaxed Graph Convolutional Neural Network (SOR GCN) designed in this invention.

[0038] Figure 2 These are experimental results using this method in a specific implementation; where, Figure 2 (a) is the input 2D visualization of human joints. Figure 2 (b) is a visualization of the predicted 3D human joints of this invention. Figure 2 (c) is a 3D visualization of the joints of a real human body. Detailed Implementation

[0039] The specific usage of this invention will be described below with reference to the accompanying drawings to help users better understand the invention. It should be noted that detailed descriptions of existing functions and methods will be simplified in the following description. This invention primarily focuses on 3D human pose estimation methods.

[0040] Unlike previous methods, this invention is inspired by the efficiency of the SOR iterative method in solving large sparse matrix equations, and combines the basic principles of Vanilla GCN update iteration with the sparsity of the adjacency matrix in human pose estimation. Through this combination, this method effectively solves the occlusion and depth blur problems in 3D human pose estimation, and also provides an effective solution to the overfitting and convergence difficulties encountered by graph convolutional networks in deep network human pose estimation.

[0041] like Figure 1 As shown, a 3D human pose estimation method based on a successive super-relaxation graph convolutional neural network includes the following steps:

[0042] Step 1: Use a 2D pose detector to detect the input image or video and obtain the 2D joint coordinates of the human body in the image or video;

[0043] In step 1, the 2D pose detector is a Cascaded Pyramid Network (CPN) model. The 2D image or video is used as input to the 2D pose detector to obtain the 2D joint coordinates of the human body in the image or video, which are then used in the subsequent 3D pose estimation steps.

[0044] Step 2: Perform uniform standardization processing on the 2D joint coordinates detected in Step 1 for use in subsequent training steps;

[0045] The standardization process includes coordinate transformation, normalization, data segmentation, and data augmentation operations.

[0046] Step 2.1, Coordinate Transformation: Transform the 2D joint coordinates detected in Step 1 from the image coordinate system to a unified coordinate system;

[0047] Step 2.2, Normalization: Normalize the coordinates of key points, and center the human body posture by taking the center of the human body (hip joint) as the origin.

[0048] Step 2.3, Data Segmentation: Segment the data into different subjects and poses according to task requirements, and downsample the number of frames in the dataset, selecting only a portion of the frames for training;

[0049] Step 2.4, Data Augmentation: Increase the diversity of data through rotation, translation, scaling, etc., thereby enhancing the robustness of the model under different conditions.

[0050] Step 3: Model the predefined human skeleton structure as a graph structure model. In the graph structure model, each joint of the human body is represented as a node in the graph. The edges connecting these nodes simulate the connection mode of the human skeleton. Construct a graph adjacency matrix to represent the graph structure model. The graph adjacency matrix reflects in detail the relationships and connection characteristics of the human skeleton structure.

[0051] The specific method for constructing the graph adjacency matrix in step 3 is as follows:

[0052] Step 3.1: Using predefined human skeleton information as input, the predefined human skeleton information generally includes the number of joints and the relationship between nodes. By traversing the joint information of the skeleton, the connection relationship between the human bone joints is extracted to form a list of edges.

[0053] Step 3.2: Convert the edge list information obtained in Step 3.1 into a sparse matrix. The rows and columns of the sparse matrix correspond to the joint numbers, and the values ​​of the sparse matrix indicate whether there is a connection between the corresponding joints. Then, based on the edge connection situation, convert the adjacency matrix into a symmetric matrix to reflect the bidirectional connection between the skeletal joints.

[0054] Step 3.3: Normalize the symmetric matrix transformed in Step 3.2 to ensure the stability and numerical range of the graph adjacency matrix, and finally construct the graph adjacency matrix.

[0055] Step 4: The standardized 2D joint coordinates from Step 2 are used as the initial input features of the graph convolutional neural network (Graph Convolutional Neural Network). This provides the Graph Convolutional Neural Network with initial 2D joint information. A Successive OverRelaxation (SOR) iterative method is employed, combined with the graph adjacency matrix constructed in Step 3, to update the initial input features. After iterative SOR updates, the Graph Convolutional Neural Network finally outputs the predicted 3D human pose, including the 3D coordinates of each joint. This achieves accurate estimation of the human pose in the input image or video.

[0056] In step 4, the graph convolutional neural network employs a successive super-relaxed graph convolutional neural network (SOR GCN). The SOR GCN comprises an input layer, four residual blocks, and an output layer. The input layer consists of a graph convolutional layer (SOR GConv) and a GELU activation function. Each residual block consists of two graph convolutional layers (SOR GConv), where layer normalization is performed after the first SOR GConv, and a GELU activation function follows the second SOR GConv. The output layer contains a nonlocal layer and a graph convolutional layer (SOR GConv). The last graph convolutional layer (SOR GConv) of the SOR GCN generates the 3D human pose. Skip connections are implemented between residual blocks, improving the ability to model long-range dependencies. The nonlocal layer is used to capture global or long-range dependencies between human joints. The nonlocal layer's role is to capture long-range dependencies through nonlocal operations, preventing over-smoothing caused by GCN using only local information. A single graph convolutional layer (SOR GCN)... GConv implements the SOR iterative update method to update the feature representation of each node in the graph. The specific implementation steps are as follows:

[0057] Step 4.1, Feature Transformation: The standardized 2D joint coordinates from Step 2 are used as the initial input features and multiplied with the learnable weight matrix in the graph convolutional layer SOR GConv to obtain a new feature representation H. At the same time, a learnable modulation vector is assigned to the feature representation H, and the modulation vector is multiplied with the feature representation H to achieve the modulation operation of the feature representation H, thereby better capturing the local structure of different joints in the human body.

[0058] Step 4.2, Adjacency Matrix Weight Modulation: Add the graph adjacency matrix from Step 3 to a learnable weight matrix of the same dimension, and perform symmetry adjustment to make the graph adjacency matrix a positive definite matrix, thus realizing the weight modulation process and obtaining the weight-modulated graph adjacency matrix.

[0059] Step 4.3, Graph Adjacency Matrix Triangular decomposition: This involves adjusting the weights of the graph adjacency matrix obtained in step 4.2. Perform upper and lower triangular decomposition, that is D is a diagonal matrix, L is a lower triangular matrix, and U is an upper triangular matrix;

[0060] Step 4.4, SOR iterative algorithm propagation update: using the graph adjacency matrix after triangular decomposition in step 4.3. The feature representation H obtained in step 4.1 is propagated and updated using the SOR iterative algorithm. In the SOR iterative algorithm, given the feature representation H of the l-th layer... (l) The feature representation H of the next layer after the layer propagation rule (l+1) Represented as:

[0061] H (l+1) =B ω H (l) +F ω (4)

[0062] B ω =(D-ωL) -1 ((1-ω)D+ωU) (5)

[0063] F ω =ω(D-ωL) -1 X (6)

[0064] In the formula, B ω F is the SOR iteration matrix; ω The residual part; D, L, U are the adjacency matrices. The decomposition form of X; X is the input feature representation in the graph convolutional layer SOR GCN in the input layer, the input feature representation in the residual block of the first graph convolutional layer SOR GCN, and the output feature representation in the graph convolutional layer SOR GCN of the output layer.

[0065] Step 4.5, Output the updated result: For the updated feature representation H from step 4.4... (l+1) Add a bias term and output the iterative update result, which is the feature representation of human joints; if the current graph convolution is not the output layer SOR GConv, it is used as the input of the next graph convolution layer SOR GConv in the SOR iterative algorithm; if the current graph convolution is the output layer SOR GConv, the output feature representation is the human 3D joint.

[0066] A 3D human pose estimation method based on successive super-relaxed graph convolutional neural network was developed, and a network model was constructed for 3D human pose estimation applications.

[0067] The experimental analysis is as follows:

[0068] like Figure 2 As shown, Figure 2 (a) shows the input 2D human joints. Figure 2 (c) represents the actual 3D coordinates of human joints in the dataset, while Figure 2 (b) shows the 3D coordinates of human joints predicted using this invention. It can be seen that the pose estimation results of this invention are very accurate.

[0069] Furthermore, as shown in Table 1, this invention achieves an MPJPE value of 35.75 and a PA-MPJPE value of 27.51 in 3D human pose estimation. Compared with mainstream graph convolutional network methods such as Sem GCN, High-order GCN, Modulated GCN, and GSNet GCN, this invention achieves superior performance in 3D human pose estimation, demonstrating significant advantages in estimation accuracy and smaller errors, thus proving its high accuracy in 3D human pose estimation.

[0070] Table 1 compares the experimental results. Each row of data represents the average MPJPE and PA-MPJPE of different methods under 15 actions. The smaller the result value of the two indicators, the better the effect.

[0071]

[0072] This invention leverages the effectiveness of successive over-relaxation iteration in solving large sparse linear systems, combining the fundamental principles of Vanilla GCN update iteration with the sparsity of the adjacency matrix in human pose estimation. Compared to other iterative methods, such as Jacobi iteration and Gauss-Seidel iteration, the SOR iteration introduces a relaxation factor. By updating node features layer by layer and adjusting the update step size, it helps alleviate the gradient vanishing or exploding problem in deep networks, making deep network training convergence more stable. Simultaneously, through the characteristics of graph convolution algorithms and the concepts of weight sharing and skip connections, this invention effectively solves the occlusion and depth blur problems in 3D human pose estimation. Experimental results demonstrate that compared to most graph convolution 3D human pose estimation methods, this method achieves better results with smaller errors compared to the true values.

[0073] This invention also provides a 3D human pose estimation system based on a successive super-relaxed graph convolutional neural network, comprising:

[0074] The human body 2D joint coordinate acquisition module is used to detect the input image or video using a 2D pose detector in step 1 and obtain the 2D joint coordinates of the human body in the image or video.

[0075] The 2D joint coordinate standardization processing module is used to perform unified standardization processing on the 2D joint coordinates detected in step 1 in step 2.

[0076] The graph adjacency matrix construction module is used to model the predefined human skeleton structure as a graph structure model in step 3. In the graph structure model, each joint of the human body is represented as a node in the graph, and the edges connecting these nodes are used to simulate the connection method of the human skeleton. The graph adjacency matrix is ​​constructed to represent the graph structure model.

[0077] The 3D human pose output module is used to implement step 4, which uses the standardized 2D joint coordinates from step 2 as the initial input features of the graph convolutional neural network, providing the graph convolutional neural network with initial 2D joint information. The module uses the successive over relaxation (SOR) iterative method, combined with the graph adjacency matrix constructed in step 3, to update the initial input features. After the SOR iterative update, the graph convolutional neural network finally outputs the predicted 3D human pose, including the 3D coordinates of each joint of the human body.

[0078] This invention also provides a 3D human pose estimation device based on a successive super-relaxation graph convolutional neural network, comprising:

[0079] Memory: A computer program that stores the above-mentioned 3D human pose estimation method based on successive hyperrelaxation graph convolutional neural network, and is a computer-readable device;

[0080] Processor: Used to implement the 3D human pose estimation method based on successive super-relaxation graph convolutional neural network when executing the computer program.

[0081] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the aforementioned 3D human pose estimation method based on a successive hyper-relaxation graph convolutional neural network.

[0082] Although the illustrative specific embodiments of the present invention have been described above to enable those skilled in the art to understand the invention, it should be understood that the invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes will be obvious as long as they are within the spirit and scope of the invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are within the scope of protection of the present invention.

Claims

1. A 3D human pose estimation method based on successive over-relaxation graph convolutional neural network, characterized in that, The method comprises the following steps: Step 1, detecting an input image or video using a 2D pose detector to obtain 2D joint coordinates of a human body in the image or video; Step 2, performing uniform standardization processing on the 2D joint coordinates detected in step 1; Step 3, modeling a predefined human skeleton structure into a graph structure model, in which each joint of the human body is represented as a node in the graph, and edges connecting the nodes are used to simulate the connection mode of the human skeleton, and a graph adjacency matrix is constructed to represent the graph structure model; Step 4, taking the 2D joint coordinates standardized in step 2 as initial input features of a graph convolutional neural network, providing initial 2D joint information for the graph convolutional neural network, and updating the initial input features by using a successive overrelaxation (SOR) iteration method in combination with the graph adjacency matrix constructed in step 3, and finally outputting, by the graph convolutional neural network, a predicted 3D human pose including 3D coordinates of each joint of the human body after SOR loop iteration and update; The graph convolutional neural network in step 4 is a successive overrelaxation graph convolutional neural network (SOR GCN), which comprises an input layer, four residual blocks and an output layer; the input layer is composed of a graph convolutional layer SOR GConv and a GELU activation function; each residual block is composed of two graph convolutional layers SOR GConv, wherein layer normalization is performed after the first graph convolutional layer SOR GConv, and a GELU activation function is arranged after the second graph convolutional layer SOR GConv; the output layer comprises a non-local layer and a graph convolutional layer SOR GConv, and the last graph convolutional layer SOR GConv of the successive overrelaxation graph convolutional neural network generates a 3D human pose; the residual blocks are connected by a skip connection; the non-local layer is used to capture global or long-distance dependency relationships between human joints; and the SOR iteration update method is implemented in a single graph convolutional layer SOR GConv to update the feature representation of each node in the graph, and the specific implementation steps are as follows: Step 4.1, feature conversion: multiplying the 2D joint coordinates standardized in step 2 as initial input features by a learnable weight matrix in the graph convolutional layer SOR GConv to obtain a new feature representation H, assigning a learnable modulation vector to the feature representation H, and multiplying the modulation vector by the feature representation H to realize a modulation operation on the feature representation H; Step 4.2, Adjacency matrix weight modulation: add the graph adjacency matrix in step 3 with a learnable weight matrix of the same dimension, and perform symmetrization adjustment to make the graph adjacency matrix a positive definite matrix, realize the weight modulation process, and obtain the graph adjacency matrix after weight modulation Step 4.3, graph adjacency matrix Triangular factorization: on the weight-modulated graph adjacency matrix of step 4.2 Perform upper and lower triangular factorization, i.e. D is a diagonal matrix, L is a lower triangular matrix, and U is an upper triangular matrix; Step 4.

4. SOR iterative algorithm propagates updates: Use the triangular factored graph adjacency matrix from Step 4.3 The SOR iterative algorithm propagates updates to the feature representation H obtained in Step 4.1, where the SOR iterative algorithm is given the feature representation Hl of the lth layer The feature representation Hl+1 of the next layer is propagated through the layer propagation rule is represented as: B ω = (D - ωL) -1 ((1 - ω)D + ωU) (5) F ω = ω (D - ωL) -1 X (6) wherein B ω is the SOR iteration matrix; F ω is the residual part; D, L, U are the decomposition of the adjacency matrix is the input feature representation in the input layer, is the input feature representation of the first graph convolutional layer SOR GCN in the residual block, and is the feature representation output by the fourth residual block in the graph convolutional layer SOR GCN of the output layer. Step 4.5, output the updated result: the feature representation of the human body joints add a bias term, output the iterative updated result, i.e., the feature representation of the human body joints; if the current graph convolution is not the graph convolution layer SOR GConv of the output layer, then the feature representation is taken as the input of the next graph convolution layer SOR GConv in the SOR iterative algorithm; if the current graph convolution layer is the graph convolution layer SOR GConv of the output layer, then the output feature representation is the 3D human body joint.

2. A 3D human pose estimation system based on successive over-relaxation graph convolutional neural network based on the method of claim 1, characterized in that, The method comprises the following steps: A human body 2D joint coordinate acquisition module is configured to detect an input image or video using a 2D pose detector to obtain 2D joint coordinates of a human body in the image or video; A 2D joint coordinate standardization processing module is configured to perform uniform standardization processing on the detected 2D joint coordinates; A graph adjacency matrix construction module is configured to model a predefined human skeleton structure into a graph structure model, in which each joint of the human body is represented as a node in the graph, and edges connecting the nodes are used to simulate the connection mode of the human skeleton, and a graph adjacency matrix is constructed to represent the graph structure model; 3D human pose output module, configured to take the standardized 2D joint coordinates as initial input features of the graph convolutional neural network, provide initial 2D joint information for the graph convolutional neural network, update the initial input features by using a successive over relaxation (SOR) iteration method in combination with the constructed graph adjacency matrix, and finally output predicted 3D human poses including 3D coordinates of each joint of the human body.

3. A 3D human pose estimation device based on successive over-relaxation graph convolutional neural network, characterized in that, The method comprises the following steps: a memory configured to store a computer program of the 3D human pose estimation method based on the successive over relaxation graph convolutional neural network according to claim 1, and the computer program is readable by a computer; a processor configured to implement the 3D human pose estimation method based on the successive over relaxation graph convolutional neural network according to claim 1 when the computer program is executed.

4. A computer-readable storage medium, characterized in that, A computer readable storage medium stores a computer program, and the computer program can implement the 3D human pose estimation method based on the successive over relaxation graph convolutional neural network according to claim 1 when the computer program is executed by a processor.

Images