A three-dimensional human body pose estimation method, system, device and medium

Abstract

The present invention discloses a three-dimensional human body pose estimation method, system, device and medium. The construction method includes: performing a coarsening operation on the graph topological structure of the human body's dense surface to obtain graph topological structures at different coarsening levels, and this coarsened graph topological structure is denser than the human body skeleton graph topological structure; combining the human body skeleton graph topological structure, constructing a graph convolutional neural network with multiple parallel branches from sparse to dense in sequence; connecting different branches through a multi-scale feature fusion module to realize information interaction between branches, and constructing a hierarchical graph convolutional neural network. The three-dimensional human body pose estimation system includes: a construction unit of the hierarchical graph convolutional network, a training unit of the graph convolutional neural network, and a three-dimensional human body pose estimation unit. Through the present invention, the number of parameters of the network model can be saved, and the accuracy of pose estimation for poses with self-occlusion and complex actions can be significantly improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method and system for constructing a graph convolutional network for three-dimensional human pose estimation, as well as a three-dimensional human pose estimation system, device and medium. Background Technique

[0002] Three-dimensional human pose estimation aims to predict the three-dimensional spatial positions of human joint points through a single monocular image. As a hot topic in the field of computer vision, three-dimensional human pose estimation has broad research prospects and is also widely applied in various fields, such as abnormal behavior detection, behavior recognition, etc. In recent years, the performance of two-dimensional human pose estimation has been greatly improved. Therefore, more and more work uses the two-dimensional coordinates of the estimated joint points to regress their corresponding three-dimensional coordinates. Since the human skeleton can be regarded as a graph topology structure, a graph convolutional neural network can be used to model the relationships between its joint points. The graph convolutional neural network simulates the convolution operation in the traditional convolutional neural network by designing an effective similarity metric or using spectral graph convolution, corresponding to the spatial domain and spectral domain graph convolutional neural networks respectively. They combine the representation ability of the graph structure for the human skeleton and the powerful feature learning ability of the convolutional network for high-dimensional signals, and have achieved remarkable results in tasks such as three-dimensional human pose estimation.

[0003] After a literature search of the existing technology, it is found that the main difficulty in the existing three-dimensional human pose estimation task based on the graph convolutional neural network is the difficulty in overcoming the non-uniqueness of the two-dimensional to three-dimensional mapping. Therefore, this problem is an ill-posed problem. When modeling the graph neural network in the existing method, the graph topology structure usually adopted is the human skeleton topology, and this graph topology structure has a relatively sparse number of nodes (such as 17), and this sparsity cannot well model the locality of the human skeleton, resulting in the above ill-posed problem becoming more serious. Therefore, in the face of scenes where human joint points are occluded and some complex actions, it is often difficult to estimate accurate three-dimensional joint point coordinates. Summary of the Invention

[0004] In view of the above problems existing in the prior art, the present invention proposes a three-dimensional human pose estimation method, system, device and medium, which saves the workload and time of artificially designing parameters, better captures localized information, can significantly improve the accuracy of pose estimation containing self-occlusion and complex actions, and reduces the number of parameters of the model.

[0005] To solve the above technical problems, the present invention is realized by the following technical solutions:

[0006] In the first aspect of the present invention, a network construction method for three-dimensional human pose estimation is provided, including:

[0007] S11: Coarsen the graph topology of the dense human surface to obtain graph topologies at different coarsening levels;

[0008] S12: According to the coarsened human surface graph topology obtained in S11, combined with the human skeleton graph topology, construct a graph convolutional neural network structure with multiple parallel branches from sparse to dense in sequence;

[0009] S13: According to the multi-parallel branch graph convolutional network structure obtained in S12, connect different branches through a multi-scale feature fusion module to realize information interaction between branches, and construct a hierarchical graph convolutional neural network.

[0010] Preferably, S11 further includes: In the coarsening stage, merge some graph nodes together to obtain the next-level coarsened graph, and repeat this process until the coarsened graph is small enough.

[0011] Preferably, S12 further includes:

[0012] The human surface graph topologies at different coarsening levels obtained according to S11 Select the two graph topologies with the highest coarsening level and the human skeleton graph topology g P , and construct three parallel graph convolutional neural network branches with different graph topologies from sparse to dense in sequence according to the number of nodes in the graph topology.

[0013] Preferably, the graph convolutional neural network branches in S12 further include multiple residual graph convolutional modules:

[0014] According to the coarsening degree in the graph topologies of the three parallel graph convolutional neural network branches in S12, allocate different numbers of residual graph convolutional modules to each graph convolutional neural network branch. Among them, the residual graph convolutional module includes two layers of graph convolutional layers, two layers of non-linear activation layers, two layers of batch normalization layers, and one layer of non-local module layer.

[0015] Preferably, S13 further includes: The multi-scale feature fusion module models the mapping relationship between nodes of different graph topologies through 1x1 convolution, so as to realize information interaction.

[0016] In the second aspect of the present invention, a graph convolutional network construction system for three-dimensional pose estimation is provided, including: a graph coarsening unit, a multi-branch graph convolutional neural network structure construction unit, and a hierarchical graph convolutional neural network construction unit; wherein,

[0017] The graph coarsening unit coarsens the graph topology of the dense human surface to obtain graph topologies at different coarsening levels;

[0018] The multi-branch graph convolutional neural network structure construction unit constructs a graph convolutional neural network structure with multiple parallel branches in sequence from sparse to dense according to the topological structure of the human coarsened surface graph obtained by the graph coarsening unit and in combination with the topological structure of the human skeleton graph;

[0019] The hierarchical graph convolutional neural network construction unit, based on the multi-parallel branch graph convolutional network structure obtained by the multi-branch graph convolutional neural network structure construction unit, connects different branches through a multi-scale feature fusion module to realize information interaction between branches and constructs a hierarchical graph convolutional neural network.

[0020] In the third aspect of the present invention, a three-dimensional human pose estimation method is provided, including:

[0021] Using the hierarchical graph convolutional neural network obtained above;

[0022] Training the hierarchical graph convolutional neural network;

[0023] Using the trained graph convolutional neural network for three-dimensional human pose estimation.

[0024] In the fourth aspect of the present invention, a three-dimensional human pose estimation system is provided, including: a construction unit of a graph convolutional neural network, a training unit of a graph convolutional neural network, and a three-dimensional human pose estimation unit; wherein,

[0025] The construction unit of the graph convolutional neural network uses the hierarchical graph convolutional neural network constructed above;

[0026] The training unit of the graph convolutional neural network is used to train the graph convolutional neural network constructed by the construction unit of the graph convolutional neural network;

[0027] The three-dimensional human pose estimation is used to perform three-dimensional human pose estimation using the graph convolutional neural network trained by the training unit of the graph convolutional neural network.

[0028] In the fifth aspect of the present invention, a three-dimensional human pose estimation device is provided, including: a memory and a processor; wherein,

[0029] The memory is used to store a computer program;

[0030] The processor is used to execute the computer program to implement the network construction method for three-dimensional human pose estimation as described above, or to implement the network construction unit for three-dimensional human pose estimation as described above, or to implement the three-dimensional human pose estimation system as described above.

[0031] In a sixth aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, it is used to execute the method for constructing a graph convolutional network for three-dimensional pose estimation or the three-dimensional human pose estimation method described above.

[0032] Compared with the prior art, the embodiments of the present invention have at least one of the following beneficial effects:

[0033] (1) The network construction method, system, three-dimensional human pose estimation system, and device for three-dimensional human pose estimation provided by the present invention can better capture local information by introducing a graph topology structure that is denser than the human skeleton.

[0034] (2) The network construction method, system, three-dimensional human pose estimation system, and device for three-dimensional human pose estimation provided by the present invention can extract features from input data more compactly through the network construction method in S13, saving the number of network model parameters.

[0035] (3) The network construction method, system, three-dimensional human pose estimation system, and device for three-dimensional human pose estimation provided by the present invention adopt a network construction algorithm that is independent of the specific graph convolution method. Therefore, it can be conveniently embedded into various existing graph convolution methods, significantly improving the accuracy of pose estimation for self-occluded and complex actions, and having strong flexibility and scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The following further describes the embodiments of the present invention with reference to the drawings:

[0037] Figure 1 It is a flowchart of the network construction method for three-dimensional human pose estimation according to an embodiment of the present invention;

[0038] Figure 2 It is a schematic structural diagram of the network construction system for three-dimensional human pose estimation according to an embodiment of the present invention;

[0039] Figure 3 It is a schematic structural diagram of three-dimensional human pose estimation according to an embodiment of the present invention;

[0040] Figure 4 It is a schematic diagram of three-dimensional human pose estimation in a specific example of the present invention;

[0041] Reference numerals: 11 - graph coarsening unit, 12 - multi-branch graph convolutional neural network structure construction unit, 13 - hierarchical graph convolutional neural network construction unit

[0042] 21 - graph convolutional neural network construction unit, 22 - graph convolutional neural network training unit, 23 - three-dimensional human pose estimation unit. Detailed implementation manners

[0043] The embodiments of the present invention will be described in detail below. These embodiments are implemented on the premise of the technical solution of the present invention, and detailed implementation manners and specific operation processes are given. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several deformations and improvements can be made, and these all belong to the protection scope of the present invention. The parts not described in detail in the following embodiments can be implemented by using the prior art.

[0044] As Figure 1 shown, it is a flowchart of a network construction method for three-dimensional human pose estimation according to an embodiment of the present invention.

[0045] Please refer to Figure 1 , the network construction method for three-dimensional human pose estimation in this embodiment includes the following steps:

[0046] S11: By performing a coarsening operation on the graph topological structure of the human dense surface, graph topological structures at different coarsening levels are obtained;

[0047] S12: According to the graph topological structure of the coarsened human surface obtained in S11, combined with the graph topological structure of the human skeleton, a graph convolutional neural network structure with multiple parallel branches is constructed sequentially from sparse to dense;

[0048] S13: According to the multi-parallel branch graph convolutional network structure obtained in S12, different branches are connected through a multi-scale feature fusion module to achieve information interaction between branches, and a hierarchical graph convolutional neural network is constructed.

[0049] In a preferred embodiment, S11 further includes:

[0050] S111: Obtain the graph topological structure of the human dense surface according to the human surface model

[0051] S112: Examine the weight relationship between the nodes of the graph topological structure, and merge some graph nodes together to obtain a coarsened graph;

[0052] S113: Repeat the above operations until the coarsened graph is small enough, and finally obtain graph topological structures at different coarsening levels

[0053] In a preferred embodiment, S12 further includes:

[0054] S121: According to the graph topological structures of the coarsened human surfaces at different levels obtained in S11 Select the two graph topologies with the highest coarsening level and the graph topological structure g of the human skeleton P, as the required three-graph topology structure.

[0055] S122: The residual graph convolutional module is constructed by two layers of graph convolutional layers, two layers of non-linear activation layers, two layers of batch normalization layers, and one layer of non-local module layer, and is arranged in series as graph convolution, non-linear activation, batch normalization, graph convolution, non-linear activation, batch normalization, and non-local module;

[0056] S123: According to the number of nodes in the graph topology structure, from sparse to dense, three parallel graph convolutional neural network branches are constructed in sequence, and each branch has a different graph topology structure and a different number of residual graph convolutional modules.

[0057] In a preferred embodiment, S13 further includes: The multi-scale feature fusion module models the mapping relationship between nodes of different graph topology structures through 1x1 convolution, so as to realize information interaction. Specifically:

[0058] Given the features X0, X1, …, X corresponding to different input graph topology structures s , where the subscript represents the scale of the graph topology structure, and the features Y corresponding to each graph topology structure after fusion and output are obtained j :

[0059]

[0060] where a represents 1x1 convolution, and i and j represent the scales corresponding to the input and output features respectively.

[0061] As Figure 2 shown, it is a schematic structural diagram of the network construction system for three-dimensional human pose estimation according to an embodiment of the present invention.

[0062] Please refer to Figure 2 , the network construction method system for three-dimensional human pose estimation in this embodiment includes: a graph coarsening unit 11, a multi-branch graph convolutional neural network structure construction unit 12, and a hierarchical graph convolutional neural network construction unit 13. Among them:

[0063] The graph coarsening unit 11 is used to coarsen the graph topology structure of the dense surface of the human body to obtain graph topology structures at different coarsening levels; the function of the graph coarsening unit is to obtain graph topology structures at different levels, and this graph topology structure is denser than the commonly used human skeleton structure, which can enable more local information to be obtained.

[0064] The multi-branch graph convolutional neural network structure construction unit 12 is used for the topological structure of the coarsened human body surface graph obtained by the graph coarsening unit S11, and combines the topological structure of the human body skeleton graph to sequentially construct a graph convolutional neural network structure with multiple parallel branches from sparse to dense. The multi-branch graph convolutional neural network structure construction unit lies in that different branches have graph topological structures of different scales for capturing the structural information of the human body at different scales.

[0065] The hierarchical graph convolutional neural network construction unit 13 is used to connect different branches through a multi-scale feature fusion module according to the multi-parallel branch graph convolutional network structure obtained by the multi-branch graph convolutional neural network structure construction unit 12, realize the information interaction between branches, and construct a hierarchical graph convolutional neural network.

[0066] As Figure 3 shown, it is a schematic structural diagram of a three-dimensional human body pose estimation system according to an embodiment of the present invention.

[0067] Please refer to Figure 3 , the three-dimensional human body pose estimation system of this embodiment includes: a graph convolutional neural network construction unit 21, a graph convolutional neural network training unit 22, and a three-dimensional human body pose estimation unit 23. Among them, the graph convolutional neural network construction unit 21 constructs a graph convolutional neural network by using the method for constructing a three-dimensional human body pose estimation network in the above embodiment; or constructs a graph convolutional neural network by using the system for constructing a three-dimensional human body pose estimation network in the above embodiment. The graph convolutional neural network training unit 22 is used to train the graph convolutional neural network constructed by the graph convolutional neural network construction unit. The three-dimensional human body pose estimation unit 23 is used to perform three-dimensional human body pose estimation by using the graph convolutional neural network trained by the graph convolutional neural network training unit 22.

[0068] Based on the same technical communication above, an embodiment of the present invention also provides a three-dimensional human body pose estimation method, which features including: constructing a hierarchical graph convolutional neural network by using the method for constructing a graph convolutional neural network for three-dimensional human body pose estimation; training the hierarchical graph convolutional neural network; performing three-dimensional human body pose estimation by using the trained graph convolutional neural network.

[0069] To better understand the above parts, a detailed application example is provided below for illustration. This embodiment provides a three-dimensional human pose estimation system, which adopts the above hierarchical graph convolutional neural network construction method. This embodiment is executed in parallel on a batch (64) of input two-dimensional human joint points. Each group of two-dimensional human joint points includes multiple points, and each point contains its two-dimensional coordinate information in the image coordinate system. Three-dimensional human pose estimation obtains the three-dimensional coordinate information corresponding to each joint point through a graph convolutional neural network. To clearly describe the execution process, the following takes the human skeleton topology with 17 nodes and the human dense surface graph topology with 6890 nodes as examples to specifically illustrate the network structure construction method for three-dimensional human pose estimation.

[0070] 1. Graph coarsening unit

[0071] The main function of the graph coarsening unit is to obtain graph topologies at different levels, and this graph topology is denser than the commonly used human skeleton structure, enabling more local information to be obtained. The graph coarsening unit obtains the graph topology of the human dense surface with 6890 nodes according to the human surface model. Examine the weight relationship between the nodes of the graph topology, merge some graph nodes together to obtain a coarsened graph; repeat the above operation until the coarsened graph is small enough, and finally obtain graph topologies at different coarsening levels.

[0072] 2. Multi-branch graph convolutional neural network structure construction unit

[0073] The multi-branch graph convolutional neural network structure construction unit 12 is used for the human coarsened surface graph topology obtained by the graph coarsening unit S11, and combines the human skeleton graph topology to sequentially construct a graph convolutional neural network structure with multiple parallel branches from sparse to dense. The specific steps are as follows:

[0074] Step 1, according to the human surface graph topologies at different coarsening levels obtained by the graph coarsening unit Select the two graph topologies with the highest coarsening level and the human skeleton graph topology g P , as the three required graph topologies, with the number of nodes being 96, 48, and 17 respectively.

[0075] Step 2, construct a residual graph convolutional module through two layers of graph convolutional layers, two layers of non-linear activation layers, two layers of batch normalization layers, and one layer of non-local module layer;

[0076] Step 3: According to the number of nodes in the three graph topological structures obtained in Step 1, construct three parallel graph convolutional neural network branches in order from sparse to dense. Each branch has a different graph topology and a different number of residual graph convolutional modules.

[0077] 3. Hierarchical Graph Convolutional Neural Network Construction Unit

[0078] The hierarchical graph convolutional neural network construction unit, based on the multi-parallel branch graph convolutional network structure obtained by the multi-branch graph convolutional neural network structure construction unit, uses a multi-scale feature multi-scale feature fusion module to model the mapping relationship between nodes in different graph topologies through 1x1 convolution, thereby realizing information interaction. The fusion module connects different branches to achieve information interaction between branches and constructs a hierarchical graph convolutional neural network.

[0079] The above graph coarsening unit, multi-branch graph convolutional neural network structure construction unit, and hierarchical graph convolutional neural network construction unit constitute a network construction system for 3D human pose estimation. In actual use, it can be embedded into various current mainstream graph convolution methods. According to the constructed hierarchical graph convolutional neural network, feature extraction is performed on the input 2D human joint point coordinates to regress the 3D human joint point coordinates, forming a complete 3D human pose estimation system (the construction unit 21 of the graph convolutional neural network, the training unit 22 of the graph convolutional neural network, and the 3D human pose estimation unit 23). The construction unit 21 of the graph convolutional neural network generates an optimized graph structure. After feature extraction through the hierarchical graph convolutional neural network, 3D human pose estimation is performed in the 3D human pose estimation unit.

[0080] Aggregate the graph signals at the neighbor sampling points to the central sampling point, while performing signal upsampling and downsampling. Compared with the original input 2D human joint point coordinates, the hierarchical graph convolutional neural network extracts more representative high-dimensional features. In the 3D human pose estimation unit, the high-dimensional features pass through a multi-layer perceptron multiple times and are finally mapped to the 3D common graph signal dimension. The common change in the graph signal dimension is 2→128→128→128→128→3, where 2 and 3 are the dimensions of the input and output human joint point coordinates respectively.

[0081] Implementation Effect:

[0082] In this embodiment, mainstream graph convolution methods can be adopted. The evaluation can be divided into objective evaluation and subjective evaluation. The former includes statistical analysis of the results of 3D human pose estimation to obtain indicators such as MPJPE; the latter includes visualizing the results of 3D human pose estimation. In this embodiment, a comparison is made with the pose estimation results of the original existing method on the Human3.6M dataset containing multiple actions.

[0083] Experiments show that in terms of objective evaluation, compared with the existing methods, the present implementation has a significant improvement in MPJPE for self-occluded and complex actions, and also an improvement in the average MPJPE. Further data analysis proves that after adopting the method of this embodiment, in the network construction method, a graph topology structure denser than the human skeleton is considered, which means that the neural network can capture more local information for the input two-dimensional human joint coordinates, and can well represent the structural information of the joints in the case of complex actions and self-occlusions in the input two-dimensional human postures, effectively improving the three-dimensional human pose estimation results of this action.

[0084] After adopting the method of this embodiment, the three-dimensional human pose estimation effect for complex actions and self-occluded actions has been improved. Refer to Figure 4 the visualization results: The three rows from top to bottom are three different input actions. In the left column, the dark and light colors are the ground truth and the three-dimensional human pose estimation results of the method of this example respectively, and the ground truth is the manually annotated semantic category. In the right column, the dark and light colors are the ground truth and the three-dimensional human pose estimation results of the traditional graph neural network structure respectively. The closer the dark and light skeletons are, the more accurate the three-dimensional human pose estimation is. It can be seen that compared with the right column, the error in the left column is significantly smaller. By introducing a hierarchical graph topology structure and constructing a hierarchical neural network structure, the method of this embodiment helps to significantly improve the three-dimensional human pose estimation accuracy for complex actions and self-occluded actions.

[0085] In another embodiment of the present invention, a three-dimensional human pose estimation device is further provided, which includes: a memory and a processor. Among them, the memory is used to store computer programs; the processor is used to execute the computer programs to implement the network construction for three-dimensional human pose estimation in the above embodiments, or to implement the network construction system for three-dimensional human pose estimation in the above embodiments, or to implement the three-dimensional human pose estimation system in the above embodiments.

[0086] In another embodiment of the present invention, a computer-readable storage medium is further provided, on which a computer program is stored, and when the program is executed by a processor, it is used to execute the graph convolutional network construction method for three-dimensional pose estimation, or to execute the three-dimensional human pose estimation method.

[0087] Optionally, a memory for storing programs; the memory may include volatile memory (e.g., random-access memory, such as static random-access memory (SRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM), etc.); the memory may also include non-volatile memory, such as flash memory. The memory 62 is used to store computer programs (such as application programs and functional modules for implementing the above method), computer instructions, etc. The above computer programs, computer instructions, etc. may be partitioned and stored in one or more memories. And the above computer programs, computer instructions, data, etc. can be called by the processor.

[0088] The above computer programs, computer instructions, etc. may be partitioned and stored in one or more memories. And the above computer programs, computer instructions, data, etc. can be called by the processor.

[0089] A processor for executing the computer programs stored in the memory to implement each step in the method involved in the above embodiments. For specific details, reference can be made to the relevant descriptions in the previous method embodiments.

[0090] The processor and the memory may be of an independent structure or an integrated structure integrated together. When the processor and the memory are of an independent structure, the memory and the processor may be coupled and connected through a bus.

[0091] It should be noted that the steps in the method provided by the present invention can be implemented by corresponding modules, devices, units, etc. in the system. Those skilled in the art can refer to the technical solution of the system to implement the step flow of the method. That is, the embodiments in the system can be understood as preferred examples for implementing the method, and will not be elaborated here.

[0092] Those skilled in the art know that, in addition to implementing the system and its various devices provided by the present invention in the form of pure computer-readable program code, the method steps can be logically programmed to enable the system and its various devices provided by the present invention to be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers, etc., to achieve the same functions. Therefore, the system and its various devices provided by the present invention can be considered as a kind of hardware component, and the devices included therein for implementing various functions can also be regarded as the structures within the hardware component; the devices for implementing various functions can also be regarded as either software modules for implementing the method or the structures within the hardware component.

[0093] Those skilled in the art can understand that, except that at least some of such features and / or processes or units are mutually exclusive, any combination can be adopted to combine all the features disclosed in this specification and all the processes or units of any device so disclosed.

[0094] Only the preferred embodiments of the present invention are disclosed herein. The present specification selects and specifically describes these embodiments to better explain the principles and practical applications of the present invention, rather than to limit the present invention. Any modifications and changes made by those skilled in the art within the scope of the specification shall fall within the scope protected by the present invention.

Claims

1. A method for constructing a graph neural network for three-dimensional human pose estimation, characterized in that, Including: S11: Coarsening the graph topology of the dense human surface to obtain graph topologies at different coarsening levels; S12: According to the graph topology of the coarsened human surface obtained in S11 and combining it with the graph topology of the human skeleton, construct a graph convolutional neural network structure with multiple parallel branches in sequence from sparse to dense; S13: According to the multi-parallel branch graph convolutional network structure obtained in S12, connect different branches through a multi-scale feature fusion module to realize information interaction between branches, and construct a hierarchical graph convolutional neural network; The S12 further includes: S121: Topological structures of human body surface maps at different coarsening levels obtained according to S11 Select the two graph topologies with the highest coarsening level among them and the topological structure g of the human skeleton map P , as the three required graph topological structures; S122: Construct a residual graph convolutional module through two graph convolutional layers, two non-linear activation layers, two batch normalization layers, and one non-local module layer, arranged serially as graph convolution, non-linear activation, batch normalization, graph convolution, non-linear activation, batch normalization, non-local module; S123: According to the number of nodes in the graph topology, construct three parallel graph convolutional neural network branches in sequence from sparse to dense, and each branch has a different graph topology and a different number of residual graph convolutional modules.

2. The method for constructing a graph neural network for three-dimensional human pose estimation according to claim 1, wherein, The S11 further includes: During the coarsening operation, merge some graph nodes together to obtain the next-level coarsened graph, and repeat this process until the coarsened graph is small enough.

3. The method for constructing a graph neural network for three-dimensional human pose estimation according to claim 1, wherein According to the coarsening degree in the graph topologies of the three parallel graph convolutional neural network branches in S12, assign different numbers of residual graph convolutional modules to each graph convolutional neural network branch.

4. The method for constructing a graph neural network for three-dimensional human pose estimation according to claim 1, characterized in that, The S13 further includes: The multi-scale feature fusion module models the mapping relationship between nodes of different graph topologies through 1x1 convolution, so as to realize information interaction.

5. A graph convolutional network construction system for three-dimensional pose estimation, characterized in that, Including: A graph coarsening unit, which coarsens the graph topology of the dense human surface to obtain graph topologies at different coarsening levels; A multi-branch graph convolutional neural network structure construction unit, which constructs a graph convolutional neural network structure with multiple parallel branches in sequence from sparse to dense according to the graph topology of the coarsened human surface obtained by the graph coarsening unit and combining it with the graph topology of the human skeleton; A hierarchical graph convolutional neural network construction unit, based on the multi-parallel branch graph convolutional network structure obtained by the multi-branch graph convolutional neural network structure construction unit, connects different branches through a multi-scale feature fusion module to realize information interaction between branches, and constructs a hierarchical graph convolutional neural network; The multi-branch graph convolutional neural network structure construction unit further includes: The topological structures of the human body surface maps at different coarsening levels obtained according to the figure coarsening unit Select the two graph topologies with the highest coarsening levels among them and the topological structure g of the human body skeleton map P , as the three required graph topological structures; Construct a residual graph convolutional module through two graph convolutional layers, two non-linear activation layers, two batch normalization layers, and one non-local module layer, arranged serially as graph convolution, non-linear activation, batch normalization, graph convolution, non-linear activation, batch normalization, non-local module; According to the number of nodes in the graph topology, construct three parallel graph convolutional neural network branches in sequence from sparse to dense, and each branch has a different graph topology and a different number of residual graph convolutional modules.

6. A three-dimensional human body pose estimation method, characterized in that, Including: A hierarchical graph convolutional neural network obtained by using any one of claims 1 to 4 or claim 5; Train the hierarchical graph convolutional neural network; Perform three-dimensional human pose estimation using the trained graph convolutional neural network.

7. A three-dimensional human body pose estimation system, characterized in that, Including: The construction unit of the graph convolutional neural network, using the hierarchical graph convolutional neural network obtained by any one of claims 1 to 4 or claim 5; The training unit of the graph convolutional neural network, training the graph convolutional neural network constructed by the construction unit of the graph convolutional neural network; The three-dimensional human pose estimation module, performing three-dimensional human pose estimation using the graph convolutional neural network trained by the training unit of the graph convolutional neural network.

8. A three-dimensional attitude estimation device, characterized in that, Including: A memory and a processor; wherein, The memory is used to store a computer program; The processor is used to execute the computer program to implement the graph neural network construction method for three-dimensional human pose estimation according to any one of claims 1 to 4, or to implement the three-dimensional human pose estimation method according to claim 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it is used to execute the graph neural network construction method for three-dimensional human pose estimation according to any one of claims 1 to 4, or to execute the three-dimensional human pose estimation method according to claim 6.

Images