Monocular 3D attitude estimation method based on cross network and related device

Abstract

The invention discloses a monocular 3D attitude estimation method based on a cross network and a related device, and the method comprises the steps: receiving a to-be-estimated monocular image, and carrying out the extraction of human body joint coordinates of the to-be-estimated monocular image through a 2D attitude model, and obtaining the 2D joint coordinate information of a human body; performing local and global feature information extraction processing on the human body 2D joint coordinate information by using a double-layer feature extraction module to obtain local feature information and global feature information; performing cross fusion processing on the local feature information and the global feature information to obtain cross fusion feature information; and performing 3D human body posture estimation processing by using the cross fusion feature information based on a layered MLP module to obtain estimated 3D human body posture information. In the embodiment of the invention, the local feature information and the global feature information of the image can be accurately extracted; meanwhile, feature missing is reduced; therefore, higher-precision 3D human body posture estimation is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a monocular 3D posture estimation method based on a cross network and a related device. Background Art

[0002] Single Vision Figure 3 3D human pose estimation aims to recover the 3D positions of human joints from images. In recent years, deep learning has made significant progress in monocular 3D human pose estimation, and its applications in action recognition, virtual reality, dance analysis, human-computer interaction and many other fields have developed rapidly.

[0003] At present, 3D human pose estimation methods are mainly divided into two types: direct estimation from images and through 2D to 3D conversion. The direct estimation method does not use the 2D pose results as an intermediate step, but relies on a large end-to-end neural network to directly infer the 3D pose from the image. Since the direct estimation method lacks 2D pose results as an intermediate representation, the performance is usually average. In contrast, the 2D to 3D conversion method first uses an advanced 2D pose estimator to obtain 2D information of human key points from a single image or video, and then converts the 2D human pose into 3D coordinates through regression. With the help of advanced 2D pose detectors, the 2D to 3D conversion method is usually more effective than the direct estimation method.

[0004] Currently, more models use the graph convolutional network (GCN) method alone in combination with human body geometry priors, or use the attention mechanism alone to capture global information, which significantly improves the accurate prediction of 3D human skeleton positions; however, using graph convolutional networks alone will lack information acquisition of distant joints; for example, when performing dance movements, the relationship between hand and foot joints is difficult to capture using only local information methods; in addition, when building deeper models based on GCN, over-smoothing often occurs. As the number of layers increases, the representations of adjacent nodes become too similar, resulting in information loss and confusion between nodes; in addition, using the attention mechanism alone will lack exploration of the topological structure of the human skeleton, unable to fully explore the geometric connections between joints, and lack structural information in 2D joint coordinates; therefore, it is of great significance to design a model that can effectively integrate the advantages of GCN and attention mechanisms to capture local and global dependencies between human skeleton joints.

[0005] Due to the complexity and interconnectedness of the human structure, accurately capturing the 3D representation of the human body requires a method that can capture both local and global information dependencies of the human body from its 2D pose; therefore, the proposed model should be able to achieve this goal by imposing local and global constraints on different human joints. Summary of the invention

[0006] The purpose of the present invention is to overcome the shortcomings of the prior art. The present invention provides a monocular 3D posture estimation method and related devices based on a cross network, which can accurately extract local feature information and global feature information of an image; while reducing feature loss; thereby achieving higher-precision 3D human posture estimation.

[0007] In order to solve the above technical problems, an embodiment of the present invention provides a monocular 3D posture estimation method based on a cross network, the method comprising:

[0008] Receiving a monocular image to be estimated, and performing human joint coordinate extraction processing on the monocular image to be estimated using a 2D posture model to obtain human 2D joint coordinate information corresponding to the monocular image to be estimated;

[0009] Using a double-layer feature extraction module to perform local and global feature information extraction processing on the 2D joint coordinate information of the human body, to obtain local feature information and global feature information corresponding to the 2D joint coordinate information of the human body;

[0010] Cross-fusion processing is performed on the local feature information and the global feature information corresponding to the 2D joint coordinate information of the human body to obtain the cross-fusion feature information corresponding to the 2D joint coordinate information of the human body;

[0011] Based on the hierarchical MLP module, 3D human body posture estimation processing is performed using the cross-fusion feature information corresponding to the 2D joint coordinate information of the human body to obtain estimated 3D human body posture information.

[0012] Optionally, the using a 2D posture model to perform human joint coordinate extraction processing on the monocular image to be estimated to obtain human 2D joint coordinate information corresponding to the monocular image to be estimated includes:

[0013] The monocular image to be estimated is input into the 2D posture model, and the joint coordinates of the target human body in the monocular image to be estimated are extracted and processed in the 2D posture model to obtain the human body 2D joint coordinate information corresponding to the monocular image to be estimated.

[0014] Optionally, the using of a double-layer feature extraction module to perform local and global feature information extraction processing on the 2D joint coordinate information of the human body to obtain local feature information and global feature information corresponding to the 2D joint coordinate information of the human body includes:

[0015] After the human body 2D joint coordinate information is input into the double-layer feature extraction module, the local graph convolution layer in the double-layer feature extraction module is used to sequentially perform local feature extraction processing on the human body 2D joint coordinate information to obtain local feature information corresponding to the human body 2D joint coordinate information;

[0016] The global attention layer in the two-layer feature extraction module is used to perform global feature information extraction processing on the 2D joint coordinate information of the human body to obtain global feature information corresponding to the 2D joint coordinate information of the human body.

[0017] Optionally, the using the local graph convolution layer in the double-layer feature extraction module to sequentially perform local feature extraction processing on the 2D joint coordinate information of the human body to obtain local feature information corresponding to the 2D joint coordinate information of the human body includes:

[0018] In the double-layer feature extraction module, a graph convolution network is used to perform convolution processing on the 2D joint coordinate information of the human body to obtain the convolved 2D joint coordinate information of the human body;

[0019] The convolved 2D joint coordinate information of the human body is normalized through batch processing to obtain a normalized result;

[0020] The normalized result is activated by a GELU activation function to obtain an activation result;

[0021] The activation processing results are stacked in sequence into a 1*1 convolution block, and after multiple local convolution layer stacking processes, local feature information corresponding to the 2D joint coordinate information of the human body is obtained.

[0022] Optionally, the using the global attention layer in the two-layer feature extraction module to perform global feature information extraction processing on the 2D joint coordinate information of the human body to obtain global feature information corresponding to the 2D joint coordinate information of the human body includes:

[0023] In the two-layer feature extraction module, the fully connected layer is called to map the input human body 2D joint coordinate information into a query Q, a key K and a value V;

[0024] The query Q, key K and value V are input into the multi-head self-attention mechanism and processed in parallel by multiple heads. The scaled dot product is used to perform attention calculation in each head to obtain the attention calculation result corresponding to each head.

[0025] Based on the attention calculation results corresponding to each head, the global feature information of the 2D joint coordinate information of the human body is extracted and processed to obtain the global feature information corresponding to the 2D joint coordinate information of the human body.

[0026] Optionally, cross-fusion processing is performed on the local feature information and the global feature information corresponding to the 2D joint coordinate information of the human body to obtain the cross-fusion feature information corresponding to the 2D joint coordinate information of the human body, including:

[0027] The global feature information corresponding to the 2D joint coordinate information of the human body is used as query query1, and the local feature information is used as key key1 and value value1 to input into the first multi-head attention module to perform global relationship capture processing of the human skeleton, so as to obtain the global feature relationship of the human skeleton;

[0028] Inputting the global characteristic relationship of the human skeleton into a feedforward neural network model to perform global relationship perception processing of human joints to obtain a global perception relationship of human joints;

[0029] Fusing the global perception relationship with the global feature information to obtain fused global feature information;

[0030] Based on the fully connected relationship, the global perception relationship is used as the key key2 and the value value2, and the local feature information is used as the query query2;

[0031] Input the key key2, the value value2 and the query query2 into the second multi-head attention module for local perception relationship extraction processing to obtain the local perception relationship;

[0032] Fusing the local fusion perception relationship with the local feature information to obtain fused local feature information;

[0033] The fused global feature information and the fused local feature information are fused to form the corresponding 2D joint coordinate information of the human body.

[0034] Optionally, the hierarchical MLP module performs 3D human body posture estimation processing using the cross-fusion feature information corresponding to the 2D joint coordinate information of the human body to obtain estimated 3D human body posture information, including:

[0035] After the cross-fusion feature information corresponding to the 2D joint coordinate information of the human body is input into the hierarchical MLP module, layer normalization is used to perform layer normalization processing on the cross-fusion feature information to obtain the cross-fusion feature information after layer normalization;

[0036] The cross-fusion feature information after layer normalization is subjected to linear processing of the number of J nodes*half the dimension of C channels to obtain the first linear processing result;

[0037] The first linear processing result is sequentially subjected to second linear processing and third linear processing of the dimension of J node number*half C channel to obtain a second linear processing result and a third linear processing result;

[0038] The first linear processing result, the second linear processing result and the third linear processing result are subjected to linear processing of the J node*C channel dimension to obtain a fourth linear processing result;

[0039] The fourth linear processing result and the input cross-fusion feature information are output through a fully connected layer to obtain output estimated 3D human body posture information.

[0040] In addition, an embodiment of the present invention further provides a monocular 3D posture estimation device based on a cross network, the device comprising:

[0041] The human body joint coordinate extraction module is used to receive the monocular image to be estimated, and use the 2D posture model to perform human body joint coordinate extraction processing on the monocular image to be estimated, so as to obtain the human body 2D joint coordinate information corresponding to the monocular image to be estimated;

[0042] Feature information extraction module: used to perform local and global feature information extraction processing on the 2D joint coordinate information of the human body using a double-layer feature extraction module to obtain local feature information and global feature information corresponding to the 2D joint coordinate information of the human body;

[0043] Cross-fusion module: used for cross-fusion processing of local feature information and global feature information corresponding to the 2D joint coordinate information of the human body, so as to obtain cross-fusion feature information corresponding to the 2D joint coordinate information of the human body;

[0044] Human body posture estimation module: used to perform 3D human body posture estimation processing based on the hierarchical MLP module using the cross-fusion feature information corresponding to the 2D joint coordinate information of the human body to obtain estimated 3D human body posture information.

[0045] In addition, an embodiment of the present invention further provides an electronic device, including a processor and a memory, wherein the processor runs a computer program or code stored in the memory to implement a monocular 3D posture estimation method as described in any one of the above.

[0046] In addition, an embodiment of the present invention further provides a computer-readable storage medium for storing a computer program or code. When the computer program or code is executed by a processor, the monocular 3D posture estimation method as described in any one of the above is implemented.

[0047] In an embodiment of the present invention, a double-layer feature extraction module, a bidirectional feature fusion module and a hierarchical MLP module are sequentially arranged, and after extracting the 2D joint coordinate information of the human body in the monocular image to be estimated, the double-layer feature extraction module, the bidirectional feature fusion module and the hierarchical MLP module are sequentially processed to realize 3D posture estimation; the double-layer feature extraction module can accurately extract the local feature information and the global feature information of the image; the bidirectional feature fusion module is used to realize feature interaction and fusion, and the hierarchical MLP module is used to reduce feature missing; thereby realizing higher-precision 3D human posture estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 is a schematic flow chart of a monocular 3D posture estimation method based on a cross network in an embodiment of the present invention;

[0050] Figure 2 is a schematic flow chart of a monocular 3D posture estimation method based on a cross network in another embodiment of the present invention;

[0051] Figure 3 Schematic diagram of the structure of a monocular 3D posture estimation device based on a cross network in an embodiment of the present invention;

[0052] Figure 4 is a schematic diagram of the structure of an electronic device in an embodiment of the present invention;

[0053] Figure 5 It is a schematic diagram of the processing flow structure of the hierarchical MLP module in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] For example, see Figure 1 , Figure 1 It is a flow chart of a monocular 3D posture estimation method based on a cross network in an embodiment of the present invention.

[0056] like Figure 1 As shown, a monocular 3D pose estimation method based on a cross network, the method comprising:

[0057] S101: receiving a monocular image to be estimated, and performing human body joint coordinate extraction processing on the monocular image to be estimated using a 2D posture model to obtain human body 2D joint coordinate information corresponding to the monocular image to be estimated;

[0058] In the specific implementation process of the present invention, the use of a 2D posture model to extract human joint coordinates from the monocular image to be estimated to obtain 2D joint coordinate information of the human body corresponding to the monocular image to be estimated includes: inputting the monocular image to be estimated into the 2D posture model, extracting the joint coordinates of the target human body in the monocular image to be estimated within the 2D posture model, and obtaining 2D joint coordinate information of the human body corresponding to the monocular image to be estimated.

[0059] Specifically, in the present technical solution, a 2D posture model is first used to extract the total human joint coordinate information of the monocular image to be estimated; that is, after receiving the monocular image to be estimated, the monocular image to be estimated is input into the 2D posture model, and after being segmented, i.e., position embedded, through the channel layer in the 2D posture model, the human 2D joint coordinate information of the human posture in the monocular image to be estimated is extracted.

[0060] S102: using a double-layer feature extraction module to perform local and global feature information extraction processing on the 2D joint coordinate information of the human body, to obtain local feature information and global feature information corresponding to the 2D joint coordinate information of the human body;

[0061] In the specific implementation process of the present invention, the double-layer feature extraction module is used to perform local and global feature information extraction processing on the human body 2D joint coordinate information to obtain local feature information and global feature information corresponding to the human body 2D joint coordinate information, including: after the human body 2D joint coordinate information is input into the double-layer feature extraction module, the local graph convolution layer in the double-layer feature extraction module is used to perform local feature extraction processing on the human body 2D joint coordinate information in sequence to obtain the local feature information corresponding to the human body 2D joint coordinate information; the global attention layer in the double-layer feature extraction module is used to perform global feature information extraction processing on the human body 2D joint coordinate information to obtain the global feature information corresponding to the human body 2D joint coordinate information.

[0062] Furthermore, the local graph convolution layer in the double-layer feature extraction module is used to perform local feature extraction processing on the human body 2D joint coordinate information in sequence to obtain local feature information corresponding to the human body 2D joint coordinate information, including: using a graph convolution network in the double-layer feature extraction module to perform convolution processing on the human body 2D joint coordinate information to obtain the convolved human body 2D joint coordinate information; normalizing the convolved human body 2D joint coordinate information through batch processing to obtain a normalized result; activating the normalized result through a GELU activation function to obtain an activation processing result; stacking the activation processing result into a 1*1 convolution block in sequence, and performing superposition processing of multiple local convolution layers to obtain the local feature information corresponding to the human body 2D joint coordinate information.

[0063] Furthermore, the global attention layer in the double-layer feature extraction module is used to perform global feature information extraction processing on the human body 2D joint coordinate information to obtain global feature information corresponding to the human body 2D joint coordinate information, including: calling the fully connected layer in the double-layer feature extraction module to map the input human body 2D joint coordinate information to query Q, key K and value V; inputting the query Q, key K and value V into a multi-head self-attention mechanism and dividing them into multiple heads for parallel processing, performing attention calculation processing on each head using the scaled dot product to obtain the attention calculation result corresponding to each head; extracting the global feature information of the human body 2D joint coordinate information based on the attention calculation result corresponding to each head to obtain the global feature information corresponding to the human body 2D joint coordinate information.

[0064] Specifically, a two-layer feature extraction module is used to extract local features and global features. In this technical solution, the graphic features of the human bone structure are fully considered. The local graph convolution layer is sequentially passed through GCN (graph convolution network) and batch normalization and GELU operations, and finally 1X1 convolution blocks are sequentially stacked and multiple local graph convolution layers are superimposed to extract the feature information of the local graph convolution. The feature information of the local graph convolution is expressed as follows:

[0065]

[0066] in, I is the identity matrix; yes The degree matrix of ; H is the input feature; H′ is the output feature.

[0067] That is, in the double-layer feature extraction module, the graph convolution network is used to perform convolution processing on the 2D joint coordinate information of the human body to obtain the convolved 2D joint coordinate information of the human body; the convolved 2D joint coordinate information of the human body is then normalized by batch processing to obtain a normalized result; the normalized result is activated by the GELU activation function to obtain an activated result; finally, the activated result is stacked in sequence into a 1*1 convolution block, and after multiple local convolution layers are superimposed, the local feature information corresponding to the 2D joint coordinate information of the human body is obtained.

[0068] The purpose of using GCN is to utilize the features of different adjacent joints to constrain the feature representation of the target joint, so as to capture the local dependencies between joints through this local constraint and achieve the goals of local constraint and dimensionality transformation; improve the stability of feature information through intermediate layer operations, enhance the linear expression ability of features, and improve the effectiveness of feature representation; then, perform point-to-point convolution by applying 1*1 convolution blocks to reduce the redundancy caused by local feature extraction and filter out high-dimensional or low-dimensional valid node features; finally, aggregate adjacent nodes by stacking multiple local graph convolution layers to obtain efficient feature representation.

[0069] In the extraction of global feature information, the fully connected layer is first called in the double-layer feature extraction module to map the input human 2D joint coordinate information into query Q, key K and value V; the query Q, key K and value V are input into the multi-head self-attention mechanism and divided into multiple heads for parallel processing, and the scaled dot product is used to perform attention calculation processing on each head to obtain the attention calculation result corresponding to each head; the global feature information of the human 2D joint coordinate information is extracted and processed according to the attention calculation result corresponding to each head to obtain the global feature information corresponding to the human 2D joint coordinate information.

[0070] That is, the global attention layer of the present technical solution fully considers the limitation that the feature representation of joints needs to rely on the features of other joints and their own features; the global attention layer first uses the fully connected layer to map the input into queries (Q), keys (K) and values ​​(V), and then inputs them into the multi-head self-attention mechanism, which is divided into multiple heads for parallel processing, and each head uses the scaled dot product for attention calculation; the attention layer can enhance the network model's perception of feature information, while expanding the utilization of feature information and solving the problem of severe loss of position information; by executing this process on each joint to constrain the features of all joints, the global dependency information of the human body is obtained, so that the model can effectively obtain various feature information from different positions; therefore, the global attention layer can effectively capture global dependency information and improve the modeling accuracy of human posture.

[0071] S103: cross-fusion processing is performed on the local feature information and the global feature information corresponding to the 2D joint coordinate information of the human body to obtain the cross-fusion feature information corresponding to the 2D joint coordinate information of the human body;

[0072] In the specific implementation process of the present invention, the local feature information and the global feature information corresponding to the 2D joint coordinate information of the human body are cross-fused to obtain the cross-fused feature information corresponding to the 2D joint coordinate information of the human body, including: the global feature information corresponding to the 2D joint coordinate information of the human body is used as query query1, and the local feature information is used as key key1 and value value1 to input into the first multi-head attention module to perform global relationship capture processing of the human skeleton to obtain the global feature relationship of the human skeleton; the global feature relationship of the human skeleton is input into the feedforward neural network model to perform global relationship perception processing of the human joints to obtain the global perception of the human joints. relationship; fusing the global perception relationship with the global feature information to obtain fused global feature information; based on the fully connected relationship, the global perception relationship is used as the key key2 and the value value2, and the local feature information is used as the query query2; the key key2, the value value2 and the query query2 are input into the second multi-head attention module to perform local perception relationship extraction and obtain the local perception relationship; fusing the local fused perception relationship with the local feature information to obtain the fused local feature information; fusing the fused global feature information and the fused local feature information to form the corresponding 2D joint coordinate information of the human body.

[0073] Specifically, the global feature information corresponding to the 2D joint coordinate information of the human body is used as the query query1, and the local feature information is used as the key key1 and the value value1 to be input into the first multi-head attention module for global relationship capture processing of the human skeleton to obtain the global feature relationship of the human skeleton; then the global feature relationship of the human skeleton is input into the feedforward neural network model for global relationship perception processing of the human joints to obtain the global perception relationship of the human joints; the global perception relationship can be fused with the global feature information to obtain fused global feature information; then the global perception relationship is used as the key key2 and the value value2 by using the fully connected relationship, and the local feature information is used as the query query2; and the key key2, the value value2 and the query query2 are input into the second multi-head attention module for local perception relationship extraction processing to obtain the local perception relationship; the local fusion perception relationship can be fused with the local feature information to obtain the fused local feature information; finally, the fused global feature information and the fused local feature information are fused to form the corresponding human 2D joint coordinate information.

[0074] That is, in this technical solution, a bidirectional feature fusion cross module is proposed to effectively process the local and global feature information obtained at the channel level; the fusion of local information is guided by global information, the global feature information is the query, and the local feature information is the key and value. The global relationship of the human skeleton captured by the attention block is fed back to the GCN block, so that the GCN can better perceive the global relationship of the human joints; at the same time, the global feature information extracted globally will also be added to each pixel of the local feature; then, the output feature information is guided by the local information to guide the fusion of information, the local feature information is the query, and the output feature information XG′ is the key and value. The local feature information relationship of the human skeleton captured by the GCN block is fed back to the output feature XG′, enhancing the understanding and representation of the model. Finally, the obtained XG′ is added to the original feature information and then cross-input into the GCN block to obtain local information, and the obtained XL′ is added to the original feature information and then cross-input into the attention block to obtain global information. Through the two-way cross-complementary information, the modeling ability of the GCN block and the attention block is enhanced.

[0075] The expression of the dual feature fusion cross module in this embodiment is:

[0076] A L→G =[Attn(Q XG ,Linear(K XL ,V XL ))];

[0077] A G→L =[Attn(Q XL ,Linear(K XG′ ,V XG′ ))];

[0078] X Fusion =Local(X L +X L′ )+Global(X G +X G′ ).

[0079] S104: Based on the hierarchical MLP module, 3D human body posture estimation processing is performed using the cross-fusion feature information corresponding to the 2D joint coordinate information of the human body to obtain estimated 3D human body posture information.

[0080] In the specific implementation process of the present invention, the hierarchical MLP module uses the cross-fusion feature information corresponding to the 2D joint coordinate information of the human body to perform 3D human posture estimation processing to obtain estimated 3D human posture information, including: after the cross-fusion feature information corresponding to the 2D joint coordinate information of the human body is input into the hierarchical MLP module, the cross-fusion feature information is layer-normalized by layer normalization to obtain the cross-fusion feature information after layer normalization; the cross-fusion feature information after layer normalization is subjected to linear processing of the number of J nodes*half of the C channel dimension to obtain a first linear processing result; the first linear processing result is subjected to second linear and third linear processing of the number of J nodes*half of the C channel dimension in sequence to obtain a second linear processing result and a third linear processing result; the first linear processing result, the second linear processing result and the third linear processing result are subjected to linear processing of the J node*C channel dimension to obtain a fourth linear processing result; the fourth linear processing result and the input cross-fusion feature information are subjected to output processing through a fully connected layer to obtain output estimated 3D human posture information.

[0081] For details, see Figure 5 ,in Figure 5 J in the figure represents the number of nodes, and C represents the channel dimension. After the hierarchical MLP module combines local features with global features and takes them as input, it applies a linear layer to reduce the channel dimension, and then performs hierarchical linear layer acquisition to obtain feature information. After adding the channel dimension, it increases the channel dimension through a linear layer and then adds it to the original input. While acquiring important information, it can retain the original features to achieve the diversity of feature information and improve the robustness of the model. The formula of the hierarchical MLP module is:

[0082] X m1 =Linear(LN(X input ));

[0083] X m2 ,X m3 =Linear(X m1 )+X m1 ;

[0084] X output =Linear(X m1 +X m2 +X m3 )+X input ;

[0085] That is, after the cross-fusion feature information corresponding to the 2D joint coordinate information of the human body is input into the layer MLP module, the cross-fusion feature information is layer-normalized by using layer normalization to obtain the cross-fusion feature information after layer normalization; then the cross-fusion feature information after layer normalization is subjected to linear processing of the number of J nodes*half of the C channel dimension to obtain the first linear processing result; and the first linear processing result is subjected to the second linear and third linear processing of the number of J nodes*half of the C channel dimension in sequence to obtain the second linear processing result and the third linear processing result; at the same time, the first linear processing result, the second linear processing result and the third linear processing result are subjected to linear processing of the J node*C channel dimension to obtain the fourth linear processing result; finally, the fourth linear processing result and the input cross-fusion feature information are subjected to output processing through a fully connected layer to obtain the output estimated 3D human body posture information.

[0086] In an embodiment of the present invention, a double-layer feature extraction module, a bidirectional feature fusion module and a hierarchical MLP module are sequentially arranged, and after extracting the 2D joint coordinate information of the human body in the monocular image to be estimated, the double-layer feature extraction module, the bidirectional feature fusion module and the hierarchical MLP module are sequentially processed to realize 3D posture estimation; the double-layer feature extraction module can accurately extract the local feature information and the global feature information of the image; the bidirectional feature fusion module is used to realize feature interaction and fusion, and the hierarchical MLP module is used to reduce feature missing; thereby realizing higher-precision 3D human posture estimation.

[0087] For example 2, please refer to Figure 2 , Figure 2 It is a flow chart of a monocular 3D posture estimation method based on a cross network in another embodiment of the present invention.

[0088] like Figure 2 As shown, a monocular 3D pose estimation method based on a cross network, the method comprising:

[0089] S201: receiving a monocular image to be estimated, and performing human body joint coordinate extraction processing on the monocular image to be estimated using a 2D posture model to obtain human body 2D joint coordinate information corresponding to the monocular image to be estimated;

[0090] S202: using a double-layer feature extraction module to perform local and global feature information extraction processing on the 2D joint coordinate information of the human body to obtain local feature information and global feature information corresponding to the 2D joint coordinate information of the human body;

[0091] S203: cross-fusion processing is performed on the local feature information and the global feature information corresponding to the 2D joint coordinate information of the human body to obtain the cross-fusion feature information corresponding to the 2D joint coordinate information of the human body;

[0092] S204: after the cross-fusion feature information corresponding to the 2D joint coordinate information of the human body is input into the hierarchical MLP module, layer normalization is used to perform layer normalization processing on the cross-fusion feature information to obtain the cross-fusion feature information after layer normalization;

[0093] S205: performing linear processing of the cross-fusion feature information after layer normalization with the number of J nodes*half the dimension of C channels to obtain a first linear processing result;

[0094] S206: performing second linear processing and third linear processing of the J node number*half C channel dimension on the first linear processing result in sequence to obtain a second linear processing result and a third linear processing result;

[0095] S207: subjecting the first linear processing result, the second linear processing result, and the third linear processing result to linear processing in the J-node*C-channel dimension to obtain a fourth linear processing result;

[0096] S208: The fourth linear processing result and the input cross-fusion feature information are processed through a fully connected layer to obtain output estimated 3D human body posture information.

[0097] The specific implementation of the second embodiment can be found in the above embodiments, which will not be described in detail here.

[0098] For example 3, please refer to Figure 3 , Figure 3 It is a schematic diagram of the structural composition of a monocular 3D posture estimation device based on a cross network in an embodiment of the present invention.

[0099] like Figure 3 As shown, a monocular 3D posture estimation device based on a cross network, the device comprises:

[0100] The human body joint coordinate extraction module 301 is used to receive a monocular image to be estimated, and perform human body joint coordinate extraction processing on the monocular image to be estimated using a 2D posture model to obtain human body 2D joint coordinate information corresponding to the monocular image to be estimated;

[0101] In the specific implementation process of the present invention, the use of a 2D posture model to extract human joint coordinates from the monocular image to be estimated to obtain 2D joint coordinate information of the human body corresponding to the monocular image to be estimated includes: inputting the monocular image to be estimated into the 2D posture model, extracting the joint coordinates of the target human body in the monocular image to be estimated within the 2D posture model, and obtaining 2D joint coordinate information of the human body corresponding to the monocular image to be estimated.

[0102] Specifically, in the present technical solution, a 2D posture model is first used to extract the total human joint coordinate information of the monocular image to be estimated; that is, after receiving the monocular image to be estimated, the monocular image to be estimated is input into the 2D posture model, and after being segmented, i.e., position embedded, through the channel layer in the 2D posture model, the human 2D joint coordinate information of the human posture in the monocular image to be estimated is extracted.

[0103] Feature information extraction module 302: used to perform local and global feature information extraction processing on the 2D joint coordinate information of the human body by using a double-layer feature extraction module to obtain local feature information and global feature information corresponding to the 2D joint coordinate information of the human body;

[0104] In the specific implementation process of the present invention, the double-layer feature extraction module is used to perform local and global feature information extraction processing on the human body 2D joint coordinate information to obtain local feature information and global feature information corresponding to the human body 2D joint coordinate information, including: after the human body 2D joint coordinate information is input into the double-layer feature extraction module, the local graph convolution layer in the double-layer feature extraction module is used to perform local feature extraction processing on the human body 2D joint coordinate information in sequence to obtain the local feature information corresponding to the human body 2D joint coordinate information; the global attention layer in the double-layer feature extraction module is used to perform global feature information extraction processing on the human body 2D joint coordinate information to obtain the global feature information corresponding to the human body 2D joint coordinate information.

[0105] Furthermore, the local graph convolution layer in the double-layer feature extraction module is used to perform local feature extraction processing on the human body 2D joint coordinate information in sequence to obtain local feature information corresponding to the human body 2D joint coordinate information, including: using a graph convolution network in the double-layer feature extraction module to perform convolution processing on the human body 2D joint coordinate information to obtain the convolved human body 2D joint coordinate information; normalizing the convolved human body 2D joint coordinate information through batch processing to obtain a normalized result; activating the normalized result through a GELU activation function to obtain an activation processing result; stacking the activation processing result into a 1*1 convolution block in sequence, and performing superposition processing of multiple local convolution layers to obtain the local feature information corresponding to the human body 2D joint coordinate information.

[0106] Furthermore, the global attention layer in the double-layer feature extraction module is used to perform global feature information extraction processing on the human body 2D joint coordinate information to obtain global feature information corresponding to the human body 2D joint coordinate information, including: calling the fully connected layer in the double-layer feature extraction module to map the input human body 2D joint coordinate information to query Q, key K and value V; inputting the query Q, key K and value V into a multi-head self-attention mechanism and dividing them into multiple heads for parallel processing, performing attention calculation processing on each head using the scaled dot product to obtain the attention calculation result corresponding to each head; extracting the global feature information of the human body 2D joint coordinate information based on the attention calculation result corresponding to each head to obtain the global feature information corresponding to the human body 2D joint coordinate information.

[0107] Specifically, a two-layer feature extraction module is used to extract local features and global features. In this technical solution, the graphic features of the human bone structure are fully considered. The local graph convolution layer is sequentially passed through GCN (graph convolution network) and batch normalization and GELU operations, and finally 1X1 convolution blocks are sequentially stacked and multiple local graph convolution layers are superimposed to extract the feature information of the local graph convolution. The feature information of the local graph convolution is expressed as follows:

[0108]

[0109] in, I is the identity matrix; yes The degree matrix of ; H is the input feature; H′ is the output feature.

[0110] That is, in the double-layer feature extraction module, the graph convolution network is used to perform convolution processing on the 2D joint coordinate information of the human body to obtain the convolved 2D joint coordinate information of the human body; the convolved 2D joint coordinate information of the human body is then normalized by batch processing to obtain a normalized result; the normalized result is activated by the GELU activation function to obtain an activated result; finally, the activated result is stacked in sequence into a 1*1 convolution block, and after multiple local convolution layers are superimposed, the local feature information corresponding to the 2D joint coordinate information of the human body is obtained.

[0111] The purpose of using GCN is to utilize the features of different adjacent joints to constrain the feature representation of the target joint, so as to capture the local dependencies between joints through this local constraint and achieve the goals of local constraint and dimensionality transformation; improve the stability of feature information through intermediate layer operations, enhance the linear expression ability of features, and improve the effectiveness of feature representation; then, perform point-to-point convolution by applying 1*1 convolution blocks to reduce the redundancy caused by local feature extraction and filter out high-dimensional or low-dimensional valid node features; finally, aggregate adjacent nodes by stacking multiple local graph convolution layers to obtain efficient feature representation.

[0112] In the extraction of global feature information, the fully connected layer is first called in the double-layer feature extraction module to map the input human 2D joint coordinate information into query Q, key K and value V; the query Q, key K and value V are input into the multi-head self-attention mechanism and divided into multiple heads for parallel processing, and the scaled dot product is used to perform attention calculation processing on each head to obtain the attention calculation result corresponding to each head; the global feature information of the human 2D joint coordinate information is extracted and processed according to the attention calculation result corresponding to each head to obtain the global feature information corresponding to the human 2D joint coordinate information.

[0113] That is, the global attention layer of the present technical solution fully considers the limitation that the feature representation of joints needs to rely on the features of other joints and their own features; the global attention layer first uses the fully connected layer to map the input into queries (Q), keys (K) and values ​​(V), and then inputs them into the multi-head self-attention mechanism, which is divided into multiple heads for parallel processing, and each head uses the scaled dot product for attention calculation; the attention layer can enhance the network model's perception of feature information, while expanding the utilization of feature information and solving the problem of severe loss of position information; by executing this process on each joint to constrain the features of all joints, the global dependency information of the human body is obtained, so that the model can effectively obtain various feature information from different positions; therefore, the global attention layer can effectively capture global dependency information and improve the modeling accuracy of human posture.

[0114] Cross-fusion module 303: used for cross-fusion processing of local feature information and global feature information corresponding to the 2D joint coordinate information of the human body to obtain cross-fusion feature information corresponding to the 2D joint coordinate information of the human body;

[0115] In the specific implementation process of the present invention, the local feature information and the global feature information corresponding to the 2D joint coordinate information of the human body are cross-fused to obtain the cross-fused feature information corresponding to the 2D joint coordinate information of the human body, including: the global feature information corresponding to the 2D joint coordinate information of the human body is used as query query1, and the local feature information is used as key key1 and value value1 to input into the first multi-head attention module to perform global relationship capture processing of the human skeleton to obtain the global feature relationship of the human skeleton; the global feature relationship of the human skeleton is input into the feedforward neural network model to perform global relationship perception processing of the human joints to obtain the global perception of the human joints. relationship; fusing the global perception relationship with the global feature information to obtain fused global feature information; based on the fully connected relationship, the global perception relationship is used as the key key2 and the value value2, and the local feature information is used as the query query2; the key key2, the value value2 and the query query2 are input into the second multi-head attention module to perform local perception relationship extraction processing to obtain the local perception relationship; fusing the local fused perception relationship with the local feature information to obtain the fused local feature information; fusing the fused global feature information and the fused local feature information to form the corresponding 2D joint coordinate information of the human body.

[0116] Specifically, the global feature information corresponding to the 2D joint coordinate information of the human body is used as the query query1, and the local feature information is used as the key key1 and the value value1 to be input into the first multi-head attention module for global relationship capture processing of the human skeleton to obtain the global feature relationship of the human skeleton; then the global feature relationship of the human skeleton is input into the feedforward neural network model for global relationship perception processing of the human joints to obtain the global perception relationship of the human joints; the global perception relationship can be fused with the global feature information to obtain fused global feature information; then the global perception relationship is used as the key key2 and the value value2 by using the fully connected relationship, and the local feature information is used as the query query2; and the key key2, the value value2 and the query query2 are input into the second multi-head attention module for local perception relationship extraction processing to obtain the local perception relationship; the local fusion perception relationship can be fused with the local feature information to obtain the fused local feature information; finally, the fused global feature information and the fused local feature information are fused to form the corresponding human 2D joint coordinate information.

[0117] That is, in this technical solution, a bidirectional feature fusion cross module is proposed to effectively process the local and global feature information obtained at the channel level; the fusion of local information is guided by global information, the global feature information is the query, and the local feature information is the key and value. The global relationship of the human skeleton captured by the attention block is fed back to the GCN block, so that the GCN can better perceive the global relationship of the human joints; at the same time, the global feature information extracted globally will also be added to each pixel of the local feature; then, the output feature information is guided by the local information to guide the fusion of information, the local feature information is the query, and the output feature information XG′ is the key and value. The local feature information relationship of the human skeleton captured by the GCN block is fed back to the output feature XG′, enhancing the understanding and representation of the model. Finally, the obtained XG′ is added to the original feature information and then cross-input into the GCN block to obtain local information, and the obtained XL′ is added to the original feature information and then cross-input into the attention block to obtain global information. Through the two-way cross-complementary information, the modeling ability of the GCN block and the attention block is enhanced.

[0118] The expression of the dual feature fusion cross module in this embodiment is:

[0119] A L→G =[Attn(Q XG ,Linear(K XL ,V XL ))];

[0120] A G→L =[Attn(Q XL ,Linear(K XG′ ,V XG′ ))];

[0121] X Fusion =Local(X L +X L′ )+Global(X G +X G′ ).

[0122] The human body posture estimation module 304 is used to perform 3D human body posture estimation processing based on the hierarchical MLP module using the cross-fusion feature information corresponding to the 2D joint coordinate information of the human body to obtain estimated 3D human body posture information.

[0123] In the specific implementation process of the present invention, the hierarchical MLP module uses the cross-fusion feature information corresponding to the 2D joint coordinate information of the human body to perform 3D human posture estimation processing to obtain estimated 3D human posture information, including: after the cross-fusion feature information corresponding to the 2D joint coordinate information of the human body is input into the hierarchical MLP module, the cross-fusion feature information is layer-normalized by layer normalization to obtain the cross-fusion feature information after layer normalization; the cross-fusion feature information after layer normalization is subjected to linear processing of the number of J nodes*half of the C channel dimension to obtain a first linear processing result; the first linear processing result is subjected to second linear and third linear processing of the number of J nodes*half of the C channel dimension in sequence to obtain a second linear processing result and a third linear processing result; the first linear processing result, the second linear processing result and the third linear processing result are subjected to linear processing of the J node*C channel dimension to obtain a fourth linear processing result; the fourth linear processing result and the input cross-fusion feature information are subjected to output processing through a fully connected layer to obtain output estimated 3D human posture information.

[0124] For details, see Figure 5 ,in Figure 5 J in the figure represents the number of nodes, and C represents the channel dimension. After the hierarchical MLP module combines local features with global features and takes them as input, it applies a linear layer to reduce the channel dimension, and then performs hierarchical linear layer acquisition to obtain feature information. After adding the channel dimension, it increases the channel dimension through a linear layer and then adds it to the original input. While acquiring important information, it can retain the original features to achieve the diversity of feature information and improve the robustness of the model. The formula of the hierarchical MLP module is:

[0125] X m1 =Linear(LN(X input ));

[0126] X m2 ,X m3 =Linear(X m1 )+X m1 ;

[0127] X output =Linear(X m1 +X m2 +X m3 )+X input ;

[0128] That is, after the cross-fusion feature information corresponding to the 2D joint coordinate information of the human body is input into the layer MLP module, the cross-fusion feature information is layer-normalized by using layer normalization to obtain the cross-fusion feature information after layer normalization; then the cross-fusion feature information after layer normalization is subjected to linear processing of the number of J nodes*half of the C channel dimension to obtain the first linear processing result; and the first linear processing result is subjected to the second linear and third linear processing of the number of J nodes*half of the C channel dimension in sequence to obtain the second linear processing result and the third linear processing result; at the same time, the first linear processing result, the second linear processing result and the third linear processing result are subjected to linear processing of the J node*C channel dimension to obtain the fourth linear processing result; finally, the fourth linear processing result and the input cross-fusion feature information are subjected to output processing through a fully connected layer to obtain the output estimated 3D human body posture information.

[0129] In an embodiment of the present invention, a double-layer feature extraction module, a bidirectional feature fusion module and a hierarchical MLP module are sequentially arranged, and after extracting the 2D joint coordinate information of the human body in the monocular image to be estimated, the double-layer feature extraction module, the bidirectional feature fusion module and the hierarchical MLP module are sequentially processed to realize 3D posture estimation; the double-layer feature extraction module can accurately extract the local feature information and the global feature information of the image; the bidirectional feature fusion module is used to realize feature interaction and fusion, and the hierarchical MLP module is used to reduce feature missing; thereby realizing higher-precision 3D human posture estimation.

[0130] A computer-readable storage medium provided in an embodiment of the present invention stores a computer program on the computer-readable storage medium, and when the program is executed by a processor, a monocular 3D posture estimation method of any one of the above embodiments is implemented. Wherein, the computer-readable storage medium includes but is not limited to any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (EraSable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic card or optical card. That is, the storage device includes any medium that can store or transmit information in a readable form by a device (for example, a computer, a mobile phone), which can be a read-only memory, a disk or an optical disk, etc.

[0131] An embodiment of the present invention further provides a computer application, which runs on a computer and is used to execute the monocular 3D posture estimation method of any one of the above embodiments.

[0132] also, Figure 4 It is a schematic diagram of the structure of an electronic device in an embodiment of the present invention.

[0133] The embodiment of the present invention further provides an electronic device, such as Figure 4 The electronic device includes a processor 402, a memory 403, an input unit 404, a display unit 405 and other devices. Those skilled in the art will understand that Figure 4 The structural components of the electronic device shown do not constitute a limitation on all devices, and may include more or fewer components than shown, or combine certain components. The memory 403 can be used to store the application 401 and various functional modules, and the processor 402 runs the application 401 stored in the memory 403, thereby executing various functional applications and data processing of the device. The memory can be an internal memory or an external memory, or include both internal and external memories. The internal memory may include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, or a random access memory. The external memory may include a hard disk, a floppy disk, a ZIP disk, a U disk, a magnetic tape, etc. The memory disclosed in the present invention includes but is not limited to these types of memories. The memory disclosed in the present invention is only used as an example and not as a limitation.

[0134] The input unit 404 is used to receive the input of the signal and the keyword input by the user. The input unit 404 may include a touch panel and other input devices. The touch panel can collect the user's touch operation on or near it (such as the user's operation on or near the touch panel using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program; other input devices may include but are not limited to one or more of a physical keyboard, a function key (such as a playback control key, a switch key, etc.), a trackball, a mouse, a joystick, etc. The display unit 405 can be used to display the information input by the user or the information provided to the user and various menus of the terminal device. The display unit 405 can be in the form of a liquid crystal display, an organic light emitting diode, etc. The processor 402 is the control center of the terminal device, which uses various interfaces and lines to connect the various parts of the entire device, and executes various functions and processes data by running or executing software programs and / or modules stored in the memory 403, and calling the data stored in the memory.

[0135] As an embodiment, the electronic device includes: one or more processors 402, a memory 403, and one or more applications 401, wherein the one or more applications 401 are stored in the memory 403 and are configured to be executed by the one or more processors 402, and the one or more applications 401 are configured to execute the corresponding monocular 3D posture estimation method in any one of the above-mentioned embodiments.

[0136] In an embodiment of the present invention, a double-layer feature extraction module, a bidirectional feature fusion module and a hierarchical MLP module are sequentially arranged, and after extracting the 2D joint coordinate information of the human body in the monocular image to be estimated, the double-layer feature extraction module, the bidirectional feature fusion module and the hierarchical MLP module are sequentially processed to realize 3D posture estimation; the double-layer feature extraction module can accurately extract the local feature information and the global feature information of the image; the bidirectional feature fusion module is used to realize feature interaction and fusion, and the hierarchical MLP module is used to reduce feature missing; thereby realizing higher-precision 3D human posture estimation.

[0137] In addition, the above is a detailed introduction to a monocular 3D posture estimation method based on a cross network and related devices provided in an embodiment of the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A monocular 3D pose estimation method based on a cross network, characterized in that: The method comprises: Receiving a monocular image to be estimated, and performing human body joint coordinate extraction processing on the monocular image to be estimated using a 2D posture model to obtain human body 2D joint coordinate information corresponding to the monocular image to be estimated; Using a double-layer feature extraction module to perform local and global feature information extraction processing on the 2D joint coordinate information of the human body, to obtain local feature information and global feature information corresponding to the 2D joint coordinate information of the human body; Cross-fusion processing is performed on the local feature information and the global feature information corresponding to the 2D joint coordinate information of the human body to obtain the cross-fusion feature information corresponding to the 2D joint coordinate information of the human body; Based on the hierarchical MLP module, 3D human body posture estimation processing is performed using the cross-fusion feature information corresponding to the 2D joint coordinate information of the human body to obtain estimated 3D human body posture information.

2. The monocular 3D posture estimation method according to claim 1, characterized in that: The method of performing human joint coordinate extraction processing on the monocular image to be estimated by using the 2D posture model to obtain human 2D joint coordinate information corresponding to the monocular image to be estimated includes: The monocular image to be estimated is input into the 2D posture model, and the joint coordinates of the target human body in the monocular image to be estimated are extracted and processed in the 2D posture model to obtain the human body 2D joint coordinate information corresponding to the monocular image to be estimated.

3. The monocular 3D posture estimation method according to claim 1, characterized in that: The method of using a double-layer feature extraction module to perform local and global feature information extraction processing on the 2D joint coordinate information of the human body to obtain local feature information and global feature information corresponding to the 2D joint coordinate information of the human body includes: After the human body 2D joint coordinate information is input into the double-layer feature extraction module, the local graph convolution layer in the double-layer feature extraction module is used to sequentially perform local feature extraction processing on the human body 2D joint coordinate information to obtain local feature information corresponding to the human body 2D joint coordinate information; The global attention layer in the two-layer feature extraction module is used to perform global feature information extraction processing on the 2D joint coordinate information of the human body to obtain global feature information corresponding to the 2D joint coordinate information of the human body.

4. The monocular 3D posture estimation method according to claim 3, characterized in that: The method of using the local graph convolution layer in the double-layer feature extraction module to sequentially perform local feature extraction processing on the 2D joint coordinate information of the human body to obtain local feature information corresponding to the 2D joint coordinate information of the human body includes: In the double-layer feature extraction module, a graph convolution network is used to perform convolution processing on the 2D joint coordinate information of the human body to obtain the convolved 2D joint coordinate information of the human body; The convolved 2D joint coordinate information of the human body is normalized through batch processing to obtain a normalized result; The normalized result is activated by a GELU activation function to obtain an activation result; The activation processing results are stacked in sequence into a 1*1 convolution block, and after multiple local convolution layer stacking processes, local feature information corresponding to the 2D joint coordinate information of the human body is obtained.

5. The monocular 3D posture estimation method according to claim 3, characterized in that: The method of performing global feature information extraction processing on the 2D joint coordinate information of the human body by using the global attention layer in the two-layer feature extraction module to obtain global feature information corresponding to the 2D joint coordinate information of the human body includes: In the two-layer feature extraction module, the fully connected layer is called to map the input human body 2D joint coordinate information into a query Q, a key K and a value V; The query Q, key K and value V are input into the multi-head self-attention mechanism and processed in parallel by multiple heads. The scaled dot product is used to perform attention calculation in each head to obtain the attention calculation result corresponding to each head. Based on the attention calculation results corresponding to each head, the global feature information of the 2D joint coordinate information of the human body is extracted and processed to obtain the global feature information corresponding to the 2D joint coordinate information of the human body.

6. The monocular 3D posture estimation method according to claim 1, characterized in that: The cross-fusion processing of the local feature information and the global feature information corresponding to the 2D joint coordinate information of the human body to obtain the cross-fusion feature information corresponding to the 2D joint coordinate information of the human body includes: The global feature information corresponding to the 2D joint coordinate information of the human body is used as query query1, and the local feature information is used as key key1 and value value1 to input into the first multi-head attention module to perform global relationship capture processing of the human skeleton, so as to obtain the global feature relationship of the human skeleton; Inputting the global characteristic relationship of the human skeleton into a feedforward neural network model to perform global relationship perception processing of human joints to obtain a global perception relationship of human joints; Fusing the global perception relationship with the global feature information to obtain fused global feature information; Based on the fully connected relationship, the global perception relationship is used as the key key2 and the value value2, and the local feature information is used as the query query2; Input the key key2, the value value2 and the query query2 into the second multi-head attention module for local perception relationship extraction processing to obtain the local perception relationship; Fusing the local fusion perception relationship with the local feature information to obtain fused local feature information; The fused global feature information and the fused local feature information are fused to form the corresponding 2D joint coordinate information of the human body.

7. The monocular 3D posture estimation method according to claim 1, characterized in that: The hierarchical MLP module performs 3D human posture estimation processing using the cross-fusion feature information corresponding to the 2D joint coordinate information of the human body to obtain estimated 3D human posture information, including: After the cross-fusion feature information corresponding to the 2D joint coordinate information of the human body is input into the hierarchical MLP module, layer normalization is used to perform layer normalization processing on the cross-fusion feature information to obtain the cross-fusion feature information after layer normalization; The cross-fusion feature information after layer normalization is subjected to linear processing of the number of J nodes*half the dimension of C channels to obtain the first linear processing result; The first linear processing result is sequentially subjected to second linear processing and third linear processing of the dimension of J node number*half C channel to obtain a second linear processing result and a third linear processing result; The first linear processing result, the second linear processing result and the third linear processing result are subjected to linear processing of the J node*C channel dimension to obtain a fourth linear processing result; The fourth linear processing result and the input cross-fusion feature information are output through a fully connected layer to obtain output estimated 3D human body posture information.

8. A monocular 3D posture estimation device based on a cross network, characterized in that: The device comprises: The human body joint coordinate extraction module is used to receive the monocular image to be estimated, and use the 2D posture model to perform human body joint coordinate extraction processing on the monocular image to be estimated, so as to obtain the human body 2D joint coordinate information corresponding to the monocular image to be estimated; Feature information extraction module: used to perform local and global feature information extraction processing on the 2D joint coordinate information of the human body using a double-layer feature extraction module to obtain local feature information and global feature information corresponding to the 2D joint coordinate information of the human body; Cross-fusion module: used for cross-fusion processing of local feature information and global feature information corresponding to the 2D joint coordinate information of the human body, so as to obtain cross-fusion feature information corresponding to the 2D joint coordinate information of the human body; Human body posture estimation module: used to perform 3D human body posture estimation processing based on the hierarchical MLP module using the cross-fusion feature information corresponding to the 2D joint coordinate information of the human body to obtain estimated 3D human body posture information.

9. An electronic device, comprising a processor and a memory, characterized in that: The processor runs a computer program or code stored in the memory to implement the monocular 3D posture estimation method according to any one of claims 1 to 7.

10. A computer-readable storage medium for storing a computer program or code, characterized in that: When the computer program or code is executed by a processor, the monocular 3D pose estimation method according to any one of claims 1 to 7 is implemented.

Images