Picture-based 3D virtual model posture generation method and system
By extracting key points and bone information from the input pictures, and using deep neural networks to generate virtual model poses, the problems of complex construction of virtual models and high equipment cost are solved, and efficient and low-cost virtual model pose generation is achieved.
Patent Information
- Application Number
- CN202310769595.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-27
- Publication Date
- 2025-01-03
AI Technical Summary
The construction of virtual models in the prior art is complex, requiring advanced modeling and animation techniques, and adjusting postures with expensive sensors or capture devices is costly and complex.
By obtaining the key points of the human body in the input picture, using deep neural network to predict the rotation information and relative displacement of the bone node, building and analyzing bones, and mapping them to the 3D virtual model model, realizing posture redirection, and the generation process is simple and efficient, avoiding the Use complex equipment.
The generation of 3D virtual model poses based on pictures is realized, which reduces costs and simplifies the process without manual adjustments and reduces external inputs. The generated virtual model poses are highly realistic and simple to use.
Smart Images
Figure CN119251360A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of three-dimensional models, and in particular to a method and system for generating a 3D virtual model posture based on a picture. Background Art
[0002] A 3D model is a polygonal representation of an object, usually displayed by a computer or other video device. Anything that exists in the physical world can be represented by a 3D model. With the continuous development of computer technology, 3D models have become an important tool in all walks of life. In the fields of games, movies, architecture, product design, etc., 3D models are widely used to create realistic scenes, visual effects and interactive experiences.
[0003] A virtual model is a specific application of a 3D model. It refers to a model generated using computer technology that can change clothes and perform in a digital environment. This technology can be used for clothing design and sales, as well as for entertainment and artistic performances. Virtual models are usually created through 3D modeling and animation technology, and can be produced and edited using a variety of software tools. Compared with real models, virtual models are not restricted by time and place, and can change clothes and perform at any time, while also allowing for more accurate data analysis and testing. Therefore, virtual models play an increasingly important role in the clothing industry in the digital age.
[0004] However, building avatars is extremely complex for several reasons:
[0005] Complex modeling technology: Virtual models require a fine three-dimensional model to ensure their realism. In the modeling process, it is necessary to master advanced modeling techniques, such as Subdivision Surface, Free Form deformation, etc., to create a fine model.
[0006] Complex animation technology: Virtual models need to perform various actions, such as walking, running, jumping, etc. During the animation process, it is necessary to master various animation techniques, such as keyframe animation, skeletal animation, muscle system, etc.
[0007] Huge computing resources: Since virtual models need to process a large amount of data and algorithms, they require huge computing resources, such as multi-core processors, high-performance graphics cards, etc.
[0008] In the prior art, there are methods of adjusting the posture of virtual models with the help of sensors or capture devices, but these sensors and capture devices are extremely expensive and complicated to use, and require constant detailed adjustments, such as adjusting facial expressions, the position of arms and legs, folds and textures of clothes, etc., to ensure the realism and aesthetics of the virtual model. Summary of the invention
[0009] In view of this, the present invention proposes a method and system for generating 3D virtual model poses based on images, and the specific scheme is as follows:
[0010] A method for generating a 3D virtual model pose based on an image, comprising:
[0011] Get an input image containing a complete human body structure;
[0012] Predicting key points of various parts of the human body based on the input image, extracting rotation information of the human body's skeletal nodes and relative displacement in three-dimensional space relative to their parent nodes from each key point, so as to analyze and obtain a posture representation of the human body;
[0013] Based on the posture representation and the input image, construct an analytical skeleton consisting of a series of joints and bones that can reflect the body structure and posture of the human body;
[0014] A mapping relationship between the parsed skeleton and the preset 3D virtual model is pre-specified, and based on the mapping relationship, the morphology of the parsed skeleton is mapped to the skeleton of the 3D virtual model model through a skeleton posture redirection method to obtain the posture of the 3D virtual model.
[0015] In a specific embodiment, if at least two human body structures are parsed based on the predicted key points, the key points are segmented so that each human body structure corresponds to a set of key points.
[0016] In a specific embodiment, the acquisition of the key points specifically includes:
[0017] Filtering pixel data related to the human body from the input image, and extracting human features from the pixel data using a preset deep neural network;
[0018] Based on human features, human body parts including the head, torso, and limbs are predicted, and the main bone nodes are selected as key points based on the human body parts to determine the 2D coordinates of the key points.
[0019] In a specific embodiment, the acquisition of the relative displacement includes:
[0020] A human joint tree structure is predefined, which uses a displacement vector to represent the position of each bone node relative to its parent node. The direction of the displacement vector represents the direction in which each bone node points to its parent node, and the magnitude of the displacement vector represents the distance between each bone node and its parent node.
[0021] Based on the 2D coordinates of two adjacent key points, calculate the displacement of each bone node relative to its parent node in the two-dimensional space to obtain the two-dimensional displacement data;
[0022] The two-dimensional displacement data is converted into relative displacement in three-dimensional space based on deep learning.
[0023] In a specific embodiment, the acquisition of the rotation information includes:
[0024] Define a global coordinate system for the entire human body in three-dimensional space;
[0025] Predict the displacement of each bone node relative to its parent joint, and derive the position of each bone node in the global coordinate system by combining these displacements;
[0026] After knowing the position of each bone node, the rotation of each bone node in the global coordinate system can be obtained by predicting the rotation of each bone node relative to its parent joint.
[0027] In a specific embodiment, the 3D avatar model is composed of a mesh and a skeleton, the mesh includes vertices and faces, each vertex is bound to a skeleton and has a certain weight;
[0028] Map the morphology of the analyzed skeleton to the skeleton of the 3D avatar so that the two have the same or similar postures;
[0029] According to the weight of each vertex and the bone it belongs to, the new position of each vertex is calculated, and the shape of the 3D virtual model is updated to obtain the final pose of the 3D virtual model.
[0030] A 3D virtual model pose generation system based on an image, comprising:
[0031] An input unit, used for obtaining an input image containing a complete human body structure;
[0032] A posture analysis unit, used to predict key points about various parts of the human body based on the input image, and extract rotation information of the human body's skeletal nodes and relative displacement in three-dimensional space relative to their parent nodes from each key point, so as to analyze and obtain a posture representation of the human body;
[0033] A parsing skeleton unit, used for constructing a parsing skeleton consisting of a series of joints and bones and capable of reflecting the body structure and posture of a human body based on the posture representation and the input image;
[0034] The output unit is used to pre-specify the mapping relationship between the parsed skeleton and the preset 3D virtual model, and map the morphology of the parsed skeleton to the skeleton of the 3D virtual model model through a skeleton posture redirection method based on the mapping relationship to obtain the posture of the 3D virtual model.
[0035] In a specific embodiment, the posture analysis unit includes:
[0036] A node estimation network is used to extract human features from pixel data related to the human body in the input image through multi-layer convolution, activation and pooling operations; predict human body parts including the head, torso, and limbs based on the human features, and select main bone nodes as key points based on the human body parts, and determine the 2D coordinates of the key points;
[0037] The posture estimation network is used to extract the rotation information of the human body's skeletal nodes and the relative displacement in three-dimensional space relative to its parent node from the key points.
[0038] In a specific embodiment, the acquisition of the relative displacement includes: predefining a human joint tree structure that uses a displacement vector to represent the position of each bone node relative to its parent node, the direction of the displacement vector represents the direction of each bone node pointing to its parent node, and the magnitude of the displacement vector represents the distance between each bone node and its parent node; based on the 2D coordinates of two adjacent key points, calculating the displacement of each bone node relative to its parent node in two-dimensional space to obtain two-dimensional displacement data; and converting the two-dimensional displacement data into relative displacement in three-dimensional space based on deep learning.
[0039] In a specific embodiment, the acquisition of the rotation information includes: defining a global coordinate system for the entire human body in three-dimensional space; predicting the displacement of each bone node relative to its parent joint, and deriving the position of each bone node in the global coordinate system by merging these displacements; after knowing the position of each bone node, by predicting the rotation of each bone node relative to its parent joint, the rotation of each bone node in the global coordinate system can be obtained.
[0040] Beneficial effects:
[0041] The present invention provides a method and system for generating 3D virtual model postures based on pictures, which utilizes the postures of characters in input pictures to drive the postures of 3D virtual models, and predicts posture representations with the help of deep neural networks, so that the 3D virtual model can reproduce the postures of characters in input pictures, and the generation process is simple and efficient, and the use cost is low. In addition, there is no need to use complex sensors or capture devices, manually standardize pictures, or manually adjust 3D virtual model posture data, and external input is reduced as much as possible, and only the input pictures are retained as drivers. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 is a flow chart of a posture generation method according to an embodiment of the present invention;
[0043] Figure 2 is a schematic diagram of the principle of a posture generation method according to an embodiment of the present invention;
[0044] Figure 3 is a schematic diagram of the analyzed bone structure according to an embodiment of the present invention;
[0045] Figure 4 is a structural diagram of a 3D virtual model according to an embodiment of the present invention;
[0046] Figure 5 yes Figure 3 and Figure 4 Schematic diagram of the combined 3D virtual model pose;
[0047] Figure 6 2 is a schematic diagram of a gesture generation system module according to an embodiment of the present invention.
[0048] Figure numerals: 1 - input unit; 2 - posture analysis unit; 3 - skeleton analysis unit; 4 - output unit. DETAILED DESCRIPTION
[0049] In the following, various embodiments of the present disclosure will be described more fully. The present disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of the present disclosure to the specific embodiments disclosed herein, but the present disclosure should be understood to cover all adjustments, equivalents and / or alternatives that fall within the spirit and scope of the various embodiments of the present disclosure.
[0050] Example 1
[0051] Embodiment 1 of the present invention discloses a method for generating a 3D virtual model posture based on an image, which uses the posture of a person in an input image to drive the posture of a 3D virtual model. The solution is simple and efficient, and the cost of use is low. The specific process is as shown in the attached manual. Figure 1 The specific plan is as follows:
[0052] A method for generating a 3D virtual model pose based on an image comprises the following steps:
[0053] 101. Obtain an input image containing a complete human body structure;
[0054] 102. Predict key points of various parts of the human body based on the input image, extract the rotation information of the human body's skeletal nodes and the relative displacement of their parent nodes in three-dimensional space from each key point, so as to analyze and obtain the posture representation of the human body;
[0055] 103. Based on the posture representation and input image, construct an analytical skeleton consisting of a series of joints and bones that can reflect the body structure and posture of the human body;
[0056] 104. Pre-specify a mapping relationship between the parsed skeleton and the preset 3D virtual model, and map the morphology of the parsed skeleton to the skeleton of the 3D virtual model model through a skeleton posture redirection method based on the mapping relationship to obtain the posture of the 3D virtual model.
[0057] The 3D virtual model posture generation method of this embodiment utilizes the character posture in the input image to drive the virtual model to generate the corresponding posture, and uses a deep neural network to predict the human body posture. The virtual model posture can be constructed and adjusted without expensive sensors or capture devices.
[0058] Specifically, the solution of this embodiment is essentially to reproduce the human posture in the input image, so it is necessary to limit the input image to ensure that the input image has a complete human body structure and cannot lack the main body structure, such as limbs. The solution of this embodiment needs to use a deep neural network to predict the posture data from the complete human body structure and then apply it to the virtual model. If the human body structure in the image is incomplete and parts such as limbs are missing, it will lead to the inability to find all the bone nodes, and thus the inability to accurately reproduce the posture of the person in the image.
[0059] Preferably, this embodiment uses a deep neural network to parse the characters from the input image and further converts it into a posture representation. The deep neural network has two sub-networks. One is the human joint point estimation network in 2D space, which extracts useful features from pixel data through multi-layer convolution, activation and pooling operations, and then uses these features to predict the key point positions of various parts of the human body. The second is the posture estimation network, which is used to extract the relative displacement and rotation information of the character's skeletal nodes to their parent nodes in three-dimensional space from the key points.
[0060] There may be more than one human body structure in the input image. In this case, it is necessary to parse them one by one and segment each complete human body structure to facilitate the subsequent steps. Preferably, if at least two human body structures are parsed based on the predicted key points, the key points are segmented so that each human body structure corresponds to a set of key points. As for which set of key points to choose to correspond to the human body structure applied to the virtual model, it is up to the user to choose. The user can select one of the sets of key points, or apply different sets of key points to different virtual models, or apply each set of key points to the virtual model in turn.
[0061] Among them, the acquisition of key points specifically includes: screening out pixel data related to the human body in the input image, extracting human features from the pixel data through a deep neural network; predicting human body parts including the head, torso, and limbs based on human features, and screening out the main bone nodes as key points based on human body parts, and determining the 2D coordinates of the key points. There may be data unrelated to the human body structure in the input image, so it is necessary to first screen out pixel data related to the human body. Use the joint point estimation network to extract relevant human features from the pixel data, and then predict the main parts of the human body structure based on the human features. The main parts include the head, torso, and limbs. These main parts can form a basic human structure, and the main posture of the human body is also inseparable from the participation of these parts.
[0062] A 3D virtual model posture generation method, wherein the relative displacement is obtained by predefining a human joint tree structure in which a displacement vector is used to represent the position of each bone node relative to its parent node, wherein the direction of the displacement vector represents the direction in which each bone node points to its parent node, and the magnitude of the displacement vector represents the distance between each bone node and its parent node; based on the 2D coordinates of two adjacent key points, the displacement of each bone node relative to its parent node in two-dimensional space is calculated to obtain two-dimensional displacement data; and based on deep learning, the two-dimensional displacement data is converted into relative displacement in three-dimensional space.
[0063] The relative displacement of human posture is based on a predefined human joint tree structure. In this joint tree, the position of each joint (node) relative to its parent joint (parent node) is represented by a relative displacement vector. For example, the displacement of the elbow joint relative to the shoulder joint is represented by a vector, the direction of the vector points from the shoulder to the elbow, and the magnitude is the distance between them.
[0064] Among them, the acquisition of rotation information includes: defining a global coordinate system for the entire human body in three-dimensional space; predicting the displacement of each bone node relative to its parent joint, and merging these displacements to obtain the position of each bone node in the global coordinate system; after knowing the position of each bone node, by predicting the rotation of each bone node relative to its parent joint, the rotation of each bone node in the global coordinate system can be obtained.
[0065] In the process of posture estimation, the neural network predicts the displacement of each joint relative to its parent joint, and by combining these displacements, the position of each joint in the global coordinate system is obtained. Subsequently, by predicting the rotation of each joint relative to its parent joint, the global rotation of each joint can be obtained. This method based on relative displacement and rotation of the joint tree can effectively handle various complex postures and perspectives, and can accurately reflect the relative position and direction relationship between various parts of the human body.
[0066] A joint is a bone node, and its rotation information cannot be measured by its parent node, but should be placed in the entire human body structure. Therefore, this embodiment defines a global coordinate system, which is constructed based on the entire human body structure. Any change in each joint will produce a corresponding coordinate change in the global coordinate system.
[0067] This embodiment constructs a corresponding parsed skeleton based on the posture representation obtained from the parsed image. Specifically, the parsed skeleton is composed of a series of joints and bones, each joint corresponds to a world coordinate, and each bone corresponds to the line between two joints. The parsed skeleton can reflect the body structure and posture changes of the character and is consistent with the input image.
[0068] Instructions attached Figure 3 A structural example diagram of an analytical skeleton is given. The skeleton nodes in the analytical skeleton include: 2 nodes located in the head, 8 nodes located in the upper limbs, 6 nodes located in the trunk, and 8 nodes located in the lower limbs.
[0069] Finally, the parsed skeleton is applied to a pre-set 3D avatar model to generate the 3D avatar model pose. The 3D avatar model consists of a mesh and a skeleton. The mesh contains vertices and faces. Each vertex is bound to a skeleton and has a certain weight. The morphology of the parsed skeleton is mapped to the skeleton of the 3D avatar so that the two have the same or similar postures. According to the weight of each vertex and the skeleton it belongs to, the new position of each vertex is calculated, and the shape of the 3D avatar is updated to obtain the final 3D avatar pose. Figure 4 A schematic diagram of a 3D virtual model is given. Figure 5 for Figure 3 The analytical skeleton and Figure 4 The pose generated by combining the 3D virtual model.
[0070] Since the structure and topology of the 3D virtual model skeleton are different from the parsed skeleton, this embodiment adopts a skeleton pose redirection method. Skeleton pose redirection is a process of applying the action of one skeleton model to another skeleton model. Through skeleton pose redirection, it is possible to apply the posture / action of one character to another character without changing the skeleton structure. Because the parsed skeleton result is the same topology, the shape of the parsed skeleton can be mapped to the 3D virtual model model skeleton through the skeleton pose redirection method by pre-specifying the mapping relationship between the parsed skeleton and the 3D virtual model model skeleton, so that the two have the same / similar posture. Finally, according to the weight of each vertex and the skeleton to which it belongs, the new position of each vertex is calculated, and the shape of the 3D virtual model model is updated to obtain the final result. Appendix Figure 3 Analysis of bones and appendages Figure 4The 3D avatar has inconsistent bone topology. Figure 3 The skeleton nodes are parsed from the image, which only contain the basic human frame. The hands do not have skeleton details. The 3D virtual model skeleton contains the bones of each joint of the hand, so the topology is inconsistent. Through skeleton posture redirection, the parsed skeleton posture is transferred to the 3D virtual model skeleton to obtain the attached Figure 5 result.
[0071] This embodiment provides a method for generating a 3D virtual model posture based on an image, which uses the posture of a person in an input image to drive the posture of a 3D virtual model, and predicts the posture representation with the help of a deep neural network, so that the 3D virtual model can reproduce the posture of the person in the input image. The generation process is simple and efficient, and the cost of use is low. In addition, there is no need to use complex sensors or capture devices, manually standardize the image, or manually adjust the 3D virtual model posture data. External input is reduced as much as possible, and only the input image is retained as a driver.
[0072] Example 2
[0073] Embodiment 2 of the present invention discloses a 3D virtual model pose generation system based on an image, which systematizes the 3D virtual model pose generation method based on an image in Embodiment 1 to make it more practical. The overall structure diagram of the 3D virtual model pose generation system based on an image is shown in the attached specification. Figure 6 The specific plan is as follows:
[0074] A 3D virtual model pose generation system based on an image, comprising:
[0075] An input unit 1, used for obtaining an input image containing a complete human body structure;
[0076] Posture analysis unit 2, used for predicting key points about various parts of the human body based on the input image, extracting the rotation information of the human body's skeletal nodes and the relative displacement in three-dimensional space relative to its parent node from each key point, so as to analyze and obtain the human body's posture representation;
[0077] A parsing skeleton unit 3, which is used to construct a parsing skeleton consisting of a series of joints and bones and can reflect the body structure and posture of the human body based on the posture representation and the input image;
[0078] The output unit 4 is used to pre-specify the mapping relationship between the parsed skeleton and the preset 3D virtual model, and map the morphology of the parsed skeleton to the skeleton of the 3D virtual model model through the skeleton posture redirection method based on the mapping relationship to obtain the posture of the 3D virtual model.
[0079] Wherein, the posture analysis unit 2 comprises:
[0080] The node estimation network is used to extract human features from pixel data related to the human body in the input image through multi-layer convolution, activation and pooling operations; predict human body parts including head, torso, and limbs based on human features, and select the main bone nodes as key points based on the human body parts to determine the 2D coordinates of the key points.
[0081] The posture estimation network is used to extract the rotation information of the human skeleton nodes and the relative displacement in three-dimensional space relative to their parent nodes from the key points;
[0082] The acquisition of relative displacement includes: predefining a human joint tree structure that uses a displacement vector to represent the position of each bone node relative to its parent node, the direction of the displacement vector represents the direction of each bone node pointing to its parent node, and the magnitude of the displacement vector represents the distance between each bone node and its parent node; based on the 2D coordinates of two adjacent key points, calculating the displacement of each bone node relative to its parent node in two-dimensional space to obtain two-dimensional displacement data; based on deep learning, converting the two-dimensional displacement data into relative displacement in three-dimensional space;
[0083] The acquisition of rotation information includes: defining a global coordinate system for the entire human body in three-dimensional space; predicting the displacement of each bone node relative to its parent joint, and merging these displacements to obtain the position of each bone node in the global coordinate system; after knowing the position of each bone node, by predicting the rotation of each bone node relative to its parent joint, the rotation of each bone node in the global coordinate system can be obtained.
[0084] The present invention provides a method and system for generating 3D virtual model postures based on pictures, which utilizes the postures of characters in input pictures to drive the postures of 3D virtual models, and predicts posture representations with the help of deep neural networks, so that the 3D virtual model can reproduce the postures of characters in input pictures, and the generation process is simple and efficient, and the use cost is low. In addition, there is no need to use complex sensors or capture devices, manually standardize pictures, or manually adjust 3D virtual model posture data, and external input is reduced as much as possible, and only the input pictures are retained as drivers.
[0085] Those skilled in the art will appreciate that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily required for implementing the present invention. Those skilled in the art will appreciate that the modules in the devices in the implementation scenario may be distributed in the devices in the implementation scenario according to the description of the implementation scenario, or may be changed accordingly and located in one or more devices different from the present implementation scenario.
Claims
1. A method for generating a 3D virtual model pose based on an image, characterized in that: include: Get an input image containing a complete human body structure; Predicting key points of various parts of the human body based on the input image, extracting rotation information of the human body's skeletal nodes and relative displacement in three-dimensional space relative to their parent nodes from each key point, so as to analyze and obtain a posture representation of the human body; Based on the posture representation and the input image, construct an analytical skeleton consisting of a series of joints and bones that can reflect the body structure and posture of the human body; A mapping relationship between the parsed skeleton and a preset 3D virtual model is pre-specified, and based on the mapping relationship, the morphology of the parsed skeleton is mapped to the skeleton of the 3D virtual model model through a skeleton posture redirection method to obtain the posture of the 3D virtual model.
2. The 3D virtual model posture generation method according to claim 1, characterized in that: If at least two human structures are parsed based on the predicted key points, the key points are segmented so that each human structure corresponds to a set of key points.
3. The 3D virtual model posture generation method according to claim 1, characterized in that: The acquisition of the key points specifically includes: Filtering pixel data related to the human body from the input image, and extracting human features from the pixel data using a preset deep neural network; Based on human features, human body parts including the head, torso, and limbs are predicted, and the main bone nodes are selected as key points based on the human body parts to determine the 2D coordinates of the key points.
4. The 3D virtual model posture generation method according to claim 1, characterized in that: The acquisition of the relative displacement includes: A human joint tree structure is predefined, which uses a displacement vector to represent the position of each bone node relative to its parent node. The direction of the displacement vector represents the direction in which each bone node points to its parent node, and the magnitude of the displacement vector represents the distance between each bone node and its parent node. Based on the 2D coordinates of two adjacent key points, calculate the displacement of each bone node relative to its parent node in the two-dimensional space to obtain the two-dimensional displacement data; The two-dimensional displacement data is converted into relative displacement in three-dimensional space based on deep learning.
5. The 3D virtual model posture generation method according to claim 1, characterized in that: The acquisition of the rotation information includes: Define a global coordinate system for the entire human body in three-dimensional space; Predict the displacement of each bone node relative to its parent joint, and derive the position of each bone node in the global coordinate system by combining these displacements; After knowing the position of each bone node, the rotation of each bone node in the global coordinate system can be obtained by predicting the rotation of each bone node relative to its parent joint.
6. The 3D virtual model posture generation method according to claim 1, characterized in that: The 3D virtual model is composed of a mesh and a skeleton, the mesh contains vertices and faces, each vertex is bound to a skeleton and has a certain weight; Map the morphology of the analyzed skeleton to the skeleton of the 3D avatar so that the two have the same or similar postures; According to the weight of each vertex and the bone it belongs to, the new position of each vertex is calculated, and the shape of the 3D virtual model is updated to obtain the final pose of the 3D virtual model.
7. A 3D virtual model pose generation system based on an image, characterized in that: include: An input unit, used for obtaining an input image containing a complete human body structure; A posture analysis unit, used to predict key points about various parts of the human body based on the input image, extract rotation information of the human body's skeletal nodes and relative displacement in three-dimensional space relative to their parent nodes from each key point, so as to analyze and obtain a posture representation of the human body; A parsing skeleton unit, used for constructing a parsing skeleton consisting of a series of joints and bones and capable of reflecting the body structure and posture of a human body based on the posture representation and the input image; The output unit is used to pre-specify the mapping relationship between the parsed skeleton and the preset 3D virtual model, and map the morphology of the parsed skeleton to the skeleton of the 3D virtual model model through a skeleton posture redirection method based on the mapping relationship to obtain the posture of the 3D virtual model.
8. The 3D virtual model pose generation system according to claim 7, characterized in that: The posture analysis unit comprises: A node estimation network is used to extract human features from pixel data related to the human body in the input image through multi-layer convolution, activation and pooling operations; predict human body parts including the head, torso, and limbs based on the human features, and select main bone nodes as key points based on the human body parts, and determine the 2D coordinates of the key points; The posture estimation network is used to extract the rotation information of the human body's skeletal nodes and the relative displacement in three-dimensional space relative to its parent node from the key points.
9. The 3D virtual model pose generation system according to claim 8, characterized in that: The acquisition of the relative displacement includes: predefining a human joint tree structure that uses a displacement vector to represent the position of each bone node relative to its parent node, the direction of the displacement vector represents the direction in which each bone node points to its parent node, and the magnitude of the displacement vector represents the distance between each bone node and its parent node; based on the 2D coordinates of two adjacent key points, calculating the displacement of each bone node relative to its parent node in two-dimensional space to obtain two-dimensional displacement data; and converting the two-dimensional displacement data into relative displacement in three-dimensional space based on deep learning.
10. The 3D virtual model pose generation system according to claim 8, characterized in that: The acquisition of the rotation information includes: defining a global coordinate system for the entire human body in three-dimensional space; predicting the displacement of each bone node relative to its parent joint, and deriving the position of each bone node in the global coordinate system by merging these displacements; after knowing the position of each bone node, by predicting the rotation of each bone node relative to its parent joint, the rotation of each bone node in the global coordinate system can be obtained.