Three-dimensional virtual model driving processing method, device, equipment and storage medium

Abstract

The present application provides a driving processing method, device, equipment and storage medium of a three-dimensional virtual model, the method comprising: obtaining the coordinate values ​​of the skeletal key points of the target object in the input image in the three-dimensional coordinate system; determining the first rotation matrix of the multiple bones of the target object according to the coordinate values ​​of the skeletal key points corresponding to the multiple skeletal key points, wherein there is a skeletal topological connection relationship between the multiple bones; determining the second rotation matrix corresponding to the multiple bones according to the skeletal topological connection relationship and the first rotation matrix; redirecting the second rotation matrix corresponding to the multiple bones to obtain the third rotation matrix corresponding to the multiple bones; driving the initial three-dimensional virtual model to perform posture changes according to the third rotation matrix corresponding to the multiple bones to obtain the target three-dimensional virtual model corresponding to the target object. The corresponding target three-dimensional virtual model is generated according to the input image of the user.

Description

Technical Field

[0001] The present invention relates to the technical field of virtual model processing, and in particular to a drive processing method, device, equipment and storage medium for a three-dimensional virtual model. Background Art

[0002] With the continuous development and progress of artificial intelligence technology and the rapid development of related technologies of three-dimensional virtual images, more and more users want to appear in online social activity scenarios (for example, virtual interactive games) in the form of virtual images rather than using their real-life photos.

[0003] In the face of the above application requirements, how to generate one or more different three-dimensional virtual image effects in a virtual interactive game scene based on the real-life photos input by the user to enhance the user's interactive experience of the virtual interactive game is a technical problem that has not yet been solved. Summary of the invention

[0004] The embodiments of the present invention provide a driving processing method, device, equipment and storage medium for a three-dimensional virtual model, so as to generate a corresponding target three-dimensional virtual model according to a user's input image in a virtual interactive game scene, thereby improving the user's game interactive experience.

[0005] In a first aspect, an embodiment of the present invention provides a driving processing method for a three-dimensional virtual model, the method comprising:

[0006] Obtaining the coordinate values ​​of the skeleton key points corresponding to the multiple skeleton key points of the target object in the input image in the three-dimensional coordinate system;

[0007] Determine a first rotation matrix of multiple bones of the target object according to the coordinate values ​​of the bone key points respectively corresponding to the multiple bone key points, wherein there is a bone topological connection relationship between the multiple bones;

[0008] Determine a second rotation matrix corresponding to each of the plurality of bones according to the bone topological connection relationship and the first rotation matrix;

[0009] Performing redirection calculation on the second rotation matrices corresponding to the plurality of bones respectively to obtain third rotation matrices corresponding to the plurality of bones respectively;

[0010] The initial three-dimensional virtual model is driven to perform posture changes according to the third rotation matrices corresponding to the multiple bones, so as to obtain a target three-dimensional virtual model corresponding to the target object.

[0011] In a second aspect, an embodiment of the present invention provides a drive processing device for a three-dimensional virtual model, the device comprising:

[0012] An acquisition module is used to acquire the coordinate values ​​of the skeleton key points corresponding to the multiple skeleton key points of the target object in the input image in the three-dimensional coordinate system;

[0013] A matrix determination module is used to determine the first rotation matrix of multiple bones of the target object according to the coordinate values ​​of the bone key points corresponding to each of the multiple bone key points, wherein there is a bone topological connection relationship between the multiple bones; determine the second rotation matrix corresponding to each of the multiple bones according to the bone topological connection relationship and the first rotation matrix; perform redirection calculation on the second rotation matrix corresponding to each of the multiple bones to obtain a third rotation matrix corresponding to each of the multiple bones;

[0014] The model generation module is used to drive the initial three-dimensional virtual model to perform posture changes according to the third rotation matrix corresponding to each of the multiple bones, so as to obtain a target three-dimensional virtual model corresponding to the target object.

[0015] In a third aspect, an embodiment of the present invention provides another driving processing method of a three-dimensional virtual model, the method comprising:

[0016] Get user image;

[0017] Detecting whether the image quality of the user image meets a preset quality standard;

[0018] If it is determined that the image quality meets the preset quality standard, then obtain the skeletal key point coordinate values ​​corresponding to each of the multiple skeletal key points of the user in the user image in the three-dimensional coordinate system; determine the first rotation matrix of the multiple bones of the user according to the skeletal key point coordinate values ​​corresponding to each of the multiple skeletal key points, wherein there is a skeletal topological connection relationship between the multiple bones; determine the second rotation matrix corresponding to each of the multiple bones according to the skeletal topological connection relationship and the first rotation matrix; perform redirection calculation on the second rotation matrix corresponding to each of the multiple bones to obtain the third rotation matrix corresponding to each of the multiple bones; and drive the initial three-dimensional virtual model to perform posture changes according to the third rotation matrix corresponding to each of the multiple bones to obtain a target three-dimensional virtual model corresponding to the user;

[0019] The target three-dimensional virtual model is displayed.

[0020] In a fourth aspect, an embodiment of the present invention provides another three-dimensional virtual model drive processing device, the device comprising:

[0021] An image acquisition unit, used to acquire a user image;

[0022] A quality detection unit, used to detect whether the image quality of the user image meets a preset quality standard;

[0023] A driving processing unit, configured to, if it is determined that the image quality meets the preset quality standard, obtain the coordinate values of the respective bone key points of the user in the user image in a three-dimensional coordinate system; determine a first rotation matrix of multiple bones of the user according to the coordinate values of the respective bone key points corresponding to the multiple bone key points; wherein, there is a bone topological connection relationship between the multiple bones; determine a second rotation matrix corresponding to each of the multiple bones according to the bone topological connection relationship and the first rotation matrix; perform a redirection calculation on the second rotation matrix corresponding to each of the multiple bones to obtain a third rotation matrix corresponding to each of the multiple bones; and drive an initial three-dimensional virtual model to perform a posture change according to the third rotation matrix corresponding to each of the multiple bones, so as to obtain a target three-dimensional virtual model corresponding to the user.

[0024] A display unit, configured to display the target three-dimensional virtual model.

[0025] In a fifth aspect, an embodiment of the present invention provides an electronic device, including: a memory, a processor, and a communication interface; wherein, an executable code is stored on the memory, and when the executable code is executed by the processor, the processor can at least implement the driving processing method of the three-dimensional virtual model as described in the first aspect.

[0026] In a sixth aspect, an embodiment of the present invention provides a non-transitory machine-readable storage medium, on which an executable code is stored, and when the executable code is executed by a processor of an electronic device, the processor can at least implement the driving processing method of the three-dimensional virtual model as described in the first aspect.

[0027] In an embodiment of the present invention, after obtaining the coordinate values of the respective bone key points of the target object in the input image in a three-dimensional coordinate system, a first rotation matrix of multiple bones of the target object is determined according to the coordinate values of the respective bone key points corresponding to the multiple bone key points; a second rotation matrix corresponding to each of the multiple bones is determined according to the bone topological connection relationship of the multiple bones and the first rotation matrix; a redirection calculation is performed on the second rotation matrix corresponding to each of the multiple bones to obtain a third rotation matrix corresponding to each of the multiple bones; and an initial three-dimensional virtual model is driven to perform a posture change according to the third rotation matrix corresponding to each of the multiple bones, so as to obtain a target three-dimensional virtual model corresponding to the target object.

[0028] In order to better construct the interactive gameplay of users in the virtual interactive game scene, increase the frequency of users participating in the virtual interactive game, and improve user stickiness. In the embodiment of the present invention, according to the coordinate values ​​of the skeleton key points of the target object (such as a person) in the input image taken by the user, the first rotation matrix of the multiple bones of the person is determined according to the coordinate values ​​of the skeleton key points corresponding to the multiple skeleton key points in the three-dimensional coordinate system; then according to the skeleton topological connection relationship of the multiple bones and the first rotation matrix, the second rotation matrix corresponding to the multiple bones is determined; after that, the posture redirection algorithm is used to redirect the calculation of the second rotation matrix corresponding to the multiple bones, and the third rotation matrix corresponding to the multiple bones is obtained, and the initial three-dimensional virtual model is driven to perform posture changes according to the third rotation matrix corresponding to the multiple bones, so as to obtain the target three-dimensional virtual model corresponding to the target object. To realize the generation of the corresponding target three-dimensional virtual model according to the user's input image, improve the user's interactive experience in the virtual interactive game scene, and then increase the frequency and duration of the user's network virtual social interaction, with a broad application space. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0030] Figure 1 A flowchart of a driving processing method of a three-dimensional virtual model provided by an embodiment of the present invention;

[0031] Figure 2 A schematic diagram of a process of driving a three-dimensional virtual model provided by an embodiment of the present invention;

[0032] Figure 3 A schematic diagram of a fourth rotation matrix provided by an embodiment of the present invention;

[0033] Figure 4 A schematic diagram of a fifth rotation matrix provided by an embodiment of the present invention;

[0034] Figure 5 A flowchart of another driving processing method of a three-dimensional virtual model provided by an embodiment of the present invention;

[0035] Figure 6 A schematic diagram of the structure of a driving processing device for a three-dimensional virtual model provided by an embodiment of the present invention;

[0036] Figure 7A schematic structural diagram of another three-dimensional virtual model driving processing device provided by an embodiment of the present invention;

[0037] Figure 8 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the case where there is no conflict between the embodiments, the following embodiments and the features in the embodiments can be combined with each other. In addition, the step sequence in the following method embodiments is only an example and not a strict limitation.

[0040] First, the terms or concepts involved in the embodiments of the present invention are explained:

[0041] Model penetration: refers to the phenomenon of objects or characters in 3D games penetrating and overlapping each other due to errors in the collision volume setting. For example, the hair of a 3D character model passes through the arm.

[0042] Page View (PV): refers to the number of page views or clicks on a website;

[0043] Unique Visitor (UV): A computer client that visits a website is a visitor. Visitors are distinguished by IP address. Repeated visits within a period of time are also considered a unique visitor.

[0044] Collision Detection Algorithm (Collison Detect): Collision detection is the computational problem of detecting the intersection of two or more objects. Collision detection is a classic problem in computational geometry and has applications in various computing fields, mainly in computer graphics, computer games, computer simulations, robotics, and computational physics. Collision detection algorithms can be divided into operations on 2D and 3D objects.

[0045] Convex Hull is a concept in computational geometry (graphics). According to the contour points of the bone, the convex hull function is used to convert the coordinates of the contour points of the convex hull, so as to draw the convex hull of the bone.

[0046] Pose Retarget: A mapping of source pose bones to target pose bones.

[0047] Inverse Kinematics (IK) is an algorithm that first determines the position information of the end bone, and then reversely derives the position information of the N-level parent bones in the bone inheritance chain based on the position information of the end bone, thereby determining the entire bone chain.

[0048] The driving processing method of the three-dimensional virtual model provided in the embodiment of the present invention can be executed by an electronic device. In practical applications, the electronic device can be a server or a user terminal such as a PC. The server can be a physical server or a virtual server (virtual machine) in the cloud.

[0049] Figure 1 A flowchart of a driving processing method of a three-dimensional virtual model provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method comprises the following steps:

[0050] 101. Obtain coordinate values ​​of the skeleton key points corresponding to the multiple skeleton key points of the target object in the input image in the three-dimensional coordinate system;

[0051] 102. Determine a first rotation matrix of multiple bones of the target object according to the coordinate values ​​of the bone key points corresponding to each of the multiple bone key points, wherein a bone topological connection relationship exists between the multiple bones.

[0052] 103. Determine second rotation matrices corresponding to each of the multiple bones according to the bone topological connection relationship and the first rotation matrix.

[0053] 104. Perform redirection calculation on the second rotation matrices corresponding to the multiple bones to obtain third rotation matrices corresponding to the multiple bones.

[0054] 105. Drive the initial three-dimensional virtual model to perform posture changes according to the third rotation matrices corresponding to the multiple bones, so as to obtain a target three-dimensional virtual model corresponding to the target object.

[0055] Optionally, the input image may be a photograph input by the user, such as a selfie, a portrait photo, a portrait photo, etc. of the user, or may be a photo used by other users to create a three-dimensional virtual character image controlled by the user in a three-dimensional virtual game scene.

[0056] Optionally, the target object may be a person or animal with a skeletal structure. Taking a person as an example, the multiple skeletal key points may correspond to key parts of the human body such as knee joints, hip joints, elbow joints, wrists, navels, heads, and feet.

[0057] Optionally, the three-dimensional coordinate system is formed by adding a Z axis to the two-dimensional coordinate system according to the right-hand rule. The three-dimensional coordinate system in the embodiment of the present invention specifically refers to the global coordinate system, which is a reference for describing the position and direction of all objects in the game scene, also known as the world coordinate system. Objects created in the game scene are all positioned with the coordinate origin (0,0,0) in the global coordinate system. The three-dimensional coordinate system can also be a screen coordinate system, and the Z axis in the screen coordinate system is measured in world units of the scene camera in the game scene.

[0058] Optionally, a posture estimation algorithm may be used, but is not limited to, to perform posture estimation calculations on the target object to determine the corresponding skeletal key point coordinate values ​​of the multiple skeletal key points in the three-dimensional coordinate system. That is, after obtaining the input image input by the user or the real-time shooting, a posture estimation algorithm may be used to perform posture estimation calculations on the skeletal key points of the target object in the input image to calculate the skeletal key point coordinate values ​​(x, y, z) corresponding to the multiple skeletal key points.

[0059] Optionally, the first rotation matrix of the above-mentioned multiple bones can be understood as a global rotation matrix, and the multiple bones can constitute a bone topological connection relationship, that is, the different bone lengths and different bone topological structures (that is, different numbers of nodes, and their graph connection relationship, that is, parent-child node relationship) presented by the multiple bones can be displayed.

[0060] After obtaining the coordinate values ​​of the bone key points corresponding to the multiple bone key points, in order to obtain the target three-dimensional virtual model corresponding to the target object, the local rotation matrix of each bone needs to be obtained. First, the global rotation matrix of the multiple bones is calculated by the coordinate values ​​of the multiple bone key points, and then the local rotation matrix corresponding to the multiple bones is determined according to the bone topological connection relationship of the multiple bones and the above global rotation matrix.

[0061] Due to the skeleton specificity of three-dimensional virtual images in some virtual interactive game scenes similar to The Sims, according to the skeleton structure of the three-dimensional virtual image in the virtual interactive game scene, a posture redirection algorithm is used to redirect the second rotation matrix corresponding to each of the multiple bones to obtain the third rotation matrix corresponding to each of the multiple bones, that is, the local rotation matrix after the redirection calculation. Finally, according to the third rotation matrix corresponding to each of the multiple bones, the initial three-dimensional virtual model is driven to perform posture changes to obtain a target three-dimensional virtual model corresponding to the target object, thereby realizing the conversion of the initial three-dimensional virtual model in the virtual interactive game into a target three-dimensional virtual model consistent with the posture of the character in the input image.

[0062] The initial 3D virtual model is a 3D virtual model in an initial state in a virtual interactive game. The initial state is the default display before the user sets the 3D virtual image according to the picture input by the user. For example, it can be a standing state in the shape of a "big" character. The target 3D virtual model is a 3D virtual model that is consistent with the posture of the character in the input image and is driven to transform the initial 3D virtual model according to the posture or pose of the character in the user input image.

[0063] It is understandable that the skeletal topological structure of the three-dimensional virtual model in different virtual interactive games is different. Therefore, according to the display or interaction requirements of the virtual interactive game, the skeletal topological connection relationship between multiple bones is predetermined, so that when generating the target three-dimensional virtual model, the second rotation matrix corresponding to each of the multiple bones is determined according to the predetermined skeletal topological connection relationship and the first rotation matrix generated in real time. Moreover, the initial three-dimensional virtual model in the initial state of the virtual interactive game is also generated in advance according to different virtual interactive games. Therefore, the target three-dimensional virtual model obtained by subsequently driving the initial three-dimensional model can be fully adapted to the virtual interactive game. Furthermore, the driving processing method of the three-dimensional virtual model provided by the embodiment of the present invention can be adapted to the independently developed rendering engine corresponding to different virtual interactive games, and the interactive experience of generating the corresponding target three-dimensional virtual model using the user's input photos can be automatically realized.

[0064] In order to better construct the interactive gameplay of users in the virtual interactive game scene, increase the frequency of users participating in the virtual interactive game, and improve user stickiness. In the embodiment of the present invention, according to the coordinate values ​​of the skeleton key points of the target object (such as a person) in the input image taken by the user, the first rotation matrix of the multiple bones of the person is determined according to the coordinate values ​​of the skeleton key points corresponding to the multiple skeleton key points in the three-dimensional coordinate system; then according to the skeleton topological connection relationship of the multiple bones and the first rotation matrix, the second rotation matrix corresponding to the multiple bones is determined; after that, the posture redirection algorithm is used to redirect the calculation of the second rotation matrix corresponding to the multiple bones, and the third rotation matrix corresponding to the multiple bones is obtained, and the initial three-dimensional virtual model is driven to perform posture changes according to the third rotation matrix corresponding to the multiple bones, so as to obtain the target three-dimensional virtual model corresponding to the target object. To realize the generation of the corresponding target three-dimensional virtual model according to the user's input image, improve the user's interactive experience in the virtual interactive game scene, and then increase the frequency and duration of the user's network virtual social interaction, with a broad application space.

[0065] To facilitate understanding of the embodiments of the present invention, refer to Figure 2 The driving process diagram of a three-dimensional virtual model shown in FIG. 1 is used to illustrate the method provided by an embodiment of the present invention. Figure 2As shown, the person in the input image may be a male, and the person is in a natural and relaxed sitting posture. The posture estimation algorithm is used to perform posture estimation calculation on the skeletal key points of the person to calculate the skeletal key point coordinate values ​​(x, y, z) corresponding to multiple skeletal key points.

[0066] An optional embodiment, since the three-dimensional virtual model and the three-dimensional background in the virtual game scene need to be realized through modeling, it is inevitable that there will be a phenomenon of model penetration. In order to avoid the influence of this phenomenon on the target three-dimensional virtual model generated later, before determining the first rotation matrix of multiple bones of the target object according to the coordinate values ​​of the bone key points corresponding to each of the multiple bone key points, it is possible to first detect whether there is a phenomenon of model penetration between the multiple bones. If there is, collision removal is performed on the phenomenon of model penetration in advance, which can be implemented specifically by the following method:

[0067] According to the topological connection relationship of the bones, corresponding enclosing cylinders are constructed for multiple bones; a collision detection algorithm is used to detect the corresponding enclosing cylinders constructed for multiple bones to determine whether there is a penetration phenomenon between the multiple bones; if it is determined that there is a penetration phenomenon between the multiple bones, the penetration phenomenon is resolved based on the topological connection relationship of the bones until there is no penetration phenomenon between the multiple bones.

[0068] Optionally, a corresponding enclosing cylinder is constructed for each bone first, that is, a cylindrical bounding box corresponding to each bone is constructed. For ease of understanding, each bone cylinder can be regarded as a convex hull, that is, according to the contour points of the bone, the contour points are converted into the coordinates of the convex hull through the Convex Hull function, so as to draw the convex hull of the bone. This step is to better perform collision detection processing on multiple bones.

[0069] Afterwards, a collision detection algorithm is used to detect multiple bones and construct their corresponding enclosing cylinders. It can be determined whether there is a penetration phenomenon between the multiple bones. If so, the penetration phenomenon can be resolved step by step according to the bone topological connection relationship for the penetration part where the penetration phenomenon exists. Finally, until there is no penetration phenomenon between the multiple bones, the subsequent model generation operation can be performed, that is, the first rotation matrix of the multiple bones of the target object can be determined according to the coordinate values ​​of the bone key points corresponding to each of the multiple bone key points.

[0070] According to the above description, a skeleton collision detection method and a corresponding collision elimination method designed in an embodiment of the present invention can avoid the penetration phenomenon that may occur in the target three-dimensional virtual model generated subsequently.

[0071] As an optional embodiment, before determining the second rotation matrices corresponding to each of the multiple bones based on the skeletal topological connection relationship of the multiple bones and the first rotation matrix, the skeletal topological connection relationship of the multiple bones is first determined by the following method: obtaining the connection relationship between multiple skeletal key points and the multiple bones, as well as the parent-child node relationship between the multiple bones; and determining the skeletal topological connection relationship of the multiple bones based on the connection relationship and the parent-child node relationship.

[0072] In the embodiment of the present invention, according to the bone topological connection relationship of the multiple bones and the first rotation matrix, the second rotation matrix corresponding to each of the multiple bones is determined, which can be specifically implemented by the following method:

[0073] The parent-child node relationship between multiple bones is determined according to the bone topological connection relationship; the bone position derivation algorithm is used to calculate the second rotation matrix corresponding to each bone node by node starting from the target bone based on the parent-child node relationship according to the first rotation matrix, so as to obtain the second rotation matrix corresponding to each of the multiple bones, wherein the target bone is any one of the multiple bones.

[0074] As an optional embodiment, taking the arm skeleton as an example, the arm driving the hand is forward dynamics, and the hand driving the arm is inverse dynamics. The process of solving the position of the upper arm skeleton and the forearm skeleton when the hand skeleton is known can be understood as the process of solving using the skeleton position derivation algorithm IK in inverse dynamics.

[0075] Specifically, this solution process must first determine the parent-child node relationship between multiple bones based on the bone topological connection relationship. For example, starting from the highest level node of the bone hierarchy, first determine any target bone among the multiple bones, and then start solving layer by layer. First, the global rotation matrix of the multiple bones is calculated by the coordinate values ​​of the bone key points corresponding to each of the multiple bones. Starting from the target bone, the local rotation matrix corresponding to the current target bone is calculated, and then the parent-child node relationship between the multiple bones determined by the bone topological connection relationship is calculated layer by layer to obtain the local rotation matrices corresponding to all the target bones required to drive the initial three-dimensional virtual model.

[0076] Through the above embodiments, the bone position derivation algorithm is used to determine the local rotation matrices corresponding to each of the multiple bones, which is very important for improving the animation quality of the target three-dimensional virtual model and showing the animation details of the target three-dimensional virtual model.

[0077] In an optional embodiment, redirection calculation is performed on the second rotation matrices corresponding to the multiple bones to obtain the third rotation matrices corresponding to the multiple bones, which can be implemented in the following optional manner:

[0078] The fourth rotation matrix corresponding to each of the multiple bones of the initial three-dimensional virtual model and the fifth rotation matrix corresponding to each of the multiple bones of the target object in the initial state are obtained, and the transformation matrix between the fourth rotation matrix and the fifth rotation matrix is ​​calculated; the posture redirection algorithm is used to redirect the second rotation matrix based on the transformation matrix to obtain the third rotation matrix corresponding to each of the multiple bones.

[0079] In the embodiment of the present invention, the multiple bones of the initial three-dimensional virtual model are in an initial state; that is, the fourth rotation matrix and the fifth rotation matrix are both rotation matrices in the initial state.

[0080] Alternatively, if Figure 3 As shown, J 10 , J 11 , J 12 , J 13 , J 14 , J 15 , J 16 They are different bone key points of the bone structure of the three-dimensional virtual model in the bone coordinate system space, such as Figure 3 The fourth rotation matrix is ​​a local rotation matrix of the bone key points of the skeletal structure of the three-dimensional virtual model corresponding to the skeletal coordinate system space. Figure 4 As shown, J 00 , J 01 , J 02 , J 03 , J 04 , J 05 , J 06 , J 07 , J 08 Different bone key points of the target object’s bone structure in the initial state in the bone coordinate system space are obtained for posture estimation, such as Figure 4 One of the styles of the fifth rotation matrix is ​​shown, and the fifth rotation matrix is ​​the local rotation matrix of the skeleton structure of the target object in the initial state obtained by posture estimation corresponding to the skeleton key points in the skeleton coordinate system space.

[0081] according to Figure 3 and Figure 4 It can be seen that the number of skeletal key points of the skeletal structure of the three-dimensional virtual model in the skeletal coordinate system space is not equal to the number of skeletal key points of the skeletal structure of the target object in the initial state obtained by posture estimation. Therefore, in order to migrate and map the skeletal structure of the target object to the initial three-dimensional virtual model to be driven, it is necessary to adopt a posture redirection algorithm to redirect the calculation of the second rotation matrix corresponding to multiple bones in the current state of the target object.

[0082] Specifically, the fourth rotation matrix of the skeletal key points of the initial three-dimensional virtual model in the initial state is first calculated; then, the fifth rotation matrix of the skeletal key points of the skeletal structure of the posture estimation in the initial state is calculated, and the transformation matrix between the fourth rotation matrix and the fifth rotation matrix is ​​obtained by calculation; finally, the posture redirection algorithm is used to redirect the second rotation matrix based on the transformation matrix to obtain the third rotation matrix corresponding to each of the multiple bones, so that the skeletal structure of the target object can be migrated and mapped to the initial three-dimensional virtual model to be driven.

[0083] Through the embodiments of the present invention, the IK algorithm and PoseRetarget algorithm for the skeletal structure of the three-dimensional virtual model provide an automated implementation method for generating a three-dimensional virtual image driven by the user's input photos; it provides strong support for the promotion and implementation of interactive gameplay in virtual interactive game scenes, and increases the number of independent visitors UV and page views PV of users in virtual interactive game scenes. Moreover, it is convenient for users to freely change virtual images and dress up in the virtual space of the virtual interactive game scene to achieve the purpose of liberalized virtual social interaction or sharing, thereby increasing the frequency of users participating in virtual interactive games and improving user stickiness.

[0084] Figure 5 A flowchart of another driving processing method of a three-dimensional virtual model provided by an embodiment of the present invention, such as Figure 5 As shown, the method comprises the following steps:

[0085] 501. Obtain user image;

[0086] 502. Detect whether the image quality of the user image meets the preset quality standard;

[0087] 503. If it is determined that the image quality meets the preset quality standard, then obtain the skeletal key point coordinate values ​​corresponding to the multiple skeletal key points of the user in the user image in the three-dimensional coordinate system; determine the first rotation matrix of the multiple bones of the user according to the skeletal key point coordinate values ​​corresponding to the multiple skeletal key points, wherein there is a skeletal topological connection relationship between the multiple bones; determine the second rotation matrix corresponding to the multiple bones according to the skeletal topological connection relationship and the first rotation matrix; perform redirection calculation on the second rotation matrix corresponding to the multiple bones to obtain the third rotation matrix corresponding to the multiple bones; and drive the initial three-dimensional virtual model to perform posture changes according to the third rotation matrix corresponding to the multiple bones to obtain a target three-dimensional virtual model corresponding to the user;

[0088] 504. Display the target three-dimensional virtual model.

[0089] Optionally, the method provided in the embodiment of the present invention can be applied to some virtual interactive game scenes similar to virtual life and network social activity scenes. Through this method embodiment, the purpose of users appearing in network social activity scenes in the form of virtual images can be achieved. For example, in some virtual life applications, a virtual interactive space is provided for users, in which users can freely change virtual images, clothing and decorations to achieve their own social or sharing purposes.

[0090] Optionally, the user image may be collected or uploaded by the user, for example, a user-uploaded photo, such as a user's selfie, image photo, portrait photo, etc., or a photo used by other users to create a three-dimensional virtual character image controlled by the user in a three-dimensional virtual game scene.

[0091] Optionally, multiple skeletal key points may correspond to key parts of the human body such as knee joints, hip joints, elbow joints, wrists, navels, heads, and feet.

[0092] Optionally, a posture estimation algorithm may be used, but is not limited to, to perform posture estimation calculations on the user to determine the corresponding skeletal key point coordinate values ​​of the multiple skeletal key points in the three-dimensional coordinate system. That is, after obtaining the user image uploaded by the user or collecting the user image taken by the user in real time, a posture estimation algorithm may be used to perform posture estimation calculations on the skeletal key points of the user in the user image to calculate the skeletal key point coordinate values ​​(x, y, z) corresponding to the multiple skeletal key points.

[0093] Optionally, the first rotation matrix of the above-mentioned multiple bones can be understood as a global rotation matrix, and the multiple bones can constitute a bone topological connection relationship, that is, the different bone lengths and different bone topological structures (that is, different numbers of nodes, and their graph connection relationship, that is, parent-child node relationship) presented by the multiple bones can be displayed.

[0094] After obtaining the coordinate values ​​of the bone key points corresponding to the multiple bone key points, in order to obtain the target three-dimensional virtual model corresponding to the user, the local rotation matrix of each bone needs to be obtained. First, the global rotation matrix of the multiple bones is calculated by the coordinate values ​​of the multiple bone key points, and then the local rotation matrix corresponding to the multiple bones is determined according to the bone topological connection relationship of the multiple bones and the above global rotation matrix.

[0095] Due to the skeletal specificity of three-dimensional virtual images in some virtual interactive game scenes similar to The Sims, a posture redirection algorithm is used to redirect the second rotation matrix corresponding to each of the multiple bones according to the skeletal structure of the three-dimensional virtual image in the virtual interactive game scene, and obtain the third rotation matrix corresponding to each of the multiple bones, that is, the local rotation matrix after the redirection calculation. Finally, the initial three-dimensional virtual model is driven to perform posture changes according to the third rotation matrix corresponding to each of the multiple bones to obtain the target three-dimensional virtual model corresponding to the user, thereby realizing the conversion of the initial three-dimensional virtual model in the virtual interactive game into a target three-dimensional virtual model with the same posture as the character in the user image. After obtaining the target three-dimensional virtual model, the target three-dimensional virtual model is rendered and displayed.

[0096] Optionally, the initial 3D virtual model is a 3D virtual model in an initial state in the virtual interactive game, which is displayed by default before the user sets the 3D virtual image according to the picture input by the user, for example, it can be a standing state in the shape of a "big" character. The target 3D virtual model is a 3D virtual model that is finally obtained by driving the initial 3D virtual model to transform according to the posture or gesture of the character in the user's image, and which is consistent with the posture of the character in the user's image.

[0097] Since the skeletal topological structures of the three-dimensional virtual models in different virtual interactive games are different, the skeletal topological connection relationship between the multiple bones is predetermined according to the display or interaction requirements of the virtual interactive game, so that when generating the target three-dimensional virtual model, the second rotation matrix corresponding to each of the multiple bones is determined according to the predetermined skeletal topological connection relationship and the first rotation matrix generated in real time. Moreover, the initial three-dimensional virtual model in the initial state of the virtual interactive game is also generated in advance according to different virtual interactive games, so the target three-dimensional virtual model obtained by subsequently driving the initial three-dimensional model can be fully adapted to the virtual interactive game. Furthermore, the driving processing method of the three-dimensional virtual model provided by the embodiment of the present invention can adapt to the independently developed rendering engine corresponding to different virtual interactive games, and can automatically realize the interactive experience of generating the corresponding target three-dimensional virtual model using the user's input photos.

[0098] In order to better construct the interactive gameplay of users in virtual interactive game scenes, enhance the interactive experience of users in virtual interactive game scenes, and thus increase the frequency and duration of users' online virtual social interaction. In the embodiment of the present invention, according to the coordinate values ​​of the skeleton key points corresponding to the multiple skeleton key points of the user in the user image in the three-dimensional coordinate system, according to the coordinate values ​​of the skeleton key points corresponding to the multiple skeleton key points, the first rotation matrix of the multiple bones of the user is determined; then, according to the skeleton topological connection relationship of the multiple bones and the first rotation matrix, the second rotation matrix corresponding to the multiple bones is determined; after that, the posture redirection algorithm is used to redirect the calculation of the second rotation matrix corresponding to the multiple bones, and the third rotation matrix corresponding to the multiple bones is obtained, and the initial three-dimensional virtual model is driven to perform posture changes according to the third rotation matrix corresponding to the multiple bones, so as to obtain the target three-dimensional virtual model corresponding to the user. In order to realize the generation of the corresponding target three-dimensional virtual model according to the user image, enhance the interactive experience of the user in the virtual interactive game scene, and then increase the frequency and duration of the user's online virtual social interaction, it has a broad application space.

[0099] The following will describe in detail the driving processing device of the three-dimensional virtual model of one or more embodiments of the present invention. Those skilled in the art will appreciate that these devices can be configured using commercially available hardware components through the steps taught in this solution.

[0100] Figure 6 A schematic diagram of a structure of a driving processing device for a three-dimensional virtual model provided by an embodiment of the present invention, such as Figure 6 As shown, the device includes: an acquisition module 11, a matrix determination module 12, and a model generation module 13.

[0101] The acquisition module 11 is used to acquire the coordinate values ​​of the skeleton key points corresponding to each of the multiple skeleton key points of the target object in the input image in the three-dimensional coordinate system.

[0102] The matrix determination module 12 is used to determine the first rotation matrix of multiple bones of the target object according to the coordinate values ​​of the bone key points corresponding to each of the multiple bone key points, wherein there is a bone topological connection relationship between the multiple bones; determine the second rotation matrix corresponding to each of the multiple bones according to the bone topological connection relationship of the multiple bones and the first rotation matrix; redirect the second rotation matrix corresponding to each of the multiple bones to obtain the third rotation matrix corresponding to each of the multiple bones.

[0103] The model generation module 13 is used to drive the initial three-dimensional virtual model to perform posture changes according to the third rotation matrices corresponding to the multiple bones, so as to obtain a target three-dimensional virtual model corresponding to the target object.

[0104] Optionally, the matrix determination module 12 includes: a first determination unit, used to determine the parent-child node relationship between multiple bones according to the bone topological connection relationship; a second determination unit, used to use the bone position derivation algorithm to calculate the second rotation matrix corresponding to each bone node by node starting from the target bone based on the parent-child node relationship according to the first rotation matrix, so as to obtain the second rotation matrices corresponding to each of the multiple bones, wherein the target bone is any one of the multiple bones.

[0105] Optionally, the matrix determination module 12 also includes: an acquisition unit, used to acquire a fourth rotation matrix corresponding to each of the multiple bones of the initial three-dimensional virtual model, and a fifth rotation matrix corresponding to each of the multiple bones of the target object in the initial state, wherein the multiple bones of the initial three-dimensional virtual model are in the initial state; a first calculation unit, used to calculate the transformation matrix between the fourth rotation matrix and the fifth rotation matrix; and a second calculation unit, used to use a posture redirection algorithm to perform redirection calculation on the second rotation matrix based on the transformation matrix to obtain a third rotation matrix corresponding to each of the multiple bones.

[0106] Optionally, the acquisition module 11 is further used to perform posture estimation calculation on the target object using a posture estimation algorithm to determine the coordinate values ​​of the skeleton key points corresponding to each of the multiple skeleton key points in the three-dimensional coordinate system.

[0107] Optionally, the above-mentioned device also includes: a construction module, which is used to construct corresponding enclosing cylinders for multiple bones according to the bone topological connection relationship; a detection module, which is used to use a collision detection algorithm to detect the enclosing cylinders corresponding to the multiple bones to determine whether there is a penetration phenomenon between the multiple bones; and a collision elimination module, which is used to eliminate the collision of the penetration phenomenon based on the bone topological connection relationship if it is determined that there is a penetration phenomenon between the multiple bones, until there is no penetration phenomenon between the multiple bones.

[0108] Optionally, the above-mentioned device also includes: a relationship acquisition module, which is used to obtain the connection relationship between multiple bone key points and multiple bones, and the parent-child node relationship between multiple bones; and a relationship determination module, which is used to determine the bone topological connection relationship of multiple bones based on the connection relationship and the parent-child node relationship.

[0109] Figure 7 A schematic diagram of the structure of another driving processing device for a three-dimensional virtual model provided by an embodiment of the present invention, such as Figure 7 As shown, the device includes: an image acquisition unit 21, a quality detection unit 22, a drive processing unit 23, and a display unit 24.

[0110] An image acquisition unit 21, used to acquire a user image;

[0111] A quality detection unit 22, used to detect whether the image quality of the user image meets a preset quality standard;

[0112] The driving processing unit 23 is used to obtain the skeletal key point coordinate values ​​corresponding to each of the multiple skeletal key points of the user in the user image in the three-dimensional coordinate system if it is determined that the image quality meets the preset quality standard; determine the first rotation matrix of the multiple bones of the user according to the skeletal key point coordinate values ​​corresponding to each of the multiple skeletal key points, wherein there is a skeletal topological connection relationship between the multiple bones; determine the second rotation matrix corresponding to each of the multiple bones according to the skeletal topological connection relationship and the first rotation matrix; perform redirection calculation on the second rotation matrix corresponding to each of the multiple bones to obtain the third rotation matrix corresponding to each of the multiple bones; and drive the initial three-dimensional virtual model to perform posture changes according to the third rotation matrix corresponding to each of the multiple bones to obtain a target three-dimensional virtual model corresponding to the user;

[0113] The display unit 24 is used to display the target three-dimensional virtual model.

[0114] In one possible design, the above Figure 6 and Figure 7 The structure of the driving processing device of the three-dimensional virtual model shown in the figure can be realized as an electronic device. Figure 8 As shown, the electronic device may include: a processor 31, a memory 32, and a communication interface 33. The memory 32 stores executable codes, and when the executable codes are executed by the processor 31, the processor 31 can at least implement the driving processing method of the three-dimensional virtual model provided in the above-mentioned embodiment.

[0115] In addition, an embodiment of the present invention provides a non-temporary machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the driving processing method of the three-dimensional virtual model provided in the aforementioned embodiment.

[0116] The device embodiments described above are merely illustrative, wherein the network elements described as separate components may or may not be physically separated. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art may understand and implement the solution without creative effort.

[0117] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on such an understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A driving processing method for a three-dimensional virtual model, It is characterized in that include: Obtaining the coordinate values ​​of the skeleton key points corresponding to the multiple skeleton key points of the target object in the input image in the three-dimensional coordinate system; Determine a first rotation matrix of multiple bones of the target object according to the coordinate values ​​of the bone key points respectively corresponding to the multiple bone key points, wherein there is a bone topological connection relationship between the multiple bones; Determine a second rotation matrix corresponding to each of the plurality of bones according to the bone topological connection relationship and the first rotation matrix; Performing redirection calculation on the second rotation matrices corresponding to the plurality of bones respectively to obtain third rotation matrices corresponding to the plurality of bones respectively; Driving the initial three-dimensional virtual model to perform posture changes according to the third rotation matrices corresponding to the multiple bones, so as to obtain a target three-dimensional virtual model corresponding to the target object; Among them, the redirection calculation of the second rotation matrix corresponding to each of the multiple bones to obtain the third rotation matrix corresponding to each of the multiple bones includes: obtaining the fourth rotation matrix corresponding to each of the multiple bones of the initial three-dimensional virtual model, and the fifth rotation matrix corresponding to each of the multiple bones of the target object in the initial state, wherein the multiple bones of the initial three-dimensional virtual model are in the initial state; calculating the transformation matrix between the fourth rotation matrix and the fifth rotation matrix; and using a posture redirection algorithm to redirect the second rotation matrix based on the transformation matrix to obtain the third rotation matrix corresponding to each of the multiple bones.

2. The method according to claim 1, It is characterized in that The step of determining the second rotation matrices corresponding to the plurality of bones respectively according to the bone topological connection relationship and the first rotation matrix comprises: Determine the parent-child node relationship between the multiple bones according to the bone topological connection relationship; A bone position derivation algorithm is used to calculate the second rotation matrix corresponding to each bone node by node starting from the target bone based on the parent-child node relationship according to the first rotation matrix, so as to obtain the second rotation matrix corresponding to each of the multiple bones, wherein the target bone is any one of the multiple bones.

3. The method according to claim 1, It is characterized in that The step of obtaining the coordinate values ​​of the skeleton key points corresponding to the multiple skeleton key points of the target object in the input image in the three-dimensional coordinate system includes: A posture estimation algorithm is used to perform posture estimation calculation on the target object to determine the coordinate values ​​of the skeleton key points corresponding to each of the multiple skeleton key points in the three-dimensional coordinate system.

4. The method according to claim 1, It is characterized in that Before determining the first rotation matrix of the multiple bones of the target object according to the coordinate values ​​of the skeleton key points respectively corresponding to the multiple skeleton key points, the method further includes: Constructing corresponding enclosing cylinders for the multiple bones according to the bone topological connection relationship; A collision detection algorithm is used to detect the enclosing cylinders corresponding to the plurality of bones to determine whether there is a penetration phenomenon between the plurality of bones; If it is determined that the penetration phenomenon exists between the multiple bones, the penetration phenomenon is collided and resolved based on the topological connection relationship of the bones until the penetration phenomenon does not exist between the multiple bones.

5. The method according to claim 1, It is characterized in that Before determining the second rotation matrices corresponding to the plurality of bones respectively according to the bone topological connection relationship and the first rotation matrix, the method further includes: Obtaining the connection relationship between the multiple skeleton key points and the multiple skeletons, and the parent-child node relationship between the multiple skeletons; The skeleton topology connection relationship is determined according to the connection relationship and the parent-child node relationship.

6. A driving processing device for a three-dimensional virtual model, It is characterized in that include: An acquisition module is used to acquire the coordinate values ​​of the skeleton key points corresponding to the multiple skeleton key points of the target object in the input image in the three-dimensional coordinate system; A matrix determination module is used to determine the first rotation matrix of multiple bones of the target object according to the coordinate values ​​of the bone key points corresponding to each of the multiple bone key points, wherein there is a bone topological connection relationship between the multiple bones; determine the second rotation matrix corresponding to each of the multiple bones according to the bone topological connection relationship and the first rotation matrix; redirect the second rotation matrix corresponding to each of the multiple bones to obtain the third rotation matrix corresponding to each of the multiple bones; wherein the redirecting calculation of the second rotation matrix corresponding to each of the multiple bones to obtain the third rotation matrix corresponding to each of the multiple bones includes: obtaining the fourth rotation matrix corresponding to each of the multiple bones of the initial three-dimensional virtual model, and the fifth rotation matrix corresponding to each of the multiple bones of the target object in the initial state, wherein the multiple bones of the initial three-dimensional virtual model are in the initial state; calculating the conversion matrix between the fourth rotation matrix and the fifth rotation matrix; using a posture redirection algorithm to redirect the second rotation matrix based on the conversion matrix to obtain the third rotation matrix corresponding to each of the multiple bones; The model generation module is used to drive the initial three-dimensional virtual model to perform posture changes according to the third rotation matrix corresponding to each of the multiple bones, so as to obtain a target three-dimensional virtual model corresponding to the target object.

7. A driving processing method for a three-dimensional virtual model, It is characterized in that include: Get user image; Detecting whether the image quality of the user image meets a preset quality standard; If it is determined that the image quality meets the preset quality standard, then obtaining the coordinate values ​​of the skeletal key points corresponding to the plurality of skeletal key points of the user in the user image in the three-dimensional coordinate system; Determine the first rotation matrix of the multiple bones of the user according to the coordinate values ​​of the bone key points corresponding to each of the multiple bone key points, wherein there is a bone topological connection relationship between the multiple bones; determine the second rotation matrix corresponding to each of the multiple bones according to the bone topological connection relationship and the first rotation matrix; perform redirection calculation on the second rotation matrix corresponding to each of the multiple bones to obtain a third rotation matrix corresponding to each of the multiple bones; and drive the initial three-dimensional virtual model to perform posture changes according to the third rotation matrix corresponding to each of the multiple bones to obtain a target three-dimensional virtual model corresponding to the user; Displaying the target three-dimensional virtual model; Among them, the redirection calculation of the second rotation matrix corresponding to each of the multiple bones to obtain the third rotation matrix corresponding to each of the multiple bones includes: obtaining the fourth rotation matrix corresponding to each of the multiple bones of the initial three-dimensional virtual model, and the fifth rotation matrix corresponding to each of the multiple bones of the target object in the initial state, wherein the multiple bones of the initial three-dimensional virtual model are in the initial state; calculating the transformation matrix between the fourth rotation matrix and the fifth rotation matrix; and using a posture redirection algorithm to redirect the second rotation matrix based on the transformation matrix to obtain the third rotation matrix corresponding to each of the multiple bones.

8. A driving processing device for a three-dimensional virtual model, It is characterized in that include: An image acquisition unit, used to acquire a user image; A quality detection unit, used to detect whether the image quality of the user image meets a preset quality standard; A driving processing unit is configured to obtain the coordinate values ​​of the skeletal key points corresponding to the plurality of skeletal key points of the user in the user image in the three-dimensional coordinate system if it is determined that the image quality meets the preset quality standard; According to the coordinate values ​​of the skeletal key points corresponding to each of the multiple skeletal key points, a first rotation matrix of the multiple bones of the user is determined, wherein there is a skeletal topological connection relationship between the multiple bones; according to the skeletal topological connection relationship and the first rotation matrix, a second rotation matrix corresponding to each of the multiple bones is determined; redirection calculation is performed on the second rotation matrix corresponding to each of the multiple bones to obtain a third rotation matrix corresponding to each of the multiple bones; and according to the third rotation matrix corresponding to each of the multiple bones, the initial three-dimensional virtual model is driven to perform posture changes to obtain a target three-dimensional virtual model corresponding to the user; wherein the redirection calculation is performed on the second rotation matrix corresponding to each of the multiple bones to obtain the third rotation matrix corresponding to each of the multiple bones, including: obtaining a fourth rotation matrix corresponding to each of the multiple bones of the initial three-dimensional virtual model, and a fifth rotation matrix corresponding to each of the multiple bones of the target object in an initial state, wherein the multiple bones of the initial three-dimensional virtual model are in the initial state; calculating a conversion matrix between the fourth rotation matrix and the fifth rotation matrix; and using a posture redirection algorithm to redirect the second rotation matrix based on the conversion matrix to obtain the third rotation matrix corresponding to each of the multiple bones; A display unit is used to display the target three-dimensional virtual model.

9. An electronic device, It is characterized in that include: A memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the driving processing method of the three-dimensional virtual model as described in any one of claims 1 to 5, or the driving processing method of the three-dimensional virtual model as described in claim 7.

10. A non-transitory machine-readable storage medium, It is characterized in that The non-temporary machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor executes the driving processing method of a three-dimensional virtual model as described in any one of claims 1 to 7, or the driving processing method of a three-dimensional virtual model described in claim 7.

Images