Virtual object driving method and device, electronic equipment and storage medium

Abstract

The present disclosure relates to a virtual object driving method, device, electronic equipment and storage medium, the method comprising: obtaining initial rotation information of a plurality of bone nodes corresponding to a target action of a source object; adjusting a rotation angle of a parent node rotation based on initial rotation information of a child node and initial rotation information of a corresponding parent node, and position information of the child node in a coordinate system corresponding to the parent node; obtaining a target rotation angle when a rotation component meets a preset requirement, and optimizing rotation information of the parent node and rotation information of the child node based on the target rotation angle; mapping the optimized rotation information of each bone node of the source object to each bone node of a preset virtual object; and driving the virtual object based on the first target rotation information. The present scheme can repair and enhance the motion capture result of the capture device, so that the virtual object is driven based on the optimized first target rotation information, making the bone rotation more smooth and natural.

Description

Technical Field

[0001] This disclosure relates to the field of virtual reality technology, and more particularly to virtual object driving methods, apparatus, electronic devices, and storage media. Background Technology

[0002] In recent years, the industry attention on virtual humans has increased significantly. The core of the virtual human production process consists of three main parts: modeling, driving, and rendering. Driving can be broadly divided into human-driven and AI-driven approaches. Currently, human-driven approaches are the primary method for offline content production and live streaming of virtual humans, playing an irreplaceable role. This is based on performance capture technology, which captures the facial expressions and movements of real actors. Performance capture includes facial expression capture and body motion capture, collectively known as motion capture. Performance capture is widely used in film, games, animation, and other industries, significantly reducing the manual production costs for animators.

[0003] However, existing body-driven methods suffer from unnatural skeletal rotation. Summary of the Invention

[0004] This disclosure provides a virtual object driving method, apparatus, electronic device, and storage medium to at least solve the problem of unnatural skeletal rotation in body driving techniques in related technologies. The technical solution of this disclosure is as follows: According to a first aspect of the present disclosure, a virtual object driving method is provided, the method comprising: Obtain the initial rotation information of multiple bone nodes corresponding to the target action of the source object; the multiple bone nodes have a parent-child hierarchical relationship; Based on the initial rotation information of the child node and the corresponding initial rotation information of the parent node, and the position information of the child node in the coordinate system corresponding to the parent node, the rotation angle of the parent node is adjusted so that the rotation components of the child node and the parent node in the target direction meet the preset requirements; the target direction is the direction in which the rotation components of the child node and the parent node are preset values ​​when they rotate. Obtain the rotation angle when the rotation component meets the preset requirements, obtain the target rotation angle, and optimize the rotation information of the parent node and the rotation information of the child node based on the target rotation angle to obtain the optimized rotation information of the parent node and the optimized rotation information of the child node; The optimized rotation information of each bone node of the source object is mapped to each bone node of the preset virtual object to obtain the first target rotation information of each bone node of the virtual object. The virtual object is driven based on the rotation information of the first target.

[0005] Optionally, before adjusting the rotation angle of the parent node, the method further includes: Based on the position information of the child node in the coordinate system corresponding to the parent node, determine the rotation axis of the child node relative to the parent node; Construct a corrected rotation matrix function based on the corrected rotation angle of the parent node and the rotation axis; Based on the initial rotation information of the child node, the initial rotation information of the parent node, and the corrected rotation matrix function, the rotation component functions of the child node and the parent node in the target direction are determined. The target direction is the direction in which the rotation component of the child node relative to the parent node is a preset value, and the preset value is 0.

[0006] Optionally, adjusting the rotation angle of the child node relative to the parent node includes: Based on the rotation component function and the corrected rotation matrix function, the rotation angle of the child node relative to the parent node is adjusted to minimize the value of the rotation component function.

[0007] Optionally, adjusting the rotation angle of the child node relative to the parent node based on the rotation component function and the corrected rotation matrix function to minimize the value of the rotation component function includes: Determine the equation relationship between the optimized rotation information of the parent node, the corrected rotation matrix, and the initial rotation information of the parent node, and determine the equation relationship between the optimized rotation information of the child node, the corrected rotation matrix, and the initial rotation information of the child node to obtain the first equation and the second equation. Using the first equation, the second equation, and the corrected rotation function, the rotation component function is converted into a transformation function with the corrected rotation angle as the variable; Determine the value of the correction rotation angle when the transformation function reaches its minimum value.

[0008] Optionally, optimizing the rotation information of the parent node and the child node based on the target rotation angle to obtain the optimized rotation information of the parent node and the optimized rotation information of the child node includes: Based on the corrected rotation matrix function, determine the target rotation matrix corresponding to the target rotation angle; Based on the target rotation matrix and the initial rotation information of the parent node, the optimized rotation information of the parent node is determined, and based on the target rotation matrix and the initial rotation information of the child node, the optimized rotation information of the child node is determined.

[0009] Optionally, before obtaining the initial rotation information of multiple skeletal nodes corresponding to the target action of the source object, the method further includes: When the source object is in the initial action state, the rotation information relative to the world coordinate system is determined when each child node and its corresponding parent node on the source object rotates to the preset standard pose, and the first global rotation matrix and the second global rotation matrix are obtained; the standard pose is the preset target pose of the standard object. When the preset virtual object is in the initial action state, the rotation information of each child node and its corresponding parent node on the virtual object relative to the world coordinate system when rotated to the standard pose is determined, and the third global rotation matrix and the fourth global rotation matrix are obtained.

[0010] Optionally, the optimized rotation information is the optimized local rotation matrix of the bone node relative to its corresponding parent node, and mapping the optimized rotation information of each bone node of the source object to each bone node of the preset virtual object includes: Based on the optimized local rotation matrix, the first global rotation matrix, and the second global rotation matrix, the optimized rotation information of each bone node of the source object is mapped to each bone node of the standard object to obtain the standard rotation information of the standard object. Based on the third global rotation matrix and the fourth global rotation matrix, the standard rotation information is mapped onto each bone node of the preset virtual object to obtain the first target rotation information of each bone node of the virtual object.

[0011] Optionally, mapping the optimized rotation information of each bone node of the source object to each bone node of the standard object includes: Determine the first product of the optimized local rotation matrix of the bone node on the source object and the corresponding second global rotation matrix, and determine the product between the first product and the reciprocal of the first global rotation matrix of the bone node to obtain the second product; Determine the first correspondence between the skeletal nodes of the source object and the skeletal nodes of the standard object; Based on the first correspondence, the rotation information corresponding to the second product of the bone node is mapped to the corresponding bone node of the standard object to obtain the standard rotation information of the standard object.

[0012] Optionally, the step of mapping the standard rotation information onto each skeletal node of a preset virtual object based on the third global rotation matrix and the fourth global rotation matrix to obtain the first target rotation information of each skeletal node of the virtual object includes: Determine a second correspondence between the skeletal nodes of the standard object and the skeletal nodes of the virtual object; Based on the second correspondence, the standard rotation information of the bone nodes on the standard object is determined, and the product of the standard rotation information and the third global rotation matrix of the corresponding bone nodes on the virtual object is obtained to obtain the third product; wherein, the standard rotation information is the second product; The fourth product is obtained by determining the product between the third product and the reciprocal of the fourth global rotation matrix; The rotation information corresponding to the fourth product is used as the first target rotation information for each skeletal node of the virtual object.

[0013] Optionally, before driving the virtual object based on the first target rotation information, the method further includes: The first target rotation information, root node position, and foot node position corresponding to each lower limb bone node of the virtual object are redirected and optimized to obtain the second target rotation information, optimized root node position, and optimized foot node position, respectively. The step of driving the virtual object based on the first target rotation information includes: The virtual object is driven based on the first target rotation information, the second target rotation information, the optimized root node position, and the optimized foot node position.

[0014] Optionally, the redirection optimization of the first target rotation information, root node position, and foot node position corresponding to each lower limb bone node of the virtual object includes: Construct a retargeting optimization function using foot position constraints, foot rotation constraints, regularization constraints, and penalty constraints; Using the aforementioned redirection optimization function, the first target rotation information and root node position of each lower limb bone node of the virtual object are redirected and optimized, so that the foot node of the virtual object lands on the target position after the rotation of the root node and each lower limb bone node. The root node is the bone node where the hip joint is located; the target position is determined based on the position of the foot node after the source object performs the target action.

[0015] Optionally, the retargeting optimization function is constructed using foot position constraints, foot rotation constraints, regularization constraints, and penalty constraints, including: The optimized position of the foot node after redirection optimization is determined based on the forward dynamics algorithm, and a foot position constraint term is constructed based on the difference between the optimized foot node position after redirection optimization and the optimization target of the foot position, wherein the optimization target of the foot position is the target position; The forward dynamics algorithm determines the redirected and optimized foot node rotation matrix, and the foot rotation constraint term is constructed based on the difference between the redirected and optimized foot node rotation information and the optimization target of the foot node rotation information. The optimization target of the foot node rotation information is the optimized rotation information of the foot node on the source object. Based on the difference between the optimized position of the root node after redirection optimization and the initial position of the root node before redirection optimization, a first regularization constraint term is constructed. For each lower limb bone node, a second regularization constraint term is constructed based on the difference between the second target rotation information of the lower limb bone node after retargeting optimization and the first target rotation information of the lower limb bone node. Based on the pre-defined penalty constraints for unreasonable postures, penalty constraint terms are constructed. A retargeting optimization function is constructed based on the weighted sum of the foot position constraint, the foot rotation preset, the first regularization constraint, the second regularization constraint, and the penalty constraint.

[0016] Optionally, the step of using the redirection optimization function to redirect and optimize the first target rotation information and root node position of each lower limb bone node of the virtual object includes: When the redirection optimization function reaches its minimum value, the optimized position value of the foot node, the value of the second target rotation information of the foot node, the optimized position value of the root node, and the value of the second target rotation information of each lower limb bone node are determined so that the foot node of the virtual object lands on the target position after being rotated by the root node and each lower limb bone node.

[0017] Optionally, before retargeting and optimizing the first target rotation information, root node position, and foot node position corresponding to each lower limb bone node of the virtual object, the method further includes: The leg lengths of the source object and the virtual object are normalized based on the ratio between the leg length of the source object and the leg length of the virtual object. Based on the normalization results, the root node position, the position of each lower limb bone node, and the foot node position in the source object and the virtual object are determined respectively.

[0018] According to a second aspect of the present disclosure, a virtual object driving device is provided, the device comprising: The initial rotation information acquisition module is configured to acquire the initial rotation information of multiple skeletal nodes corresponding to the target action of the source object; the multiple skeletal nodes have a parent-child hierarchical relationship. The rotation angle adjustment module is configured to perform rotation based on the initial rotation information of the child node and the corresponding initial rotation information of the parent node, and the position information of the child node in the coordinate system corresponding to the parent node, to adjust the rotation angle of the parent node so that the rotation components of the child node and the parent node in the target direction meet preset requirements; the target direction is the direction in which the rotation components of the child node and the parent node are preset values ​​when they rotate. The optimized rotation information determination module is configured to obtain the rotation angle when the rotation component meets the preset requirements, obtain the target rotation angle, and optimize the rotation information of the parent node and the rotation information of the child node based on the target rotation angle to obtain the optimized rotation information of the parent node and the optimized rotation information of the child node. The mapping module is configured to perform the mapping of the optimized rotation information of each bone node of the source object to each bone node of the preset virtual object, so as to obtain the first target rotation information of each bone node of the virtual object; The driving module is configured to drive the virtual object based on the rotation information of the first target.

[0019] Optionally, the device further includes: The rotation axis determination module is configured to determine the rotation axis of the child node relative to the parent node based on the position information of the child node in the coordinate system corresponding to the parent node; The first function construction module is configured to execute a function that constructs a corrected rotation matrix based on the corrected rotation angle and the rotation axis of the parent node. The second function communication module is configured to execute, based on the initial rotation information of the child node, the initial rotation information of the parent node, and the correction rotation matrix function, to determine the rotation component function of the child node and the parent node in a target direction, wherein the target direction is the direction in which the rotation component of the child node relative to the parent node is a preset value, and the preset value is 0.

[0020] Optionally, the rotation angle adjustment module is specifically configured to perform: Based on the rotation component function and the corrected rotation matrix function, the rotation angle of the child node relative to the parent node is adjusted to minimize the value of the rotation component function.

[0021] Optionally, the rotation angle adjustment module is further configured to perform: Determine the equation relationship between the optimized rotation information of the parent node, the corrected rotation matrix, and the initial rotation information of the parent node, and determine the equation relationship between the optimized rotation information of the child node, the corrected rotation matrix, and the initial rotation information of the child node to obtain the first equation and the second equation. Using the first equation, the second equation, and the corrected rotation function, the rotation component function is converted into a transformation function with the corrected rotation angle as the variable; Determine the value of the correction rotation angle when the transformation function reaches its minimum value.

[0022] Optionally, the rotation angle adjustment module is specifically configured to perform: Based on the corrected rotation matrix function, determine the target rotation matrix corresponding to the target rotation angle; Based on the target rotation matrix and the initial rotation information of the parent node, the optimized rotation information of the parent node is determined, and based on the target rotation matrix and the initial rotation information of the child node, the optimized rotation information of the child node is determined.

[0023] Optionally, the device further includes: The first rotation information determination module is configured to determine the rotation information relative to the world coordinate system when each child node and its corresponding parent node on the source object rotate to a preset standard pose when the source object is in the initial action state, and to obtain a first global rotation matrix and a second global rotation matrix; the standard pose is the preset target pose of the standard object. The second rotation information determination module is configured to determine the rotation information of each child node and its corresponding parent node on the virtual object relative to the world coordinate system when the virtual object is in the initial action state, and obtain the third global rotation matrix and the fourth global rotation matrix.

[0024] Optionally, the mapping module is specifically configured to execute: Based on the optimized local rotation matrix, the first global rotation matrix, and the second global rotation matrix, the optimized rotation information of each bone node of the source object is mapped to each bone node of the standard object to obtain the standard rotation information of the standard object. Based on the third global rotation matrix and the fourth global rotation matrix, the standard rotation information is mapped onto each bone node of the preset virtual object to obtain the first target rotation information of each bone node of the virtual object.

[0025] Optionally, the mapping module is further configured to perform: Determine the first product of the optimized local rotation matrix of the bone node on the source object and the corresponding second global rotation matrix, and determine the product between the first product and the reciprocal of the first global rotation matrix of the bone node to obtain the second product; Determine the first correspondence between the skeletal nodes of the source object and the skeletal nodes of the standard object; Based on the first correspondence, the rotation information corresponding to the second product of the bone node is mapped to the corresponding bone node of the standard object to obtain the standard rotation information of the standard object.

[0026] Optionally, the mapping module is further configured to perform: Determine a second correspondence between the skeletal nodes of the standard object and the skeletal nodes of the virtual object; Based on the second correspondence, the standard rotation information of the bone nodes on the standard object is determined, and the product of the standard rotation information and the third global rotation matrix of the corresponding bone nodes on the virtual object is obtained to obtain the third product; wherein, the standard rotation information is the second product; The fourth product is obtained by determining the product between the third product and the reciprocal of the fourth global rotation matrix; The rotation information corresponding to the fourth product is used as the first target rotation information for each skeletal node of the virtual object.

[0027] Optionally, the device further includes: The redirection optimization module is configured to perform redirection optimization on the first target rotation information, root node position, and foot node position corresponding to each lower limb bone node of the virtual object, so as to obtain the second target rotation information, the optimized root node position, and the optimized foot node position, respectively. The driver module is specifically configured to execute: The virtual object is driven based on the first target rotation information, the second target rotation information, the optimized root node position, and the optimized foot node position.

[0028] Optionally, the redirection optimization module is specifically configured to execute: Construct a retargeting optimization function using foot position constraints, foot rotation constraints, regularization constraints, and penalty constraints; Using the aforementioned redirection optimization function, the first target rotation information and root node position of each lower limb bone node of the virtual object are redirected and optimized, so that the foot node of the virtual object lands on the target position after the rotation of the root node and each lower limb bone node. The root node is the bone node where the hip joint is located; the target position is determined based on the position of the foot node after the source object performs the target action.

[0029] Optionally, the redirection optimization module is specifically configured to execute: The optimized position of the foot node after redirection optimization is determined based on the forward dynamics algorithm, and a foot position constraint term is constructed based on the difference between the optimized foot node position after redirection optimization and the optimization target of the foot position, wherein the optimization target of the foot position is the target position; The forward dynamics algorithm determines the redirected and optimized foot node rotation matrix, and the foot rotation constraint term is constructed based on the difference between the redirected and optimized foot node rotation information and the optimization target of the foot node rotation information. The optimization target of the foot node rotation information is the optimized rotation information of the foot node on the source object. Based on the difference between the optimized position of the root node after redirection optimization and the initial position of the root node before redirection optimization, a first regularization constraint term is constructed. For each lower limb bone node, a second regularization constraint term is constructed based on the difference between the second target rotation information of the lower limb bone node after retargeting optimization and the first target rotation information of the lower limb bone node. Based on the pre-defined penalty constraints for unreasonable postures, penalty constraint terms are constructed. A retargeting optimization function is constructed based on the weighted sum of the foot position constraint, the foot rotation preset, the first regularization constraint, the second regularization constraint, and the penalty constraint.

[0030] Optionally, the redirection optimization module is specifically configured to execute: When the redirection optimization function reaches its minimum value, the optimized position value of the foot node, the value of the second target rotation information of the foot node, the optimized position value of the root node, and the value of the second target rotation information of each lower limb bone node are determined so that the foot node of the virtual object lands on the target position after being rotated by the root node and each lower limb bone node.

[0031] Optionally, the device further includes: The normalization module is configured to perform normalization processing on the leg length of the source object and the leg length of the virtual object based on the ratio between the leg length of the source object and the leg length of the virtual object; The position determination module is configured to determine the root node position, the position of each lower limb bone node, and the foot node position in the source object and the virtual object respectively based on the normalization processing result.

[0032] According to a third aspect of the present disclosure, a computer-readable storage medium is provided that, when instructions in the computer-readable storage medium are executed by a processor of a server, enables the server to perform the virtual object driving method as described in the first aspect.

[0033] According to a fourth aspect of the present disclosure, a computer program product is provided, including a computer program / instructions, wherein the computer program / instructions, when executed by a processor, implement the virtual object driving method described in the first aspect.

[0034] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects: In the embodiments of this disclosure, initial rotation information of multiple skeletal nodes corresponding to the target action of the source object is obtained; the multiple skeletal nodes have a parent-child hierarchical relationship; based on the initial rotation information of the child node and the corresponding initial rotation information of the parent node, and the position information of the child node in the coordinate system corresponding to the parent node, the rotation angle of the parent node is adjusted so that the rotation components of the child node and the parent node in the target direction meet a preset requirement; the target direction is the direction in which the rotation components of the child node and the parent node are at a preset value when they rotate; the rotation angle when the rotation components meet the preset requirement is obtained to obtain the target rotation angle, and the rotation information of the parent node and the child node is optimized based on the target rotation angle to obtain the optimized rotation information of the parent node and the optimized rotation information of the child node; the optimized rotation information of each skeletal node of the source object is mapped to each skeletal node of a preset virtual object to obtain the first target rotation information of each skeletal node of the virtual object; the virtual object is driven based on the first target rotation information. In this solution, the target rotation angle when the rotation component meets the preset requirements is obtained, and the rotation information of the parent node and the child node is optimized based on the target rotation angle. This can repair and enhance the motion capture results of the capture device. In this way, the virtual object is driven based on the optimized first target rotation information, making the skeletal rotation smoother and more natural.

[0035] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0036] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0037] Figure 1 This is a flowchart illustrating the steps of a first virtual object-driven method according to an exemplary embodiment; Figure 2 This is a flowchart illustrating the steps of a second virtual object-driven method according to an exemplary embodiment; Figure 3 This is a schematic diagram illustrating a skeletal node according to an exemplary embodiment; Figure 4 This is a flowchart illustrating the steps of a third virtual object driving method according to an exemplary embodiment; Figure 5 This is an exemplary embodiment illustrating an architecture diagram of a capture component and a driver component; Figure 6 This is a flowchart illustrating the steps of a fourth virtual object driving method according to an exemplary embodiment; Figure 7 This is a structural block diagram of a virtual object driving device according to an exemplary embodiment; Figure 8 This is a block diagram illustrating an electronic device for virtual object driving according to an exemplary embodiment. Detailed Implementation

[0038] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0039] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0040] Figure 1 This is a flowchart illustrating the steps of a first virtual object-driven method according to an exemplary embodiment, such as... Figure 1 As shown, the method includes the following steps.

[0041] In step S11, the initial rotation information of multiple bone nodes corresponding to the target action of the source object is obtained; the multiple bone nodes have a parent-child hierarchical relationship.

[0042] In this embodiment of the invention, the virtual object driver uses a capture component to collect action data of the source object and maps the action data onto the virtual object, thereby achieving the driving of the virtual object.

[0043] A source object generally refers to a real object, while a virtual object is a three-dimensional model synthesized using digital technology to simulate a real object. This object can include humans or animals. The source object and the virtual object are of the same kind, for example, both may be humans, or both may be animals.

[0044] The initial rotation information can be obtained by directly capturing the motion information of the source object using the capture component, or by mapping the motion information of the source object onto a standard skeletal model and then obtaining the motion information corresponding to the standard skeletal model obtained from the mapping.

[0045] In this embodiment of the invention, the capture component achieves the capture of human facial expressions and motion data by accessing various motion capture algorithms, capture devices or capture software.

[0046] The capture components include facial, limb, and finger capture, and can utilize various devices or solutions currently available on the market. For example, facial capture can use head-mounted cameras; limb capture can use inertial motion capture solutions or optical motion capture solutions; finger capture can use inertial motion gloves or integrated solutions that incorporate limb capture. Additionally, vision-based markerless motion capture can be used to achieve limb and finger motion capture in specific scenarios. In short, the capture components can be flexibly combined with various capture solutions based on scenario requirements and budget.

[0047] This invention primarily addresses the problem of driving virtual objects to perform actions using motion data collected by a capture component. The motion data collected by the capture component mainly includes the positions of the skeletal nodes involved in the target action and the initial rotation information of each skeletal node from its initial state to the target action state when performing the target action.

[0048] Skeletal nodes have a parent-child hierarchical relationship, meaning that when a parent node skeleton rotates or moves, all its child node skeletons will rotate or move accordingly.

[0049] In step S12, based on the initial rotation information of the child node and the initial rotation information of the corresponding parent node, and the position information of the child node in the coordinate system corresponding to the parent node, the rotation angle of the parent node is adjusted so that the rotation components of the child node and the parent node in the target direction meet the preset requirements; the target direction is the direction in which the rotation components of the child node and the parent node are preset values ​​when they rotate.

[0050] In this embodiment of the invention, the motion data output by the capture device or software may contain errors and defects, and a post-processing algorithm is introduced to correct or optimize them.

[0051] For post-processing of motion, this solution proposes a general bone rotation allocation optimization strategy to improve problems such as unnatural bone rotation. Its basic principle is to adjust the axial rotation between parent and child nodes to minimize a specific optimization objective.

[0052] Specifically, the optimization target is the rotation components of the child and parent nodes in a target direction. This target direction is the direction in which the rotation components of the child and parent nodes are at preset values ​​when they rotate.

[0053] By using the initial rotation information of the child node, the initial rotation information of its corresponding parent node, and the position information of the child node in the coordinate system of the parent node, the rotation angle of the parent node is adjusted so that the rotation components of the child node and the parent node in the target direction meet the preset requirements.

[0054] The position information of the child node in the coordinate system corresponding to the parent node can be the three-dimensional coordinates of the child node in the local coordinate system of the parent node.

[0055] In step S13, the rotation angle when the rotation component meets the preset requirements is obtained to obtain the target rotation angle, and the rotation information of the parent node and the rotation information of the child node are optimized based on the target rotation angle to obtain the optimized rotation information of the parent node and the optimized rotation information of the child node.

[0056] If the direction of the rotation component of the child node relative to the parent node is a preset value is taken as the direction of the target coordinate axis, and the rotation component of the child node in the direction of the target coordinate axis is taken as the rotation component function, then the optimization problem can be constructed based on the rotation component function.

[0057] When the rotation component meets the preset requirements, the variables in the rotation component function reach their optimal values. Based on these values, the target rotation angle can be obtained. This target rotation angle is the angle value that is closest to the actual rotation degree.

[0058] Rotate the parent node from its initial rotation information to the target rotation angle to obtain the optimized rotation information of the parent node; rotate the child node from its initial rotation information to the target rotation angle to obtain the optimized rotation information of the child node.

[0059] In step S13, the optimized rotation information of each bone node of the source object is mapped to each bone node of the preset virtual object to obtain the first target rotation information of each bone node of the virtual object.

[0060] For rotational data, since different capture devices or solutions use different skeletal structures, it is necessary to map the skeletal nodes of the source object to the skeletal nodes of the virtual object.

[0061] The optimized rotation information and bone node information of each bone of the source object are sent as rotation data to the driver module. The driver model determines the mapping relationship between the source object and the virtual object based on the bone node information, and converts the optimized rotation information into the first target rotation information corresponding to the virtual object according to the proportional relationship between the bones of the two.

[0062] In step S14, the virtual object is driven based on the first target rotation information.

[0063] The rotation motion data of the first target rotation information is assigned to the corresponding bones of the virtual object, so that each bone of the virtual object moves according to the first target rotation information. This drives the virtual object to perform the target action.

[0064] In summary, the initial rotation information of multiple skeletal nodes corresponding to the target action of the source object is obtained; the multiple skeletal nodes have a parent-child hierarchical relationship; based on the initial rotation information of the child nodes and the corresponding initial rotation information of the parent nodes, and the position information of the child nodes in the coordinate system corresponding to the parent nodes, the rotation angle of the parent nodes is adjusted so that the rotation components of the child nodes and the parent nodes in the target direction meet preset requirements; the target direction is the direction in which the rotation components of the child nodes and the parent nodes are at preset values ​​when they rotate; the rotation angle when the rotation components meet the preset requirements is obtained to obtain the target rotation angle, and the rotation information of the parent nodes and the child nodes is optimized based on the target rotation angle to obtain the optimized rotation information of the parent nodes and the optimized rotation information of the child nodes; the optimized rotation information of each skeletal node of the source object is mapped to each skeletal node of the preset virtual object to obtain the first target rotation information of each skeletal node of the virtual object; the virtual object is driven based on the first target rotation information. In this solution, the target rotation angle when the rotation component meets the preset requirements is obtained, and the rotation information of the parent node and the child node is optimized based on the target rotation angle. This can repair and enhance the motion capture results of the capture device. In this way, the virtual object is driven based on the optimized first target rotation information, making the skeletal rotation smoother and more natural.

[0065] Figure 2 This is a flowchart illustrating the steps of a second virtual object-driven method according to an exemplary embodiment, such as... Figure 2 As shown, the method includes the following steps.

[0066] In step S21, the initial rotation information of multiple bone nodes corresponding to the target action of the source object is obtained; the multiple bone nodes have a parent-child hierarchical relationship.

[0067] In this embodiment of the invention, step S21 can refer to step S11, and will not be repeated here.

[0068] In step S22, the rotation axis of the child node relative to the parent node is determined based on the position information of the child node in the coordinate system corresponding to the parent node.

[0069] Figure 3This is a schematic diagram of a skeletal node according to an exemplary embodiment.

[0070] like Figure 3 As shown, node 0 is the parent node and node 1 is the child node. The position of node 1 in the coordinate system of node 0 is P1(x1,y1,z1).

[0071] The axis of rotation of a child node relative to its parent node is the line connecting the child and parent nodes. Based on the three-dimensional coordinates of the child node in the local coordinate system of the parent node, the line connecting the child and parent nodes can be obtained, thus yielding the axis of rotation of the child node relative to its parent node.

[0072] In step S23, a corrected rotation matrix function is constructed based on the corrected rotation angle of the parent node rotation and the rotation axis.

[0073] Specifically, the corrected rotation matrix function can be expressed as: (1) in, The function represents the construction of the rotation matrix from the corrected rotation angle and rotation axis. Represents the corrected rotation matrix of the parent node. Indicates the correction of rotation angle. The axis of rotation is P1, which is the three-dimensional coordinate of the child node in the coordinate system of the parent node, represented as (x1, y1, z1).

[0074] Specifically, the rotation axis is determined by the three-dimensional coordinates of the child node in the local coordinate system of the parent node; the corrected rotation angle is the angle by which the parent node rotates around the rotation axis from the initial rotation information to the optimized rotation information. The corrected rotation matrix is ​​the difference in rotation information between the initial rotation information and the optimized rotation information.

[0075] In step S24, based on the initial rotation information of the child node, the initial rotation information of the parent node, and the corrected rotation matrix function, the rotation component functions of the child node and the parent node in the target direction are determined. The target direction is the direction in which the rotation component of the child node relative to the parent node is a preset value, and the preset value is 0.

[0076] In this embodiment of the disclosure, the direction of the coordinate axis in which the rotation component is 0 when the child node rotates relative to the corresponding parent node is determined, thereby obtaining the target coordinate axis direction.

[0077] When the parent node rotates, the child nodes rotate relative to the parent node. Theoretically, the rotation components of the child node's rotation should correspond to the x, y, and z coordinate axes. However, in reality, due to the characteristics of the human skeleton, there is always a rotation component of 0 in one coordinate axis direction.

[0078] For example, the human elbow joint cannot rotate around the z-axis. Therefore, if the parent node is the upper arm joint and the child node is the lower arm joint, the target coordinate axis direction is the Z-axis direction.

[0079] In the skeletal system composed of the parent and child nodes described above, node 0 is the parent node of node 1, and Rj represents the node before optimization. The local rotation matrix, Rj' represents the optimized node. The local rotation matrix, 1 represents the position of the child node in the local coordinate system of the parent node.

[0080] Thus, R1 represents the local rotation matrix of node 1 before optimization, R0 represents the local rotation matrix of node 0 before optimization, R1' represents the local rotation matrix of node 1 after optimization, and R0' represents the local rotation matrix of node 0 after optimization.

[0081] The rotation component function is .

[0082] In step S25, based on the rotation component function and the corrected rotation matrix function, the rotation angle of the child node relative to the parent node is adjusted to minimize the value of the rotation component function.

[0083] Specifically, the rotation component function Minimizing the value of is the optimization objective, and when When it is 0, The value will Substituting the value into the correction rotation matrix function yields the correction rotation angle of the child node relative to the parent node.

[0084] In one possible implementation, step S25 includes the following steps: Step S251: Determine the equation relationship between the optimized rotation information of the parent node, the corrected rotation matrix, and the initial rotation information of the parent node; and determine the equation relationship between the optimized rotation information of the child node, the corrected rotation matrix, and the initial rotation information of the child node, to obtain the first equation and the second equation. Step S252: Using the first equation, the second equation, and the corrected rotation function, the rotation component function is converted into a transformation function with the corrected rotation angle as the variable; Step S253: Determine the value of the correction rotation angle when the transformation function reaches its minimum value.

[0085] Specifically, in steps S251-S253, the equation relationship between the optimized rotation information of the parent node and the corrected rotation matrix and the initial rotation information of the parent node is determined, and the equation relationship between the optimized rotation information of the child node and the corrected rotation matrix and the initial rotation information of the child node is determined, thus obtaining the first equation and the second equation.

[0086] Specifically, the first equation is: (2) The second equation is: (3) in, This represents the optimized rotation information of the parent node. This indicates the initial rotation information of the parent node. Represents the corrected rotation matrix; This represents the optimized rotation information of the child nodes. This indicates the initial rotation information of the child nodes; Using the first equation, the second equation, and the corrected rotation function, the rotation component function is transformed into a transformation function with the corrected rotation angle as the variable. That is, by substituting the first equation, the second equation, and the corrected rotation function into the rotation component function, we obtain: =

[0087] = (4) As can be seen, the transformation function is... A function with variables.

[0088] Minimization is taken as the optimization objective of the transformation function, i.e., solving for min The result shows that when the transformation function reaches its minimum value, the corrected rotation angle is... The value of .

[0089] In step S26, the rotation angle when the rotation component meets the preset requirements is obtained, and the target rotation angle is obtained.

[0090] The final result obtained in step S253 The value is the rotation angle when the rotation component meets the preset requirements, i.e., the target rotation angle.

[0091] In step S27, the target rotation matrix corresponding to the target rotation angle is determined based on the corrected rotation matrix function.

[0092] Will Substitute the value ,available The value of is the target rotation matrix.

[0093] In step S28, the optimized rotation information of the parent node is determined based on the target rotation matrix and the initial rotation information of the parent node, and the optimized rotation information of the child node is determined based on the target rotation matrix and the initial rotation information of the child node.

[0094] Will Substituting the value of into the first and second equations, we can obtain the following results respectively. The value represents the optimized rotation information of the parent node and the optimized rotation information of the child node.

[0095] In step S29, the optimized rotation information of each bone node of the source object is mapped to each bone node of the preset virtual object to obtain the first target rotation information of each bone node of the virtual object.

[0096] In this embodiment of the disclosure, step S29 can refer to step S14, and will not be repeated here.

[0097] In step S30, the virtual object is driven based on the first target rotation information.

[0098] In this embodiment of the disclosure, step S30 can refer to step S15, and will not be repeated here.

[0099] In summary, in this embodiment of the present disclosure, the rotation component function is defined as the rotation component of the child node in the direction of the target coordinate axis, and the target coordinate axis is the direction of the coordinate axis where the rotation component is 0. In this way, the optimization problem can be cleverly transformed into the problem of minimizing the rotation component function, thereby obtaining optimized rotation information. The obtained optimized rotation information is more accurate and optimized in representing motion data, further improving the smoothness and naturalness of the virtual object's motion.

[0100] Figure 4 This is a flowchart illustrating the steps of a third virtual object-driven method according to an exemplary embodiment. Figure 4 As shown, the method includes the following steps: In step S31, when the source object is in the initial action state, the rotation information of each child node and its corresponding parent node on the source object relative to the world coordinate system when rotated to the preset standard pose is determined, and the first global rotation matrix and the second global rotation matrix are obtained; the standard pose is the preset target pose of the standard object.

[0101] In this embodiment of the disclosure, for motion data, since various capture devices or schemes use different skeletal structures, it can be mapped to a standard skeletal model, which can be the currently commonly used Humannoid skeletal model. The standard skeletal model is a standard object.

[0102] The initial action state refers to the action state that the source object is in before it performs the target action.

[0103] A standard pose is the target pose of a standard object. A standard pose can be one of the following: T-pose, A-pose, etc. This embodiment of the invention selects T-pose as the standard pose for illustration.

[0104] When the source object is in its initial motion state, the rotation information relative to the world coordinate system for each child node and its corresponding parent node on the source object when rotated to the standard pose is determined, resulting in the first global rotation matrix and the second global rotation matrix. This provides the rotation information for each skeletal node on the source object when rotated to the standard pose.

[0105] Specifically, the global rotation matrix of each bone on the source object when rotated to the T-pose is the first global rotation matrix, which can be represented as follows: The global rotation matrix of the parent nodes of each bone on the source object when rotated to the T-pose is the second global rotation matrix, which can be represented as follows: .

[0106] Similarly, facial expression data obtained from the capture component can be mapped to a standard facial expression model. This standard facial expression model can be the currently widely used Animoji facial expression model.

[0107] Animoji is a commonly used Blendshape standard for facial expressions, containing 52 skeletal bases. Humannoid is a skeletal structure for the human body defined in Unity, including the main bones of the body and fingers. For facial expression data, if the facial capture module works according to the Animoji standard, the coefficients of the 52 skeletal bases can be directly passed.

[0108] Because different capture devices use different skeletal structures and facial expression schemes, after mapping them to standard skeletal models and standard facial expression models, they can be directly connected to standard driver component interfaces to drive virtual objects.

[0109] In this way, by simply setting the driver component interface to a standardized interface, the driver algorithm capabilities can be reused for different capture devices, and it can support the expansion to more types of driver devices. This achieves decoupling between the capture component and the driver component.

[0110] It is understandable that Animoji facial expression models and Humannoid skeletal models are just commonly used standards or templates, and the above solutions can be replaced with other standards or templates.

[0111] Figure 5This is an exemplary embodiment illustrating an architecture diagram of a capture component and a driver component.

[0112] like Figure 5 As shown, this solution comprises two parts: a capture component and a driving component. The capture component captures the facial expressions and motion data of an object by connecting to various motion capture algorithms, devices, or software. The capture component also includes a post-processing algorithm component to optimize and enhance the facial expression and motion data. The driving component redirects the facial expression and motion data to the virtual object character and, in conjunction with components such as motion adjustment, collision, and cloth simulation, achieves high-quality driving effects.

[0113] In step S32, when the preset virtual object is in the initial action state, the rotation information of each child node and its corresponding parent node on the virtual object relative to the world coordinate system when rotated to the standard pose is determined, and the third global rotation matrix and the fourth global rotation matrix are obtained.

[0114] Similar to the source object, the skeletal model of a virtual object is not necessarily a standard skeletal model. By determining the rotation information of each child node and its corresponding parent node relative to the world coordinate system when rotated to the standard pose, the rotation information of each skeletal node on the virtual object when rotated to the standard pose can be obtained.

[0115] By obtaining the rotation information of each bone node on the source object when it rotates to the standard pose, and the rotation information of each bone node on the virtual object when it rotates to the standard pose, it is convenient to subsequently map the target motion rotation information of the source object to the standard object, and then map the target motion rotation information to the virtual object through the standard object, so as to drive the virtual object.

[0116] The global rotation matrix of each bone on the virtual object when rotated to the T-pose is the third global rotation matrix, which can be represented as follows: The global rotation matrix of each bone's parent node when rotated to the T-pose on the virtual object is the fourth global rotation matrix, which can be represented as: .

[0117] In step S33, the initial rotation information of multiple bone nodes corresponding to the target action of the source object is obtained; the multiple bone nodes have a parent-child hierarchical relationship.

[0118] In this embodiment of the invention, step S33 can refer to step S11, and will not be repeated here.

[0119] In step S34, based on the initial rotation information of the child node and the initial rotation information of the corresponding parent node, and the position information of the child node in the coordinate system corresponding to the parent node, the rotation angle of the parent node is adjusted so that the rotation components of the child node and the parent node in the target direction meet the preset requirements; the target direction is the direction in which the rotation components of the child node and the parent node are preset values ​​when they rotate.

[0120] In this embodiment of the invention, step S34 can refer to step S12, and will not be repeated here.

[0121] In step S35, the rotation angle when the rotation component meets the preset requirements is obtained to obtain the target rotation angle, and the rotation information of the parent node and the rotation information of the child node are optimized based on the target rotation angle to obtain the optimized rotation information of the parent node and the optimized rotation information of the child node.

[0122] In this embodiment of the invention, step S35 can refer to step S13, and will not be repeated here.

[0123] In step S36, the optimized rotation information is the optimized local rotation matrix of the bone node relative to its corresponding parent node; based on the optimized local rotation matrix, the first global rotation matrix, and the second global rotation matrix, the optimized rotation information of each bone node of the source object is mapped to each bone node of the standard object to obtain the standard rotation information of the standard object.

[0124] Rotation information is represented in the form of rotation matrices. Based on the optimized local rotation matrix, the first global rotation matrix, and the second global rotation matrix, the optimized rotation information of each bone node of the source object is mapped to the rotation information of each bone node of the standard object, thus obtaining the standard rotation information of the standard object.

[0125] In one possible implementation, step S36 includes the following steps: In step S361, the first product of the optimized local rotation matrix of the bone node on the source object and the corresponding second global rotation matrix is ​​determined, and the product between the first product and the reciprocal of the first global rotation matrix of the bone node is determined to obtain the second product; In step S362, a first correspondence between the skeletal nodes of the source object and the skeletal nodes of the standard object is determined; In step S363, based on the first correspondence, the rotation information corresponding to the second product of the bone node is mapped to the corresponding bone node of the standard object to obtain the standard rotation information of the standard object.

[0126] In steps S361-S363, the first product is .

[0127] in, This represents the optimized local rotation matrix for a specific bone node on the source object. This represents the second global rotation matrix of the parent node of the aforementioned skeletal node.

[0128] The second product result is the transformation matrix, represented as follows: (5) in, This represents the first global rotation matrix of the aforementioned skeletal nodes.

[0129] Based on the first correspondence between the skeletal nodes of the source object and the skeletal nodes of the standard object, the second product is mapped onto the corresponding skeletal node of the standard object to obtain the standard rotation information of the standard object.

[0130] In step S37, based on the third global rotation matrix and the fourth global rotation matrix, the standard rotation information is mapped onto each bone node of the preset virtual object to obtain the first target rotation information of each bone node of the virtual object.

[0131] Based on the third global rotation matrix and the fourth global rotation matrix, the rotation information that maps the standard rotation information to each bone node of the virtual object is calculated to obtain the first target rotation information of the virtual object.

[0132] In one possible implementation, step S37 includes the following steps: In step S371, a second correspondence is determined between the skeletal nodes of the standard object and the skeletal nodes of the virtual object; In step S372, the standard rotation information of the bone nodes on the standard object is determined according to the second correspondence, and the product of the standard rotation information and the third global rotation matrix of the corresponding bone nodes on the virtual object is obtained to obtain the third product; wherein, the standard rotation information is the second product; In step S373, the product between the third product and the reciprocal of the fourth global rotation matrix is ​​determined to obtain the fourth product; In step S374, the rotation information corresponding to the fourth product is used as the first target rotation information of each skeletal node of the virtual object.

[0133] In steps S371-S374, the third product is .

[0134] in, The standard rotation information of the skeletal nodes on the standard object is represented by the second product. This is the third global rotation matrix for the skeletal nodes on the virtual object.

[0135] The fourth product represents the first target rotation information of each skeletal node of the virtual object, as shown below: (6) in, This indicates the rotation information of the first target. This represents the fourth global rotation matrix.

[0136] Based on the second correspondence between the skeletal nodes of the standard object and the skeletal nodes of the virtual object, the fourth product is mapped onto the corresponding skeletal node of the virtual object to obtain the first target rotation information of the virtual object.

[0137] In step S38, the virtual object is driven based on the first target rotation information.

[0138] In this embodiment of the disclosure, step S38 can refer to step S15, and will not be repeated here.

[0139] In summary, by using a preset standard pose as a relay for the rotation data mapping between the source object and the virtual object in this embodiment, the rotation mapping of each skeletal node of the source object and the virtual object can be achieved simply, quickly, and accurately, thus improving work efficiency. Furthermore, this solution only requires setting the driver component interface to a standardized interface; for different capture devices, the driver algorithm capabilities can be reused, and it supports the expansion to more types of driver devices, achieving decoupling between the capture component and the driver component.

[0140] Figure 6 This is a flowchart illustrating the steps of a fourth virtual object-driven method according to an exemplary embodiment. Figure 6 As shown, the method includes the following steps: In step S41, the initial rotation information of multiple bone nodes corresponding to the target action of the source object is obtained; the multiple bone nodes have a parent-child hierarchical relationship.

[0141] In this embodiment of the invention, step S41 can refer to step S11, and will not be repeated here.

[0142] In step S42, based on the initial rotation information of the child node and the initial rotation information of the corresponding parent node, and the position information of the child node in the coordinate system corresponding to the parent node, the rotation angle of the parent node is adjusted so that the rotation components of the child node and the parent node in the target direction meet the preset requirements; the target direction is the direction in which the rotation components of the child node and the parent node are preset values ​​when they rotate.

[0143] In this embodiment of the invention, step S42 can refer to step S12, and will not be repeated here.

[0144] In step S43, the rotation angle when the rotation component meets the preset requirements is obtained to obtain the target rotation angle, and the rotation information of the parent node and the rotation information of the child node are optimized based on the target rotation angle to obtain the optimized rotation information of the parent node and the optimized rotation information of the child node.

[0145] In this embodiment of the invention, step S43 can refer to step S13, and will not be repeated here.

[0146] In step S44, the optimized rotation information of each bone node of the source object is mapped to each bone node of the preset virtual object to obtain the first target rotation information of each bone node of the virtual object.

[0147] In this embodiment of the invention, step S44 can refer to step S14, and will not be repeated here.

[0148] In step S45, the first target rotation information, root node position, and foot node position corresponding to each lower limb bone node of the virtual object are redirected and optimized to obtain the second target rotation information, optimized root node position, and optimized foot node position, respectively.

[0149] Rotation mapping cannot preserve positional information, which can cause problems such as foot slippage. This invention introduces a lower limb IK (Inverse Kinematics) optimization algorithm to solve this problem.

[0150] The IK optimization algorithm first determines the position information of the terminal bone, and then reverses the position information of the N-level parent bone in the inheritance chain of that bone based on the position information of the terminal bone.

[0151] Specifically, the optimization variables are the first target rotation information, root node position, and foot node position corresponding to each lower limb bone node of the virtual object. The lower limb bone nodes include thigh and knee nodes, and the root node is the hip node.

[0152] Finally, the rotation information of the first target is redirected and optimized to obtain the rotation information of the second target, and the root node position and foot node position are optimized to obtain the optimized root node position and optimized foot node position.

[0153] In one possible implementation, the redirection optimization of the first target rotation information, root node position, and foot node position corresponding to each lower limb bone node of the virtual object includes the following steps: In step S451, a retargeting optimization function is constructed using foot position constraints, foot rotation constraints, regularization constraints, and penalty constraints.

[0154] The redirection optimization function specifically includes constraints on four types of variables. Among them, the regularization constraint refers to the constraint on the position of each node after optimization and the initial position of the node before optimization, so that the difference between the two is not too large.

[0155] Penalty constraints refer to restrictions on unreasonable postures, which can be set according to specific circumstances. For example, it can be set that the knees cannot bend backwards.

[0156] In step S452, the redirection optimization function is used to redirect and optimize the first target rotation information and root node position of each lower limb bone node of the virtual object, so that the foot node of the virtual object lands on the target position after the rotation of the root node and each lower limb bone node; wherein, the root node is the bone node where the hip joint is located; the target position is determined based on the position of the foot node after the source object performs the target action.

[0157] By redirecting optimization to solve the first target rotation information, root node position, and foot node position of each lower limb bone node of the virtual object, the foot node of the virtual object can be placed on the target position to prevent slippage and drift.

[0158] The target location is determined based on the foot position of the source object.

[0159] In one possible implementation, step S451 includes the following steps S4511-S4516: In step S4511, the optimized position of the foot node after redirection optimization is calculated based on the forward dynamics algorithm, and a foot position constraint term is constructed based on the difference between the optimized foot node position and the optimization target of the foot position, wherein the optimization target of the foot position is the foot node position on the source object.

[0160] First, the foot position constraint is the difference between the retargeted foot node position and the foot position optimization objective, which can be expressed as: .

[0161] in, This represents a function that calculates the optimal position of the foot node using the forward dynamics algorithm, with the function variable being... .in, This indicates the optimized root node position. This represents the rotation matrix of each lower limb bone node after optimization.

[0162] The optimization objective for the foot position can be the foot position of the original model.

[0163] This represents the L2 norm.

[0164] In step S4512, the redirected optimized foot node rotation matrix is ​​calculated based on the forward dynamics algorithm, and a foot rotation constraint term is constructed based on the difference between the redirected optimized foot node rotation matrix and the optimization objective of the foot node rotation matrix. The optimization objective of the foot node rotation matrix is ​​the optimized rotation information of the foot nodes on the source object.

[0165] The foot node rotation matrix constraint term is the difference between the redirected optimized foot node rotation matrix and the optimization objective of the foot node rotation matrix, which can be expressed as: .

[0166] in, This function represents the method for calculating the rotation matrix of the foot nodes using forward dynamics. The rotation matrix representing the optimized lower limb bone nodes is a variable of this function.

[0167] The optimization objective of the foot node rotation matrix can be expressed as the foot node rotation matrix in the original model.

[0168] In step S4513, a first regularization constraint term is constructed based on the difference between the optimized position of the root node after redirection optimization and the initial position of the root node before redirection optimization.

[0169] The first regularization constraint term can be expressed as: ,in, This indicates the optimized root node position. This indicates the initial position of the root node before optimization.

[0170] In step S4514, for each lower limb bone node, a second regularization constraint term is constructed based on the difference between the second target rotation information of the lower limb bone node after retargeting optimization and the first target rotation information of the lower limb bone node.

[0171] The second regularization constraint term can be expressed as: ,in, This represents the optimized rotation matrix for each lower limb bone node. This represents the rotation matrix of each lower limb bone node before optimization.

[0172] In step S4515, a penalty constraint term is constructed based on the preset penalty constraint for unreasonable postures.

[0173] The penalty constraint can be represented as This refers to the constraint on unreasonable postures, which can be preset according to specific circumstances. For example, it can be set so that the knees cannot bend backwards.

[0174] In step S4516, a retargeting optimization function is constructed based on the weighted sum of the foot position constraint, the foot rotation preset, the first regular constraint, the second regular constraint, and the penalty constraint.

[0175] Weights can be set for the foot position constraint, foot rotation preset, first regularization constraint, second regularization constraint, and penalty constraint. The weights can be obtained from experimental testing.

[0176] Thus, the resulting retargeting optimization function is: (7) in, , , , , These represent the weights of each constraint term.

[0177] In one possible implementation, step S452 includes: When the redirection optimization function reaches its minimum value, the optimized position value of the foot node, the value of the second target rotation information of the foot node, the optimized position value of the root node, and the value of the second target rotation information of each lower limb bone node are determined so that the foot node of the virtual object lands on the target position after being rotated by the root node and each lower limb bone node.

[0178] Specifically, we minimize equation (3), that is:

[0179] When the redirection optimization function reaches its minimum value, the values ​​of each constraint term are also at their minimum. At this point, the foot node position, foot node rotation matrix, rotation matrices of each lower limb bone node, and position of each lower limb bone node reach their optimal values, ensuring that the foot node position is as close as possible to the foot node position of the source object. Therefore, the foot node of the virtual object can be positioned at the target location after rotation through the root node and each lower limb bone node.

[0180] In step S46, the virtual object is driven based on the first target rotation information, the second target rotation information, the optimized root node position, and the optimized foot node position.

[0181] Specifically, the upper limbs of the virtual object are driven by the rotation information of the first target, and the lower limbs are driven by the rotation information of the second target, optimized heel node position, and optimized foot node position. In this way, the foot node of the virtual object can be placed in the target position, so as not to cause foot slippage and drift.

[0182] In summary, in this embodiment, a redirection optimization function is constructed using foot position constraints, foot rotation constraints, regularization constraints, and penalty constraints. This function is then used to redirect and optimize the first target rotation information and root node position of each lower limb bone node of the virtual object. This ensures that the foot nodes of the virtual object land on the target position after rotating through the root node and each lower limb bone node, thereby avoiding foot slippage and drifting problems and optimizing the driving effect of the virtual object.

[0183] By employing foot position constraints, foot rotation constraints, regularization constraints, and penalty constraints, a redirection optimization function is constructed. The constraints in these four aspects are reasonable and comprehensive, achieving smooth connection between various nodes of the lower limbs.

[0184] In one possible implementation, before constructing the redirection optimization function to redirect and optimize the first target rotation information, root node position, and foot node position corresponding to each lower limb bone node of the virtual object, the following steps S51-S52 are also included: In step S51, the leg lengths of the source object and the virtual object are normalized according to the ratio between the leg length of the source object and the leg length of the virtual object.

[0185] In step S52, the root node position, the position of each lower limb bone node, and the foot node position in the source object and the virtual object are determined according to the normalization processing result.

[0186] In steps S51-S52, in order to handle the large difference in body size between the source model and the target model, normalization can be performed based on the leg length ratio. Then, based on the normalization result, the root node position, the position of each lower limb bone node, and the foot node position in the source object and the virtual object are determined respectively.

[0187] Figure 7 This is a structural block diagram illustrating a virtual object driving device according to an exemplary embodiment. For example... Figure 7 As shown, the virtual object driving device 60 includes: The initial rotation information acquisition module 61 is configured to acquire the initial rotation information of multiple skeletal nodes corresponding to the target action of the source object; the multiple skeletal nodes have a parent-child hierarchical relationship. The rotation angle adjustment module 62 is configured to perform an adjustment of the rotation angle of the parent node based on the initial rotation information of the child node and the corresponding initial rotation information of the parent node, and the position information of the child node in the coordinate system corresponding to the parent node, so that the rotation components of the child node and the parent node in the target direction meet the preset requirements; the target direction is the direction in which the rotation components of the child node and the parent node are preset values ​​when they rotate. The optimized rotation information determination module 63 is configured to obtain the rotation angle when the rotation component meets the preset requirements, obtain the target rotation angle, and optimize the rotation information of the parent node and the rotation information of the child node based on the target rotation angle to obtain the optimized rotation information of the parent node and the optimized rotation information of the child node. The mapping module 64 is configured to perform the mapping of the optimized rotation information of each bone node of the source object to each bone node of the preset virtual object, so as to obtain the first target rotation information of each bone node of the virtual object. The driving module 65 is configured to drive the virtual object based on the first target rotation information.

[0188] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0189] Figure 8 This is a block diagram illustrating an electronic device for virtual object driving according to an exemplary embodiment. Its internal structure diagram can be as follows: Figure 8 As shown, the server or electronic device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a virtual object-driven method. Those skilled in the art will understand that Figure 8The structure shown is merely a block diagram of a portion of the structure related to the present disclosure and does not constitute a limitation on the server or electronic device to which the present disclosure is applied. A specific server or electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements. In an exemplary embodiment, a server or electronic device is also provided, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the virtual object-driven method as described in the embodiments of this disclosure.

[0190] In an exemplary embodiment, a computer-readable storage medium is also provided, which, when executed by a processor of a server or electronic device, enables the server or electronic device to perform the virtual object driving method of the present disclosure embodiments. The computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc.

[0191] In an exemplary embodiment, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute the virtual object driving method of the embodiments of this disclosure.

[0192] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0193] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0194] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A virtual object-driven method, characterized in that, The method includes: Obtain the initial rotation information of multiple bone nodes corresponding to the target action of the source object; the multiple bone nodes have a parent-child hierarchical relationship; Based on the initial rotation information of the child node and the corresponding initial rotation information of the parent node, and the position information of the child node in the coordinate system corresponding to the parent node, the rotation angle of the parent node is adjusted so that the rotation components of the child node and the parent node in the target direction meet the preset requirements; the target direction is the direction in which the rotation components of the child node and the parent node are at a preset value when they rotate; the preset value is 0. Obtain the rotation angle when the rotation component meets the preset requirements, obtain the target rotation angle, and optimize the rotation information of the parent node and the rotation information of the child node based on the target rotation angle to obtain the optimized rotation information of the parent node and the optimized rotation information of the child node; The optimized rotation information of each bone node of the source object is mapped to each bone node of the preset virtual object to obtain the first target rotation information of each bone node of the virtual object. The virtual object is driven based on the rotation information of the first target; Adjusting the rotation angle of the parent node so that the rotation components of the child node and the parent node in the target direction meet preset requirements includes: Based on the rotation component function and the corrected rotation matrix function, the rotation angle of the child node relative to the parent node is adjusted to minimize the value of the rotation component function; the corrected rotation matrix function is a function constructed based on the corrected rotation angle of the parent node and the rotation axis; the rotation component function is a function composed of the rotation components of the child node and the parent node in the target direction; the rotation axis is the axis of rotation of the child node relative to the parent node.

2. The method according to claim 1, characterized in that, Before adjusting the rotation angle of the parent node, the following is also included: Based on the position information of the child node in the coordinate system corresponding to the parent node, determine the rotation axis of the child node relative to the parent node; Construct a corrected rotation matrix function based on the corrected rotation angle of the parent node and the rotation axis; Based on the initial rotation information of the child node, the initial rotation information of the parent node, and the corrected rotation matrix function, the rotation component functions of the child node and the parent node in the target direction are determined.

3. The method according to claim 1, characterized in that, The step of adjusting the rotation angle of the child node relative to the parent node based on the rotation component function and the corrected rotation matrix function to minimize the value of the rotation component function includes: Determine the equation relationship between the optimized rotation information of the parent node, the corrected rotation matrix, and the initial rotation information of the parent node, and determine the equation relationship between the optimized rotation information of the child node, the corrected rotation matrix, and the initial rotation information of the child node to obtain the first equation and the second equation. Using the first equation, the second equation, and the corrected rotation matrix function, the rotation component function is converted into a transformation function with the corrected rotation angle as the variable; Determine the value of the correction rotation angle when the transformation function reaches its minimum value.

4. The method according to claim 2, characterized in that, The process of optimizing the rotation information of the parent node and the child node based on the target rotation angle to obtain the optimized rotation information of the parent node and the optimized rotation information of the child node includes: Based on the corrected rotation matrix function, determine the target rotation matrix corresponding to the target rotation angle; Based on the target rotation matrix and the initial rotation information of the parent node, the optimized rotation information of the parent node is determined, and based on the target rotation matrix and the initial rotation information of the child node, the optimized rotation information of the child node is determined.

5. The method according to claim 1, characterized in that, Before obtaining the initial rotation information of multiple bone nodes corresponding to the target action of the source object, the process also includes: When the source object is in the initial action state, the rotation information relative to the world coordinate system is determined when each child node and its corresponding parent node on the source object rotates to the preset standard pose, and the first global rotation matrix and the second global rotation matrix are obtained; the standard pose is the preset target pose of the standard object. When the preset virtual object is in the initial action state, the rotation information of each child node and its corresponding parent node on the virtual object relative to the world coordinate system when rotated to the standard pose is determined, and the third global rotation matrix and the fourth global rotation matrix are obtained.

6. The method according to claim 5, characterized in that, The optimized rotation information is the optimized local rotation matrix of the bone node relative to its corresponding parent node. Mapping the optimized rotation information of each bone node of the source object to each bone node of the preset virtual object includes: Based on the optimized local rotation matrix, the first global rotation matrix, and the second global rotation matrix, the optimized rotation information of each bone node of the source object is mapped to each bone node of the standard object to obtain the standard rotation information of the standard object. Based on the third global rotation matrix and the fourth global rotation matrix, the standard rotation information is mapped onto each bone node of the preset virtual object to obtain the first target rotation information of each bone node of the virtual object.

7. The method according to claim 6, characterized in that, The step of mapping the optimized rotation information of each bone node of the source object to each bone node of the standard object includes: Determine the first product of the optimized local rotation matrix of the bone node on the source object and the corresponding second global rotation matrix, and determine the product between the first product and the reciprocal of the first global rotation matrix of the bone node to obtain the second product; Determine the first correspondence between the skeletal nodes of the source object and the skeletal nodes of the standard object; Based on the first correspondence, the rotation information corresponding to the second product of the bone node is mapped to the corresponding bone node of the standard object to obtain the standard rotation information of the standard object.

8. The method according to claim 7, characterized in that, The step of mapping the standard rotation information onto each skeletal node of a preset virtual object based on the third and fourth global rotation matrices to obtain the first target rotation information of each skeletal node of the virtual object includes: Determine a second correspondence between the skeletal nodes of the standard object and the skeletal nodes of the virtual object; Based on the second correspondence, the standard rotation information of the bone nodes on the standard object is determined, and the product of the standard rotation information and the third global rotation matrix of the corresponding bone nodes on the virtual object is obtained to obtain the third product; wherein, the standard rotation information is the second product; The fourth product is obtained by determining the product between the third product and the reciprocal of the fourth global rotation matrix; The rotation information corresponding to the fourth product is used as the first target rotation information for each skeletal node of the virtual object.

9. The method according to any one of claims 1-8, characterized in that, Before driving the virtual object based on the first target rotation information, the method further includes: The first target rotation information, root node position, and foot node position corresponding to each lower limb bone node of the virtual object are redirected and optimized to obtain the second target rotation information, optimized root node position, and optimized foot node position, respectively. The step of driving the virtual object based on the first target rotation information includes: The virtual object is driven based on the first target rotation information, the second target rotation information, the optimized root node position, and the optimized foot node position.

10. The method according to claim 9, characterized in that, The process of redirecting and optimizing the first target rotation information, root node position, and foot node position corresponding to each lower limb bone node of the virtual object includes: Construct a retargeting optimization function using foot position constraints, foot rotation constraints, regularization constraints, and penalty constraints; Using the aforementioned redirection optimization function, the first target rotation information and root node position of each lower limb bone node of the virtual object are redirected and optimized, so that the foot node of the virtual object lands on the target position after the rotation of the root node and each lower limb bone node. The root node is the bone node where the hip joint is located; the target position is determined based on the position of the foot node after the source object performs the target action.

11. The method according to claim 10, characterized in that, The retargeting optimization function is constructed using foot position constraints, foot rotation constraints, regularization constraints, and penalty constraints, including: The optimized position of the foot node after redirection optimization is determined based on the forward dynamics algorithm, and a foot position constraint term is constructed based on the difference between the optimized foot node position after redirection optimization and the optimization target of the foot position, wherein the optimization target of the foot position is the target position; The forward dynamics algorithm determines the redirected and optimized foot node rotation matrix, and the foot rotation constraint term is constructed based on the difference between the redirected and optimized foot node rotation information and the optimization target of the foot node rotation information. The optimization target of the foot node rotation information is the optimized rotation information of the foot node on the source object. Based on the difference between the optimized position of the root node after redirection optimization and the initial position of the root node before redirection optimization, a first regularization constraint term is constructed. For each lower limb bone node, a second regularization constraint term is constructed based on the difference between the second target rotation information of the lower limb bone node after retargeting optimization and the first target rotation information of the lower limb bone node. Based on the pre-defined penalty constraints for unreasonable postures, penalty constraint terms are constructed. A retargeting optimization function is constructed based on the weighted sum of the foot position constraint, the foot rotation constraint, the first regularization constraint, the second regularization constraint, and the penalty constraint.

12. The method according to claim 11, characterized in that, The process of using the retargeting optimization function to retarget and optimize the first target rotation information and root node position of each lower limb bone node of the virtual object includes: When the redirection optimization function reaches its minimum value, the optimized position value of the foot node, the value of the second target rotation information of the foot node, the optimized position value of the root node, and the value of the second target rotation information of each lower limb bone node are determined so that the foot node of the virtual object lands on the target position after being rotated by the root node and each lower limb bone node.

13. The method according to any one of claims 10-12, characterized in that, Before retargeting and optimizing the first target rotation information, root node position, and foot node position corresponding to each lower limb bone node of the virtual object, the following steps are also included: The leg lengths of the source object and the virtual object are normalized based on the ratio between the leg length of the source object and the leg length of the virtual object. Based on the normalization results, the root node position, the position of each lower limb bone node, and the foot node position in the source object and the virtual object are determined respectively.

14. A virtual object driving device, characterized in that, The device includes: The initial rotation information acquisition module is configured to acquire the initial rotation information of multiple skeletal nodes corresponding to the target action of the source object; the multiple skeletal nodes have a parent-child hierarchical relationship. The rotation angle adjustment module is configured to perform an adjustment based on the initial rotation information of the child node and the corresponding initial rotation information of the parent node, and the position information of the child node in the coordinate system corresponding to the parent node, to adjust the rotation angle of the parent node so that the rotation components of the child node and the parent node in the target direction meet a preset requirement; the target direction is the direction in which the rotation components of the child node and the parent node are at a preset value when they rotate; the preset value is 0. The optimized rotation information determination module is configured to obtain the rotation angle when the rotation component meets the preset requirements, obtain the target rotation angle, and optimize the rotation information of the parent node and the rotation information of the child node based on the target rotation angle to obtain the optimized rotation information of the parent node and the optimized rotation information of the child node. The mapping module is configured to perform the mapping of the optimized rotation information of each bone node of the source object to each bone node of the preset virtual object, so as to obtain the first target rotation information of each bone node of the virtual object; The driving module is configured to drive the virtual object based on the rotation information of the first target; The rotation angle adjustment module is specifically configured to perform: Based on the rotation component function and the corrected rotation matrix function, the rotation angle of the child node relative to the parent node is adjusted to minimize the value of the rotation component function; the corrected rotation matrix function is a function constructed based on the corrected rotation angle of the parent node and the rotation axis; the rotation component function is a function composed of the rotation components of the child node and the parent node in the target direction; the rotation axis is the axis of rotation of the child node relative to the parent node.

15. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the virtual object driving method as described in any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the server's processor, the server is able to perform the virtual object driving method as described in any one of claims 1 to 13.

Images