Method and apparatus for re-targeting motion capture

Abstract

The present disclosure provides a motion capture redirection method and device, relates to the technical field of artificial intelligence, in particular to the technical fields of computer vision, augmented reality, virtual reality, deep learning and the like, and can be applied to metaverse, virtual digital people and the like. It comprises: obtaining a first bone node in a first skeletal animation, obtaining a second bone node and a third bone node in a second skeletal animation, the second skeletal animation being obtained by performing rotation-consistent redirection on the first skeletal animation, the first bone node and the second bone node being bone nodes of the same part, calculating incremental rotation information of the third bone node according to first position information of the first bone node, second position information of the second bone node and third position information of the third bone node, and performing optimization processing on original rotation information of the third bone node according to the incremental rotation information to obtain a redirected third bone node, thereby avoiding loss of semantic information and improving the reliability of the redirection.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, in particular to the technical field of computer vision, augmented reality, virtual reality, deep learning and the like, which can be applied to the scene of meta universe, virtual digital person and the like, and in particular to a motion capture redirection method and device. BACKGROUND

[0002] With the wide application of three-dimensional virtual images, such as the application of three-dimensional virtual images in social, live broadcast, game and the like, the application value of three-dimensional virtual images is highlighted.

[0003] In some embodiments, rotation can be migrated in a redirected manner, and driving of the three-dimensional virtual image is completed to ensure rotation consistency of the redirection. SUMMARY

[0004] The present disclosure provides a motion capture redirection method and device for improving the reliability of redirection.

[0005] According to a first aspect of the present disclosure, a motion capture redirection method is provided, comprising:

[0006] obtaining a first bone node in a first skeletal animation, obtaining a second bone node and a third bone node in a second skeletal animation, wherein the second skeletal animation is a skeletal animation obtained by rotation-consistent redirection of the first skeletal animation, and the first bone node and the second bone node are bone nodes of the same part in different bone models;

[0007] According to the first position information of the first bone node, the second position information of the second bone node and the third position information of the third bone node, incremental rotation information of the third bone node is calculated;

[0008] According to the incremental rotation information, the original rotation information of the third bone node is optimized to obtain a redirected third bone node.

[0009] According to a second aspect of the present disclosure, a motion capture redirection device is provided, comprising:

[0010] an obtaining unit configured to obtain a first bone node in a first skeletal animation, and obtain a second bone node and a third bone node in a second skeletal animation, wherein the second skeletal animation is a skeletal animation obtained by rotation-consistent redirection of the first skeletal animation, and the first bone node and the second bone node are bone nodes of the same part in different bone models;

[0011] a calculation unit, configured to calculate, according to the first position information of the first bone node, the second position information of the second bone node, and the third position information of the third bone node, the incremental rotation information of the third bone node;

[0012] an optimization unit, configured to optimize the original rotation information of the third bone node according to the incremental rotation information, to obtain a redirected third bone node.

[0013] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0014] at least one processor; and

[0015] a memory connected with the at least one processor in communication; wherein,

[0016] the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of the first aspect.

[0017] According to a fourth aspect of the present disclosure, a non-transitory computer readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to perform the method of the first aspect.

[0018] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising: a computer program stored in a readable storage medium, and at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to enable the electronic device to perform the method of the first aspect.

[0019] The motion capture redirection method and device provided by the present disclosure include: obtaining a first bone node in a first skeletal animation, obtaining a second bone node and a third bone node in a second skeletal animation, wherein the second skeletal animation is a skeletal animation obtained by performing rotation-consistent redirection on the first skeletal animation, the first bone node and the second bone node are bone nodes of the same part in different bone models, and the third bone node is calculated according to first position information of the first bone node, second position information of the second bone node, and third position information of the third bone node, the original rotation information of the third bone node is optimized according to the incremental rotation information of the third bone node, and the redirected third bone node is obtained. By combining the position information (first position information, second position information, and third position information) corresponding to each bone node (first bone node, second bone node, and third bone node), the incremental rotation information is calculated to combine the incremental rotation information and the original rotation information to complete the redirection. The difference between different bone models is considered, the disadvantage of losing part of the semantic information caused by directly completing the redirection based on the original rotation information is avoided, and the effectiveness and reliability of the redirection are improved.

[0020] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0021] The accompanying drawings are used to better understand the present scheme and do not limit the present disclosure. Among them:

[0022] Figure 1 is a schematic diagram of a bone model according to an embodiment of the present disclosure;

[0023] Figure 2 is a schematic diagram according to a first embodiment of the present disclosure;

[0024] Figure 3 is a first skeletal animation schematic diagram of a first bone model according to the present disclosure;

[0025] Figure 4 is a second skeletal animation schematic diagram of a second bone model according to the present disclosure;

[0026] Figure 5 is a schematic diagram according to a second embodiment of the present disclosure;

[0027] Figure 6 is a principle schematic diagram of a motion capture redirection method according to an embodiment of the present disclosure;

[0028] Figure 7 is a third skeletal animation schematic diagram of a second bone model according to the present disclosure;

[0029] Figure 8 is a schematic diagram according to a third embodiment of the present disclosure;

[0030] Figure 9 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0031] Figure 10 is a schematic diagram according to a fifth embodiment of the present disclosure;

[0032] Figure 11 is a block diagram of an electronic device for implementing a redirection method of motion capture according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to help understanding, and should be considered as merely exemplary. Thus, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, for the sake of brevity and clarity, descriptions of well-known functions and constructions are omitted from the following description.

[0034] To facilitate the understanding of the present disclosure, at least some of the terms of the present disclosure are explained as follows:

[0035] Augmented Reality (AR) technology refers to a technology of skillfully fusing virtual information with the real world, and widely uses multimedia, three-dimensional modeling, real-time tracking and registration, intelligent interaction, sensing and other technical means, and simulates virtual information such as computer-generated text, images, three-dimensional models, music, video, etc., and applies it to the real world, so that the two kinds of information complement each other, thereby achieving the "augmentation" of the real world.

[0036] Virtual Reality (VR) is also called virtual reality or spiritual environment technology. Virtual reality technology includes computer, electronic information, simulation technology, and its basic implementation is mainly based on computer technology, which uses and integrates three-dimensional graphics technology, multimedia technology, simulation technology, display technology, servo technology and other latest developments of various high-tech technologies, and generates a virtual world with realistic three-dimensional vision, touch, smell and other sensory experiences through computers and other devices, so that people in the virtual world have a sense of being there.

[0037] Deep Learning (DL) is a subfield of Machine Learning (ML) that focuses on learning the internal rules and representation levels of sample data, and the information obtained in this learning process is very helpful for the interpretation of data such as text, images, and sound.

[0038] Metaverse refers to a virtual world of digital life space constructed by using digital technology, which is mapped from or beyond the real world and can interact with the real world.

[0039] Virtual digital human, also known as three-dimensional virtual image, refers to a computer-generated character with a three-dimensional image, which can be understood as a three-dimensional human model.

[0040] The skeleton model, also known as the skeleton tree, is used to represent a three-dimensional virtual image, which includes a human model with skeleton nodes (also known as joint nodes).

[0041] As shown in Figure 1 The skeleton model includes multiple skeleton nodes, such as the skeleton node of the left hand, the skeleton node of the left elbow, the skeleton node of the left shoulder, and so on, which will not be listed here.

[0042] It should be understood that Figure 1 It is only used to illustrate the skeleton model and the skeleton node, and cannot be understood as a limitation on the skeleton model and the skeleton node.

[0043] Redirection refers to giving a character's action and migrating this action to another character to keep the action consistent.

[0044] Correspondingly, in this embodiment, redirection can be understood as giving an action in the first skeleton model (which can be referred to as the first skeleton animation or the first skeleton pose), migrating this action to the second skeleton model, and keeping the action of the second skeleton model (which can be referred to as the second skeleton animation or the second skeleton pose) consistent with the action of the first skeleton model, that is, the second skeleton animation after redirection is the same as the first skeleton animation.

[0045] With the wide application of three-dimensional virtual images, such as the application of three-dimensional virtual images in social, live, and game scenarios, the application value of three-dimensional virtual images is highlighted.

[0046] For example, to drive a three-dimensional virtual image, a large amount of art cost is needed to build keyframe skeleton animation, and obtaining driving parameters of a three-dimensional virtual image through motion capture of a video image can effectively reduce the art cost and meet the needs of users for various action production.

[0047] Therefore, in some embodiments, the driving of the three-dimensional virtual image can be completed by obtaining the driving parameters of the virtual image from the video image and based on the obtained driving parameters.

[0048] For example, the driving of the three-dimensional virtual image can be completed by migrating rotation in a redirected manner to ensure the rotation consistency of the redirection.

[0049] However, due to the differences between different bone models, especially when the difference between two different bone models is large, the rotation-consistent redirection can cause the loss of part of the semantic information, thereby causing the technical problem of low effectiveness and reliability of the redirection.

[0050] To avoid at least one of the above technical problems, the present disclosure provides a technical concept after creative labor: obtaining a first bone node in a first bone animation, obtaining a second bone node and a third bone node in a second bone animation, the second bone animation being a bone animation obtained by rotation-consistent redirection of the first bone animation, the first bone node and the second bone node being bone nodes of the same part, determining the incremental rotation information of the third bone node in combination with the position information corresponding to the first bone node, the second bone node and the third bone node, and determining the redirected third bone node according to the incremental rotation information.

[0051] Based on the above technical concept, the present disclosure provides a motion capture redirection method and device, which relates to the field of artificial intelligence, specifically the fields of computer vision, augmented reality, virtual reality, deep learning, etc., and can be applied to scenarios such as metaverse and virtual digital people to achieve the effectiveness and reliability of driving virtual images.

[0052] Figure 2 is a schematic diagram according to the first embodiment of the present disclosure, as Figure 2 shown, the motion capture redirection method provided by the embodiment of the present disclosure comprises:

[0053] S201: obtaining a first bone node in a first bone animation, obtaining a second bone node and a third bone node in a second bone animation.

[0054] Among them, the second bone animation is a bone animation obtained by rotation-consistent redirection of the first bone animation, and the first bone node and the second bone node are bone nodes of the same part in different bone models.

[0055] For example, the execution subject of the embodiment can be a motion capture redirection device (hereinafter referred to as a redirection device), which can be a server, a computer, a terminal device, a processor, a chip, etc., which will not be listed one by one here.

[0056] For example, if the redirection device is a server, the redirection device can be a standalone server or a server cluster; can be a cloud server or a local server, and the embodiments are not limited thereto.

[0057] For another example, the motion capture redirection method of the embodiments is applied to an augmented reality scene or a virtual reality scene, and the redirection device can be a head-mounted device in the augmented reality scene or the virtual reality scene.

[0058] The first skeletal animation and the second skeletal animation are relative concepts and cannot be understood as a limitation on the first skeletal animation and the second skeletal animation. In combination with the above example, the first skeletal animation can be understood as a skeletal animation of the first skeletal model, for example, the first skeletal animation is a skeletal animation of a certain frame corresponding to the first skeletal model. The second skeletal animation is a skeletal animation obtained by rotation-consistent redirection of the first skeletal animation.

[0059] The rotation-consistent redirection refers to migration of rotation information of the first skeletal animation to obtain the second skeletal animation by redirection, and the specific implementation principle is not limited in the embodiments, and can be referred to related technologies.

[0060] For example, Figure 3 is a first skeletal animation diagram of a first skeletal model according to the present disclosure, which represents an animation of a squatting action. Figure 3 Three skeletal nodes in the first skeletal model are exemplarily shown, which are a skeletal node of a hip, a skeletal node of a knee, and a skeletal node of a foot, and Figure 3 The animation of the squatting action of the first skeletal model is exemplarily shown, that is, Figure 3 The change of the skeletal nodes of the first skeletal model in the squatting action is exemplarily shown.

[0061] Figure 4 is a second skeletal animation diagram of a second skeletal model, which represents an animation diagram obtained by motion capture redirection of the first skeletal animation of the first skeletal model, and specifically is obtained by migration of the animation of the squatting action of the first skeletal model (i.e. the first skeletal animation) from the dimension of rotation information.

[0062] Correspondingly, Figure 4 Three skeletal nodes in the second skeletal model are exemplarily shown, which are a skeletal node of a hip, a skeletal node of a knee, and a skeletal node of a foot. And Figure 4 The skeletal node of the hip in is obtained by migrating the rotation information of the skeletal node of the hip shown in Figure 3 The skeletal node of the hip shown in is obtained by migrating the rotation information of the skeletal node of the hip shown in Figure 4 The skeletal node of the knee in is obtained by migrating the rotation information of the skeletal node of the knee shown in Figure 3 The skeletal node of the knee shown in is obtained by migrating the rotation information of the skeletal node of the knee shown inFigure 4 The bone node of the foot in the first skeleton animation is obtained by migrating the rotation information of the bone node of the foot shown in the second skeleton animation. Figure 3 The bone node of the foot shown in the second skeleton animation is obtained by migrating the rotation information of the bone node of the foot shown in the first skeleton animation.

[0063] Similarly, the first bone node, the second bone node and the third bone node are relative concepts and cannot be understood as a limitation on the first bone node, the second bone node and the third bone node.

[0064] For example, the first bone node can be a bone node of a foot shown in the first skeleton animation, as shown in FIG. 1A. Figure 3 Since the first bone node is a bone node of the same part as the second bone node, the second bone node is a bone node of a foot shown in the second skeleton animation, as shown in FIG. 1B. Figure 4 Since the third bone node and the second bone node are both bone nodes in the second skeleton animation, the third bone node can be a bone node of a hip shown in the second skeleton animation, as shown in FIG. 1B. Figure 4

[0065] It should be understood that the above is only used to exemplarily illustrate possible first bone nodes, second bone nodes and third bone nodes, and cannot be understood as a limitation on the selection of the first bone node, the second bone node and the third bone node.

[0066] It can be understood that the embodiment does not limit the order of obtaining the first bone node, the second bone node and the third bone node.

[0067] For example, the first bone node can be obtained from the first skeleton animation first, and then the second bone node of the same part as the first bone node can be obtained from the second skeleton animation, and finally a bone node (i.e., the third bone node) different from the second bone node can be obtained from the second skeleton animation. The third bone node is a redirected bone node and can be understood as a bone node to be optimized.

[0068] Since the bone node to be optimized is a bone node in the second skeleton animation, the bone node to be optimized, such as the third bone node, can be obtained first, and then a bone node different from the third bone node can be obtained from the second skeleton animation as the second bone node, and finally a bone node of the same part as the second bone node, i.e., the first bone node, can be obtained from the first skeleton animation.

[0069] S202: Obtain the incremental rotation information of the third bone node according to the first position information of the first bone node, the second position information of the second bone node and the third position information of the third bone node.

[0070] ​The position information can be understood as coordinates, and accordingly, the first position information can be understood as coordinates of the first bone node, the second position information can be understood as coordinates of the second bone node, and the third position information can be understood as coordinates of the third bone node.

[0071] The incremental rotation information can be understood as an incremental change in the rotation dimension of the third bone node based on the second bone animation.

[0072] For example, in combination with the above example, the third bone node is a bone node of a hip in the second bone animation, and the incremental rotation information can be understood as an incremental change in the rotation of the bone node of the hip relative to the bone node of the hip in the second bone animation obtained by rotation-consistent reorientation.

[0073] That is, in the embodiment, in order to avoid directly migrating the rotation of the bone node of the hip in the first bone animation to the third bone node, the difference between the first bone model and the second bone model can be considered to determine the difference change when the rotation of the bone node of the hip in the first bone animation is migrated to the third bone node, and the difference change is the incremental rotation information.

[0074] S203: Optimizing the original rotation information of the third bone node according to the incremental rotation information to obtain a reoriented third bone node.

[0075] The original rotation information can be understood as the rotation when the reorientation is based on rotation consistency.

[0076] In combination with the above analysis, on the one hand, since the incremental rotation information is determined by considering the difference between the first bone model and the second bone model, when the reoriented third bone node is obtained in combination with the incremental rotation information, it is equivalent to considering both the semantic information of the first bone model and the semantic information of the second bone model, thereby avoiding the disadvantage of loss of semantic information.

[0077] On the other hand, since the difference between the first bone model and the second bone model is determined based on the respective position information (first position information, second position information, third position information) of each bone node (first bone node, second bone node, third bone node), the reorientation considers the position difference of the bone nodes between the first bone model and the second bone model, and therefore, the consistency of the position dimension can be realized to avoid causing the reality of the three-dimensional virtual image to be low, thereby improving the visual experience and interactive experience of the user.

[0078] Based on the above analysis, the motion capture redirection method provided by the present disclosure comprises: obtaining a first bone node in a first skeletal animation, obtaining a second bone node and a third bone node in a second skeletal animation, wherein the second skeletal animation is a skeletal animation obtained by performing rotation-consistent redirection on the first skeletal animation, the first bone node and the second bone node are bone nodes of the same part in different bone models, and the third bone node is calculated based on first position information of the first bone node, second position information of the second bone node, and third position information of the third bone node to obtain incremental rotation information of the third bone node, and the original rotation information of the third bone node is optimized based on the incremental rotation information to obtain a redirected third bone node. In this embodiment, the incremental rotation information is calculated by combining the position information (first position information, second position information, and third position information) corresponding to each bone node (first bone node, second bone node, and third bone node), and the technical feature of combining the incremental rotation information and the original rotation information to complete the redirection is considered. The difference between different bone models is considered, the disadvantage of losing part of the semantic information caused by directly completing the redirection based on the original rotation information is avoided, and the effectiveness and reliability of the redirection are improved.

[0079] To enable the reader to have a more profound understanding of the implementation principle of the present disclosure, the following will describe the present disclosure in detail in conjunction with Figure 5 The motion capture redirection method of the present disclosure will be described in more detail. Among them, Figure 5 is a schematic diagram according to the second embodiment of the present disclosure, as Figure 5 indicated, the motion capture redirection method provided by the present embodiment comprises:

[0080] S501: obtaining a first bone node in a first skeletal animation, and obtaining a second bone node and a third bone node in a second skeletal animation.

[0081] Among them, the second skeletal animation is a skeletal animation obtained by performing rotation-consistent redirection on the first skeletal animation, and the first bone node and the second bone node are bone nodes of the same part in different bone models.

[0082] It can be understood that, in order to avoid tedious description, the same technical features as in the above embodiments will not be limited in this embodiment.

[0083] For example, for the implementation principle of S501, please refer to the description of S201, which will not be repeated here.

[0084] In some embodiments, the second bone node and the third bone node are bone nodes having a parent-child node relationship in the second skeletal animation.

[0085] For example, the second bone node and the third bone node can be bone nodes having a direct connection relationship.

[0086] For example, such as Figure 4 As shown, the second bone node can be a bone node of the foot, and the third bone node can be a bone node of the knee.

[0087] In other embodiments, the second and third bone nodes are bone nodes in the second skeletal animation that have a grandparent-grandchild relationship.

[0088] For example, the second and third skeletal nodes can be skeletal nodes with an indirect connection relationship.

[0089] For example, such as Figure 4 As shown, the second bone node can be a bone node of the foot, and the third bone node can be a bone node of the hip.

[0090] In this embodiment, two bone nodes with a parent-child relationship can be designated as the second and third bone nodes, or two bone nodes with a grandparent-grandchild relationship can be designated as the second and third bone nodes. This ensures that the second and third bone nodes are bone nodes with a direct connection or an indirect connection. Due to the transmission of force, bone nodes with a direct connection or an indirect connection have a certain correlation and interaction in their actions. Therefore, by designating bone nodes with a direct connection or an indirect connection as the second and third bone nodes, the effectiveness and reliability of redirection can be achieved.

[0091] S502: Based on the first position information and the third position information, the first difference information is calculated, and based on the second position information and the third position information, the second difference information is calculated.

[0092] For example, the first skeletal node is as follows: Figure 6 The target constraint point (target_point A) is shown below, and the second skeleton node is as follows: Figure 6 The point to be constrained (source_point B) shown, the third bone node is as follows: Figure 6 The point to be optimized (node ​​C) is shown.

[0093] Accordingly, the first difference information is the difference between the target constraint point and the point to be optimized in the position dimension; the second difference information is the difference between the point to be constrained and the point to be optimized in the position dimension.

[0094] In some embodiments, the first difference information and the second difference information can be determined by constructing vectors.

[0095] For example, the first difference information can be represented by a vector Vec_src, and the vector Vec_src can be represented by formula 1, formula 1:

[0096] Vec_src = first position information - third position information.

[0097] The second difference information can be represented by a vector Vec_tar, and the vector Vec_tar can be represented by formula 2, formula 2:

[0098] Vec_tar = second position information - third position information.

[0099] Correspondingly, in combination with the above analysis, if the target constraint point is a skeletal node of a foot as shown in Figure 3 , the to-be-constrained point is a skeletal node of a foot as shown in Figure 4 , and the to-be-optimized point is a skeletal node of a hip as shown in Figure 4 , then the first difference information = position information of the skeletal node of the foot (i.e., the first position information) - the skeletal node of the hip (i.e., the third position information) as shown in Figure 3 ; the second difference information = position information of the skeletal node of the foot (i.e., the second position information) - the skeletal node of the hip (i.e., the third position information) as shown in Figure 4 . Figure 4 Figure 4

[0100] S503: determining the incremental rotation information according to the first difference information and the second difference information.

[0101] In combination with the above analysis, it can be known that the first difference information is the difference information of the first skeletal node and the third skeletal node in the position dimension, the second difference information is the difference information of the second skeletal node and the third skeletal node in the position dimension, and the first skeletal node and the second skeletal node are skeletal nodes of the same part of two different skeletal models.

[0102] Therefore, the first difference information and the second difference information can represent the difference of the two skeletal models in the position dimension, and determining the incremental rotation information through the difference of the two skeletal models in the position dimension can make the incremental rotation information highly related to the difference of the two skeletal models in the position dimension, so that when the redirection is performed based on the incremental rotation information, the consistency of the position dimension can be realized, and the effectiveness and reliability of the redirection are improved.

[0103] In some embodiments, S503 can include the following steps:

[0104] First step: calculating the axis angle vector and radian of the third skeletal node according to the first difference information and the second difference information.

[0105] ​​The axis-angle vector refers to a rotation expressed by four elements, with three elements describing the axis of rotation and the other element describing the angle of rotation.

[0106] For example, the three elements used to describe the rotation axis are the x-coordinate, y-coordinate, and z-coordinate of the third bone node, and the element used to describe the angle is θ.

[0107] In some embodiments, the axis-angle vector is obtained by multiplying the first difference information and the second difference information. The radians are obtained by dot product of the first difference information and the second difference information.

[0108] For example, such as Figure 6 As shown, the first difference information and the second difference information are multiplied to obtain the axis-angle vector; the first difference information and the second difference information are multiplied to obtain the radians.

[0109] Based on the above analysis, the first difference information can be represented by the vector Vec_src, and the second difference information can be represented by the vector Vec_tar. Correspondingly, the axis-angle vector axis can be calculated using Equation 3: Equation 3:

[0110] axis=cross(Vec_src, Vec_tar).

[0111] Here, cross represents the product operation.

[0112] The angle in radians can be calculated using Equation 4:

[0113] angle = dot(Vec_src, Vec_tar)

[0114] Here, dot represents dot multiplication.

[0115] For the calculation principles of product and dot product operations, please refer to relevant technical documents, which will not be elaborated here.

[0116] The second step is to determine the incremental rotation information based on the axis-angle vector and radians.

[0117] For example, such as Figure 6 As shown, after obtaining the axis-angle vector and radians, incremental rotation information can be calculated based on the axis-angle vector and radians.

[0118] In this embodiment, incremental rotation information is determined from two dimensions: axis-angle vector and radians, so that the incremental rotation information has high effectiveness and reliability.

[0119] In some embodiments, the second step may include: performing rotation information transformation processing on the axis-angle vector and radians according to the Rodrigues rotation algorithm to obtain incremental rotation information.

[0120] For example, the incremental rotation information can be represented by a matrix, which can also be referred to as an incremental rotation matrix, and the incremental rotation information rotation_diff_matrix can be calculated by formula 5.

[0121] rotation_diff_matrix = axisangle_to_matrix(axis, a * angle).

[0122] Wherein, axisangle_to_matrix represents a Rodrigues rotation algorithm, and a is a preset step coefficient, 0 < a < 1.

[0123] It should be understood that the preset step coefficient can be determined based on requirements, historical records, and experiments, and the present embodiment is not limited.

[0124] S504: Perform a product operation on the incremental rotation information and the original rotation information to obtain optimized rotation information.

[0125] Wherein, the rotation information of the redirected third bone node is the optimized rotation information.

[0126] For example, as Figure 6 indicated, the to-be-optimized point (i.e., the third bone node) has original rotation information, and the product operation is performed on the incremental rotation information and the original rotation information, so as to obtain the optimized rotation information, i.e., the redirected rotation information of the to-be-optimized node.

[0127] In combination with the above examples, in combination with Figure 3 and Figure 4 , if the third bone node is a hip bone node as Figure 4 indicated, the original rotation information can be understood as the rotation information obtained by performing rotation consistency redirection on the hip bone node as Figure 3 indicated.

[0128] In some embodiments, the optimized rotation information rotationC can be calculated by formula 6.

[0129] rotationC = rotation * rotation_diff_matrix

[0130] Wherein, rotation is the original rotation information.

[0131] In this embodiment, by combining the original rotation information and the incremental rotation information, the optimized rotation information is determined, which not only considers the result of rotation consistency redirection (characterized by the original rotation information), but also considers the difference between the two skeleton models (characterized by the incremental rotation information), so that the optimized rotation information has high reliability and high position consistency, that is, the positions of the skeleton nodes after redirection are relatively reliable and the possibility of deviation is small.

[0132] As shown in Figure 7 , Figure 7 the third skeleton animation obtained by the action capture redirection of the method of this embodiment. Figure 3

[0133] That is, as shown in Figure 3 , Figure 4 , Figure 7 , Figure 3 the first skeleton animation shown is the animation of the squatting action, Figure 4 the second skeleton animation is the animation of the squatting action obtained by the rotation migration of the first skeleton animation using the rotation consistency redirection method, Figure 7 the third skeleton animation is the animation of the squatting action obtained by the position consistency migration of the first skeleton animation using the method provided by the embodiments of the present disclosure.

[0134] As shown in Figure 3 and Figure 4 , if the rotation consistency redirection method is used to redirect the first skeleton animation as shown in Figure 3 , in order to ensure consistency, the squatting angle of the second skeleton animation is known as the squatting angle of the first skeleton animation, and since the first skeleton and the second skeleton have differences, as shown in Figure 4 , the skeleton nodes of the feet of the second skeleton animation will lift off the ground and drift, and the quality of the second skeleton animation is relatively low, and the user experience is relatively poor.

[0135] And by the action capture redirection method of this embodiment, since both rotation consistency and the difference between the first skeleton model and the second skeleton model are considered, position consistency redirection is achieved. As shown in Figure 7 , the squatting angle will change accordingly, and the skeleton nodes of the feet of the third skeleton animation will not lift off the ground and avoid drifting.

[0136] ​Therefore, by the action capture and redirection method of the embodiment, the consistency of semantics such as contact and stay of the skeletal animation (also referred to as skeletal animation) before and after migration of different skeletal models is ensured, the migration of action capture in different virtual images can be implemented, the range of action capture capability is improved, the expressiveness of animation migration is more complete and loss is reduced, the quality of skeletal animation is improved as a whole, and the visual effect and user experience are enhanced.

[0137] It should be noted that the action capture is the action capture of multiple skeletal nodes, and the above embodiment is only exemplarily described by taking the action capture skeletal node as the third skeletal node. The action capture can be understood as local action capture (or referred to as local skeletal node optimization). In order to implement the whole action capture (or referred to as global skeletal node optimization), different skeletal nodes can be taken as the third skeletal node to respectively execute the above-described embodiments, so as to implement the global action capture.

[0138] For example, in combination with Figure 4 As shown in the figure, if the third skeletal node acquired for the first time is the skeletal node of the hip, the skeletal node of the knee can be taken as the third skeletal node for the second time, the skeletal node of the leg can be taken as the third skeletal node for the third time, and so on, until the global action capture is implemented.

[0139] In combination with the above analysis, the connection relationship between the second skeletal node and the third skeletal node is not limited, and can be the parent-child node relationship skeletal node or the grandparent-grandchild node relationship skeletal node. Therefore, in some embodiments, different second skeletal nodes can be acquired for each third skeletal node, and the first skeletal node also changes accordingly, so as to continuously perform local action capture on the third skeletal node based on different first skeletal nodes and second skeletal nodes, thereby improving the effectiveness and reliability of the redirection.

[0140] For example, the third skeletal node is the skeletal node of the hip in the second skeletal animation, the second skeletal node selected for the first time is the skeletal node of the leg in the second skeletal animation, and the first skeletal node selected for the first time is the skeletal node of the leg in the first skeletal animation. After the first optimization processing of the third skeletal node in combination with the first selected first skeletal node and second skeletal node, the second skeletal node and the first skeletal node are reselected, such as taking the skeletal node of the knee in the second skeletal animation as the second skeletal node and taking the skeletal node of the knee in the first skeletal animation as the first skeletal node. The third skeletal node (i.e., the skeletal node of the hip in the second skeletal animation) is subjected to the second optimization processing, and so on. Here, the above is not listed one by one.

[0141] Figure 8 is a schematic diagram according to the third embodiment of the present disclosure, as Figure 8As shown, the action capture redirection apparatus 800 in the present disclosure comprises:

[0142] The acquisition unit 801 is configured to acquire a first bone node in a first skeletal animation, and acquire a second bone node and a third bone node in a second skeletal animation, wherein the second skeletal animation is a skeletal animation obtained by performing rotation-consistent redirection on the first skeletal animation, and the first bone node and the second bone node are bone nodes of the same part in different bone models.

[0143] The calculation unit 802 is configured to calculate, according to first position information of the first bone node, second position information of the second bone node, and third position information of the third bone node, incremental rotation information of the third bone node.

[0144] The optimization unit 803 is configured to perform optimization processing on original rotation information of the third bone node according to the incremental rotation information, to obtain a redirected third bone node.

[0145] Figure 9 is a schematic diagram according to the fourth embodiment of the present disclosure, as Figure 9 As shown, the action capture redirection apparatus 900 in the present disclosure comprises:

[0146] The acquisition unit 901 is configured to acquire a first bone node in a first skeletal animation, and acquire a second bone node and a third bone node in a second skeletal animation, wherein the second skeletal animation is a skeletal animation obtained by performing rotation-consistent redirection on the first skeletal animation, and the first bone node and the second bone node are bone nodes of the same part in different bone models.

[0147] The calculation unit 902 is configured to calculate, according to first position information of the first bone node, second position information of the second bone node, and third position information of the third bone node, incremental rotation information of the third bone node.

[0148] In some embodiments, in combination with Figure 9 It can be known that the calculation unit 902 comprises:

[0149] The calculation sub-unit 9021 is configured to calculate first difference information based on the first position information and the third position information, and calculate second difference information based on the second position information and the third position information.

[0150] The determination sub-unit 9022 is configured to determine the incremental rotation information according to the first difference information and the second difference information.

[0151] In some embodiments, the determination sub-unit 9022 comprises:

[0152] a calculation module, configured to calculate an axis angle vector and an arc of the third bone node according to the first difference information and the second difference information.

[0153] a determination module, configured to determine the incremental rotation information according to the axis angle vector and the arc.

[0154] In some embodiments, the axis angle vector is obtained by performing a product operation on the first difference information and the second difference information; and the arc is obtained by performing a dot product operation on the first difference information and the second difference information.

[0155] In some embodiments, the determination module is configured to perform rotation information conversion processing on the axis angle vector and the arc according to a Rodrigues rotation algorithm to obtain the incremental rotation information.

[0156] an optimization unit 903, configured to perform optimization processing on original rotation information of the third bone node according to the incremental rotation information to obtain a redirected third bone node.

[0157] In some embodiments, the optimization unit 903 is configured to perform a product operation on the incremental rotation information and the original rotation information to obtain optimized rotation information.

[0158] wherein the rotation information of the redirected third bone node is the optimized rotation information.

[0159] In some embodiments, the second bone node and the third bone node are bone nodes having a parent-child node relationship in the second bone animation; or,

[0160] the second bone node and the third bone node are bone nodes having a grandparent-grandchild node relationship in the second bone animation.

[0161] Figure 10 is a schematic diagram according to the fifth embodiment of the present disclosure, as Figure 10 shown, the electronic device 1000 in the present disclosure can include a processor 1001 and a memory 1002.

[0162] The memory 1002 is configured to store programs; the memory 1002 can include volatile memory (for example, random-access memory (RAM), such as static random-access memory (SRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM), etc.), and can also include non-volatile memory (for example, flash memory). The memory 1002 is configured to store computer programs (for example, application programs, functional modules, etc. for implementing the above method), computer instructions, etc. The above computer programs, computer instructions, etc. can be stored in one or more memories 1002 in a partitioned manner. And the above computer programs, computer instructions, data, etc. can be called by the processor 1001.

[0163] The above computer programs, computer instructions, etc. can be stored in one or more memories 1002 in a partitioned manner. And the above computer programs, computer instructions, data, etc. can be called by the processor 1001.

[0164] The processor 1001 is configured to execute the computer programs stored in the memory 1002 to implement each step in the method related by the above embodiments.

[0165] For details, please refer to the related description in the above method embodiments.

[0166] The processor 1001 and the memory 1002 can be an independent structure, or an integrated structure. When the processor 1001 and the memory 1002 are independent structures, the memory 1002 and the processor 1001 can be coupled and connected through the bus 1003.

[0167] The electronic device of the embodiment can execute the technical solutions in the above method, and the specific implementation process and technical principles are the same, which will not be repeated here.

[0168] It should be noted that the bone model (for example, the first bone model and the second bone model) in the embodiment is not a bone model for a specific user, and cannot reflect the personal information of a specific user. It should be noted that the bone model in the embodiment comes from a public data set.

[0169] In the technical solutions of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information (such as bone nodes) comply with relevant laws and regulations and do not violate public order and good customs.

[0170] According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0171] According to embodiments of the present disclosure, the present disclosure also provides a computer program product, which comprises a computer program stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to make the electronic device execute the scheme provided in any of the above embodiments.

[0172] Figure 11 A schematic block diagram of an example electronic device 1100 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.

[0173] As shown in Figure 11 The device 1100 includes a computing unit 1101 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded into a random access memory (RAM) 1103 from a storage unit 1108. Various programs and data required for the operation of the device 1100 can also be stored in the RAM 1103. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other through a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0174] Various components in the device 1100 are connected to the I / O interface 1105, including an input unit 1106, such as a keyboard, a mouse, etc., an output unit 1107, such as various types of displays, speakers, etc., a storage unit 1108, such as a magnetic disk, an optical disk, etc., and a communication unit 1109, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1109 allows the device 1100 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0175] The computing unit 1101 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1101 performs various methods and processes described above, such as the redirection method of motion capture. For example, in some embodiments, the redirection method of motion capture can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1108. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 1100 via the ROM 1102 and / or the communication unit 1109. When the computer program is loaded onto the RAM 1103 and executed by the computing unit 1101, one or more steps of the redirection method of motion capture described above can be performed. Alternatively, in other embodiments, the computing unit 1101 can be configured to perform the redirection method of motion capture by any other suitable means, such as by means of firmware.

[0176] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0177] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a means for implementing the functions / acts specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0178] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0179] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0180] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0181] The computer system can include clients and servers. This relationship can be. The servers are generally remote from the users and can be accessed via the Internet using a communication network. The relationship can be a client-server relationship over a communications network, and as such, the servers can be accessed by the clients using computer programs. The servers can be cloud servers, also known as cloud computing servers or cloud hosts, which are mainframe products in the cloud computing service system, and solve the defects of large management difficulty and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or simply "VPS").

[0182] It should be understood that the various forms of flow shown above can be reordered, added to, or have steps deleted. For example, the steps described in the present disclosure can be performed in parallel, in series, or in a different order, as long as the desired results of the technical solutions of the present disclosure can be achieved, and the present disclosure is not limited herein.

[0183] The above detailed description does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for redirecting motion capture, comprising: Obtain the first bone node in the first skeletal animation, and obtain the second bone node and the third bone node in the second skeletal animation. The second skeletal animation is a skeletal animation obtained by rotating the first skeletal animation for consistency redirection. The first bone node and the second bone node are bone nodes of the same part in different skeletal models. Based on the first position information of the first bone node, the second position information of the second bone node, and the third position information of the third bone node, the incremental rotation information of the third bone node is calculated. The original rotation information of the third bone node is optimized based on the incremental rotation information to obtain the redirected third bone node; Based on the first position information of the first bone node, the second position information of the second bone node, and the third position information of the third bone node, the incremental rotation information of the third bone node is calculated, including: Based on the first location information and the third location information, the first difference information is calculated, and based on the second location information and the third location information, the second difference information is calculated. Based on the first difference information and the second difference information, the axis-angle vector and radian of the third bone node are calculated; The incremental rotation information is determined based on the axis-angle vector and the radian.

2. The method according to claim 1, wherein, The axis-angle vector is obtained by multiplying the first difference information and the second difference information; the radian is obtained by multiplying the first difference information and the second difference information.

3. The method according to claim 2, wherein, The incremental rotation information is determined based on the axis-angle vector and the radian, including: According to the Rodriguez rotation algorithm, the axis-angle vector and the radian are processed to convert rotation information to obtain the incremental rotation information.

4. The method according to any one of claims 1-3, wherein, The original rotation information of the third bone node is optimized based on the incremental rotation information to obtain a redirected third bone node, including: The incremental rotation information and the original rotation information are multiplied to obtain the optimized rotation information; The rotation information of the redirected third bone node is the optimized rotation information.

5. The method according to any one of claims 1-3, wherein, The second bone node and the third bone node are bone nodes in the second bone animation that have a parent-child relationship; or, The second bone node and the third bone node are bone nodes in the second bone animation that have a grandparent-grandchild relationship.

6. A motion capture redirection device, comprising: The acquisition unit is used to acquire the first bone node in the first skeletal animation, and to acquire the second bone node and the third bone node in the second skeletal animation, wherein the second skeletal animation is a skeletal animation obtained by rotating the first skeletal animation to achieve consistent rotation, and the first bone node and the second bone node are bone nodes of the same part in different skeletal models. The calculation unit is used to calculate the incremental rotation information of the third bone node based on the first position information of the first bone node, the second position information of the second bone node, and the third position information of the third bone node. An optimization unit is used to optimize the original rotation information of the third bone node based on the incremental rotation information to obtain a redirected third bone node. The computing unit includes: The calculation subunit is used to calculate a first difference information based on the first location information and the third location information, and to calculate a second difference information based on the second location information and the third location information; The sub-units are defined, including the calculation module and the determination module; The calculation module is used to calculate the axis-angle vector and radian of the third bone node based on the first difference information and the second difference information. The determining module is used to determine the incremental rotation information based on the axis angle vector and the radian.

7. The apparatus according to claim 6, wherein, The axis-angle vector is obtained by multiplying the first difference information and the second difference information; the radian is obtained by multiplying the first difference information and the second difference information.

8. The apparatus according to claim 7, wherein, The determining module is used to perform rotation information conversion processing on the axis-angle vector and the radian according to the Rodrigues rotation algorithm to obtain the incremental rotation information.

9. The apparatus according to any one of claims 6-8, wherein, The optimization unit is used to perform a product operation on the incremental rotation information and the original rotation information to obtain the optimized rotation information. The rotation information of the redirected third bone node is the optimized rotation information.

10. The apparatus according to any one of claims 6-8, wherein, The second bone node and the third bone node are bone nodes in the second bone animation that have a parent-child relationship; or, The second bone node and the third bone node are bone nodes in the second bone animation that have a grandparent-grandchild relationship.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

13. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-5.

Images