Virtual character motion control method, device, equipment and storage medium

Abstract

The embodiments of the present application disclose a method, apparatus, device and storage medium for controlling the motion of a virtual character, including: obtaining original motion data, which is the position data of the joints of an original model when performing a target motion; determining the initial position data of the joints of the virtual character based on the original motion data; constructing a target function using the initial position data and the original motion data; generating collision constraints between the skeletal joints and the external joints of the virtual character, and solving the minimum distance of the target function under the length constraint and the collision constraint with the distance between adjacent joints on the virtual character being constant as a length constraint to obtain the target position data of the joints of the virtual character; controlling the joints of the virtual character to move to the position indicated by the target position data. Since the target position data of the joints of the virtual character are solved with the distance between adjacent joints being constant and the collision of the virtual character's external shape being constrained, the virtual character can accurately perform the motion performed by the original model.

Description

Technical Field

[0001] The present application relates to the field of virtual interaction technology, and in particular to a method, apparatus, device and storage medium for controlling the actions of a virtual character. Background Art

[0002] With the emergence of virtual characters such as virtual anchors, virtual idols, and virtual employees, there are more and more applications of controlling virtual character movements based on captured motion data, and the demand for real-time reuse of captured motion data for various virtual characters with different appearances (fat, thin, tall, short, limb length, big head, puffy skirt, etc.) will also become greater and greater.

[0003] Currently, there are two main ways to control the motion of virtual characters. One way is to scale the skeleton of the original model to the skeleton of the virtual character in proportion to obtain a translation, modify the motion data based on the translation, and apply the modified motion data to the virtual character. The other way is to directly apply the rotation data of the original model's skeleton to virtual characters of different proportions using forward kinematics.

[0004] The above-mentioned method of controlling the virtual character by geometric scaling does not take into account the appearance of the virtual character, resulting in loss of action semantics and even model penetration. The method of controlling the virtual character by forward kinematics will result in the following Figure 1 The semantics shown are lost or distorted, Figure 1 The diagram on the left shows the original model performing a salute, and the diagram in the middle shows the action performed by the virtual character after forward kinematics is used to control the virtual character. It can be seen that the salute performed by the virtual character in the diagram in the middle has some defects, such as missing or distorted, compared with the expected salute performed by the virtual character in the diagram on the right. Summary of the Invention

[0005] The embodiments of the present application provide a method, apparatus, device, and storage medium for controlling the motion of a virtual character to solve the problems of motion semantic loss, distortion, and model penetration in the prior art of controlling the motion of a virtual character.

[0006] In a first aspect, an embodiment of the present application provides a method for controlling the motion of a virtual character, comprising:

[0007] Acquire original motion data, where the original motion data is position data of joint points on the skeleton when the original model performs the target motion;

[0008] Determining initial motion data of the virtual character based on the original motion data, wherein the initial motion data is initial position data of joints on the skeleton of the virtual character, and the joints of the original model and the virtual character include skeleton joints and shape joints;

[0009] constructing an objective function using the initial motion data and the original motion data, wherein the objective function is used to calculate the similarity between the initial motion data and the original motion data;

[0010] generating collision constraints between the skeleton joint points and the shape joint points on the virtual character, and generating length constraints between adjacent joint points with the distance between the skeleton joint points on the virtual character unchanged;

[0011] Solving the minimum distance value of the objective function under the length constraint and the collision constraint to obtain target action data of the virtual character, wherein the target action data is target position data of the joint points of the virtual character;

[0012] Control each joint point of the virtual character to move to the position indicated by the target position data, so as to drive the virtual character to perform the target action.

[0013] In a second aspect, an embodiment of the present application provides a motion control device for a virtual character, comprising:

[0014] An original motion data acquisition module is used to acquire original motion data, wherein the original motion data is position data of joint points on the skeleton when the original model performs the target motion;

[0015] An initial motion data determination module is used to determine initial motion data of the virtual character based on the original motion data, wherein the initial motion data is initial position data of joints on the skeleton of the virtual character, and the joints of the original model and the virtual character include skeletal joints and shape joints;

[0016] an objective function generating module, configured to construct an objective function using the initial motion data and the original motion data, wherein the objective function is configured to calculate a similarity between the initial motion data and the original motion data;

[0017] a constraint construction module, configured to generate collision constraints between the skeletal joint points and the shape joint points on the virtual character, and to generate length constraints between adjacent joint points by keeping the distances between the skeletal joint points on the virtual character unchanged;

[0018] an objective function solving module, configured to solve the minimum distance value of the objective function under the length constraint and the collision constraint, and obtain target action data of the virtual character, wherein the target action data is target position data of the joint points of the virtual character;

[0019] The virtual character control module is used to control each joint point of the virtual character to move to the position indicated by the target position data, so as to drive the virtual character to perform the target action.

[0020] In a third aspect, an embodiment of the present application provides a virtual character motion control device, the virtual character motion control device comprising:

[0021] one or more processors;

[0022] a storage device for storing one or more computer programs,

[0023] When the one or more computer programs are executed by the one or more processors, the one or more processors implement the action control method of the virtual character described in the first aspect of the present application.

[0024] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for controlling the action of a virtual character described in the first aspect of the present application.

[0025] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when instructions in the computer program product are executed by a processor, implements the method for controlling the action of the virtual character described in the first aspect.

[0026] In an embodiment of the present application, after obtaining the original action data of the original model when performing the target action, the initial action data of the virtual character is determined based on the original action data, and an objective function is constructed using the initial action data and the original action data. The objective function is used to calculate the similarity between the initial action data and the original action data, and generate collision constraints between the skeletal joints and the shape joints on the virtual character, and generate length constraints between adjacent joints with the distance between the skeletal joints on the virtual character unchanged. The minimum distance value of the objective function under the length constraint and the collision constraint is further solved to obtain the target action data of the virtual character. The target action data is the target position data of the joints of the virtual character. Ultimately, the joints of the virtual character are controlled to move to the positions indicated by the target position data to drive the virtual character to perform the target action. On the one hand, the smaller the distance between the initial action data and the original action data, the closer the action of the virtual character is to the action of the original model, ensuring that the virtual character can accurately perform the target action performed by the original model. On the other hand, the length constraint and the collision constraint can ensure the integrity of the action semantics and avoid model penetration. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a schematic diagram of action semantic distortion in virtual character action control in the prior art;

[0028] Figure 2 This is a flow chart of a method for controlling the motion of a virtual character provided in Example 1 of the present application;

[0029] Figure 3A This is a flow chart of a method for controlling the motion of a virtual character provided in Example 2 of the present application;

[0030] Figure 3B This is a schematic diagram of adding shape joint points in an example of this application;

[0031] Figure 3C This is a schematic diagram of the joint points on the skeleton in the embodiment of the present application;

[0032] Figure 3D This is a schematic diagram of the graphed adjacency matrix of the joint points in the embodiment of the present application;

[0033] Figure 3E is a schematic diagram of the action adjacency relationship in an embodiment of the present application;

[0034] Figure 3F is a schematic diagram of collision constraints in an embodiment of the present application;

[0035] Figure 4 This is a structural block diagram of a virtual character motion control device provided in Example 3 of the present application;

[0036] Figure 5 This is a structural block diagram of a virtual character motion control device provided in Example 4 of the present application. DETAILED DESCRIPTION

[0037] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the present application and are not intended to limit the present application. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions of the present application, not all of the structures.

[0038] Example 1

[0039] Figure 2 This is a flow chart of a method for controlling the motion of a virtual character provided in the first embodiment of the present application. The embodiment of the present application is applicable to the case where the motion of the original model is captured to control the simulated motion of the virtual character. The method can be executed by the motion control device of the virtual character implemented in the present application. The motion control device of the virtual character can be implemented by hardware or software and integrated into the motion control of the virtual character provided in the embodiment of the present application. Specifically, Figure 2 As shown, the action control method of the virtual character in the embodiment of the present application may include the following steps:

[0040] S201. Obtain original action data, where the original action data is position data of joint points on the skeleton when the original model performs a target action.

[0041] In an embodiment of the present application, the original model can be a model that performs the target action, and the virtual character can be a character that imitates the target action performed by the original model. In one example, the original model can be a human model, and the virtual character can be a digital human. For example, the original model can be a real human body in real life, and the virtual character can be a virtual anchor, virtual host, virtual puppet, robot, etc. Of course, the original model can also be an animal, and the virtual character can be a virtual animal. It should be noted that the original model and the virtual character have the same skeletal structure. For the sake of convenience, the embodiment of the present application takes the original model as a real-life person and the virtual character as a virtual person as an example to illustrate the motion control method of the virtual character.

[0042] In an application scenario, when controlling a virtual puppet in the network by capturing the host's movements, the host is the original model, the virtual puppet is the virtual character, and the virtual puppet needs to make the same movements as the host. When obtaining the original movement data, at least one image of the host can be captured by a camera, and joints can be recognized through at least one image to obtain the position data of each joint on the skeleton when the host makes the target movement as the original movement data. Optionally, at least one image can be input into a pre-trained human joint recognition network to obtain the position data of each joint of the host as the original movement data.

[0043] S202. Determine initial motion data of the virtual character based on the original motion data. The initial motion data is initial position data of joints on the skeleton of the virtual character. The joints of the original model and the virtual character include skeleton joints and shape joints.

[0044] The action data can be the rotation data of the bones on the model. The original model performing the target action can be regarded as the skeleton of the original model in motion. The skeleton is composed of multiple bones connected with a parent-child connection relationship. The rotational movement of the child bone relative to the parent bone can be recorded by a matrix. In the human skeleton, the root bone can be the pelvis, and the child bones connected to the pelvis and the child bones of the child bones have a transmission relationship. For example, from the pelvis upward through the spine, upper arm, forearm, and then to the hand, the rotation matrices of all the bones passing from the pelvis to the hand are multiplied in sequence to obtain the action data of the hand. Since the joint point is the node connecting the bones, after obtaining the rotation data of the bone, the position data of each joint point on the skeleton can be determined, or after obtaining the position data of each joint point, the rotation data of the skeleton formed by the joint point can also be obtained.

[0045] Specifically in the embodiment of the present application, the original action data represents the target action performed by the original model. The original action data can be assigned to the skeleton of the virtual character, that is, the various bones of the virtual character are initialized, so that the various bones of the virtual character have the same rotation data as the corresponding bones in the original model, that is, the position data of each joint point on the virtual character is initialized, so that the optimal position of the joint point can be solved within the range near the position data, reducing the difficulty of solving and improving the efficiency of solving to achieve real-time control of the virtual character's actions.

[0046] In another optional embodiment, the joint points may include skeletal joint points and external shape joint points. The skeletal joint points are nodes connecting the root bones, and the external shape joint points may be virtual points set on the original model and the virtual character's external shape to avoid penetration. In one example, a frame of original motion data may be applied to virtual characters with the same skeleton but different external shapes. For example, virtual dolls may be fat or thin, or some virtual dolls may have larger heads, or may wear skirts, resulting in differences in external shapes. In this case, one or more virtual points may be set as external shape joint points on the outer contours of the original model and virtual character where penetration needs to be avoided.

[0047] S203: Using the distance between the initial motion data and the original motion data as a target function.

[0048] The original action data represents the action performed by the original model, and the initial action data represents the initialization action of the virtual character. The initial action data and the original action data can be substituted into the objective function, which is used to calculate the similarity between the initial action data and the original action data, that is, to calculate the similarity between the action performed by the original model and the action of the virtual character.

[0049] In one example, the similarity can be expressed by calculating the distance between the initial action data and the original action data. The smaller the similarity, the closer the action of the virtual character is to the action of the original model. Specifically, in the objective function, the original action data is a fixed value, the initial action data is a variable, and the objective function value is a dependent variable. The objective function value can be minimized by continuously iteratively updating the initial action data. In one example, the objective function can be a function that calculates the distance between two values, such as the L2 norm (Euclidean) distance function, the Chebyshev distance function, etc. The embodiment of the present invention does not limit the objective function for calculating the similarity.

[0050] S204: generating collision constraints between the skeleton joint points and the shape joint points on the virtual character, and generating length constraints between adjacent joint points with the distance between the skeleton joint points on the virtual character unchanged.

[0051] The length of bones in the same position on the skeleton may be inconsistent between the original model and the virtual character, and the length of bones in different virtual characters is also different. For example, virtual character A has long arms and virtual character B has short arms. For the same virtual character, the length of each bone is fixed. The length of each bone in the virtual character can be constrained to remain unchanged. Specifically, the length of each bone of the virtual character can be calculated, and then the distance between the two adjacent joint points that constitute the skeleton in the initial motion data can be calculated. The difference between this distance and the original distance of the bones is used as the length constraint, that is, while changing the position of the joint points, the length of the bone formed by the two joint points must be guaranteed to remain unchanged.

[0052] At the same time, the pre-set joint points and collision points on the virtual character that need to be subject to collision constraints can be determined, and the distance between the joint points subject to collision constraints and the collision points can be calculated as the collision depth. Constraining the collision depth to be less than or equal to 0 can ensure that the joint points subject to collision constraints will not collide with the collision points, and the virtual character's movements will not be penetrated.

[0053] S205 , solving the minimum distance value of the objective function under the length constraint and the collision constraint to obtain target action data of the virtual character, where the target action data is target position data of the joint points of the virtual character.

[0054] Specifically, the process of solving the minimum distance value of the objective function under the length constraint and collision constraint is: while ensuring that the bone length remains unchanged and the joint points subject to collision constraints will not collide with the collision point, the initial action data is continuously changed to minimize the function value of the objective function, that is, the positions of the various joint points of the virtual character are continuously changed to minimize the objective function value, so that the position data of the various joint points of the virtual character can be obtained. In practical applications, the optimal solution of the objective function can be solved by sequential quadratic programming (SQP) or augmented Lagrange method (ALM), so that the target position data of the joint points of the virtual character can be obtained. The solution method of sequential quadratic programming (SQP) or augmented Lagrange method (ALM) can refer to the existing technology and will not be described in detail here.

[0055] S206: Control each joint point of the virtual character to move to the position indicated by the target position data, so as to drive the virtual character to perform the target action.

[0056] After obtaining the target position data of each joint point of the virtual character, the various joint points on the virtual character's skeleton can be controlled to move to the position indicated by the target position data. After each joint point is located at the position indicated by the target position data, the action performed by the various skeletons composed of the joint points is the target action performed by the original model.

[0057] In an embodiment of the present application, after obtaining the original action data of the original model when performing the target action, the initial action data of the virtual character is determined based on the original action data, and an objective function is constructed using the initial action data and the original action data. The objective function is used to calculate the similarity between the initial action data and the original action data, and generate collision constraints between the skeletal joints and the shape joints on the virtual character, and generate length constraints between adjacent joints with the distance between the skeletal joints on the virtual character unchanged. The minimum distance value of the objective function under the length constraint and the collision constraint is further solved to obtain the target action data of the virtual character. The target action data is the target position data of the joints of the virtual character. Ultimately, the joints of the virtual character are controlled to move to the positions indicated by the target position data to drive the virtual character to perform the target action. On the one hand, the smaller the distance between the initial action data and the original action data, the closer the action of the virtual character is to the action of the original model, ensuring that the virtual character can accurately perform the target action performed by the original model. On the other hand, the length constraint and the collision constraint can ensure the integrity of the action semantics and avoid model penetration.

[0058] Example 2

[0059] Figure 3A This is a flow chart of a method for controlling the motion of a virtual character provided in the second embodiment of the present application. The embodiment of the present application is optimized based on the above-mentioned first embodiment. Specifically, Figure 3A As shown, the action control method of the virtual character in the embodiment of the present application may include the following steps:

[0060] S301. Obtain original action data, where the original action data is position data of joint points on the skeleton when the original model performs a target action.

[0061] Before obtaining the original action data, the embodiment of the present application can first set the joint points of the original model and the virtual character. The set joint points include skeletal joint points and appearance joint points. Among them, the skeletal joint points are two nodes that constitute the skeleton. The appearance joint points are virtual joint points set according to the appearance of the virtual character to avoid penetration of the virtual character during the action. The appearance joint points can be set according to the different appearances of the virtual character.

[0062] like Figure 3B In the virtual doll shown, the head of the virtual doll is relatively large and the skirt it wears is relatively large. In order to avoid the hand from penetrating into the head or the skirt during hand movements, external joint points can be added to the head and the skirt. In one example, external joint points are added to the head as shown in FIG. Figure 3B The black blocks P1, P2, and P3 in the middle of the head, add shape joints to the skirt as follows Figure 3BOf course, in actual applications, different appearance joint points can be pre-set according to different virtual characters, and this embodiment of the present application does not limit this.

[0063] In an optional embodiment, when obtaining the original motion data, an image of the original model can be collected, and joint point recognition can be performed on the image to obtain the position data of the joint points of each bone on the original model as the original motion data. For example, when the original model is the anchor, at least one frame of the anchor's image can be collected by a camera and input into a pre-trained human joint point recognition network to obtain the position data of the anchor's joint points as the original motion data.

[0064] like Figure 3C The figure is a schematic diagram of a human skeleton. The human skeleton is composed of multiple bones, and the two ends of each bone are joints. In the embodiment of the present application, the position data of 17 joints (point 0-point 16) are used to represent the movement of the model. In an example of the present application, the original action data can be the rotation data of the bones in the skeleton. Specifically, the pelvis can be used as the root bone, and the other bones are the child bones or secondary bones of the pelvis. The rotation data is the position data of each bone relative to its parent bone. For example, Figure 3C In the figure, assuming that the rotation data of bone P07 relative to joint point 0 is D07, and the rotation data of bone P78 relative to bone P07 is D78, then the rotation data of bone P78 relative to joint point 0 is D07×D78. Similarly, the rotation data of each bone relative to joint point 0 can be obtained. Since the joint point is the node connecting bones, the position data of each joint point on the bone can be determined after obtaining the rotation data of the bone, or after obtaining the position data of each joint point, the rotation data of the skeleton formed by the joint point can also be obtained.

[0065] S302: Calculate the rotation data of the skeleton between the two adjacent joint points according to the position data of the two adjacent joint points of the original model in the original motion data.

[0066] In an optional embodiment of the present application, the position data may be the three-dimensional coordinates of the joints, and the human joints are identified by the three-dimensional coordinates of each joint relative to the human body coordinate system. For example, the origin of the coordinate system is as follows: Figure 3C Regarding the joint point 0, the three-dimensional coordinates of each joint point from joint point 1 to joint point 16 relative to joint point 0 can be obtained during human body joint point identification, so that the rotation data of the skeleton relative to the auxiliary skeleton can be calculated based on the coordinates of the two joint points that constitute the skeleton.

[0067] like Figure 3CAs shown, with the joint point 0 of the pelvis as the coordinate origin, after obtaining the three-dimensional coordinates of the joint points 0 and 7, the rotation data of the bone P07 can be calculated by the three-dimensional coordinates of the joint points 7 and 0, and then the rotation data of the bone P78 relative to its parent bone P07 can be calculated according to the three-dimensional coordinates of the joint points 8 and 7. The rotation data of the bones P07 and P78 can be multiplied to obtain the rotation data of the bone P78 relative to the joint point 0, and so on to obtain the rotation data of each bone relative to the joint point 0.

[0068] S303: Transplant the rotation data of each bone in the original model to the corresponding bone of the virtual character as the rotation data of the skeleton of the virtual character, and obtain the initial action data of the virtual character.

[0069] The original action data represents the target action performed by the original model. The original action data can be assigned to the skeleton of the virtual character, that is, the various bones of the virtual character are initialized so that the various bones of the virtual character have the same rotation data as the corresponding bones in the original model, that is, the position data of each joint point on the virtual character is initialized, so that the optimal position of the joint point can be solved within the range near the position data, reducing the difficulty of solving and improving the solving efficiency to achieve real-time control of the virtual character's actions.

[0070] Specifically, for virtual characters, the pelvic joint point 0 can also be used as the origin, and the rotation data of each bone can be set in sequence, so that the rotation data of each bone of the virtual character is the same as the rotation data of the corresponding bone in the original model, so that each joint point of the virtual character is initialized to a position, so that the optimal position of the joint point can be solved within a range near the initial position, reducing the difficulty of solving and improving the solving efficiency to achieve real-time control of the virtual character's movements.

[0071] S304: Calculate original vectors of joint points of the original model using the original motion data, and calculate initial vectors of joint points of the virtual character using the initial motion data.

[0072] For example, the three-dimensional coordinates of each joint point can be calculated through the rotation data of the skeleton, and the three-dimensional coordinates of each joint point can be connected into a vector, which is the vector of all the joint points of the model, such as Figure 3C As shown, there are a total of 17 joints, each of which has coordinate values ​​in three dimensions: x, y, and z. All the joints of the model can be written as a vector x∈R^51.

[0073] S305: Generate an action semantic matrix for the target action based on the skeleton structure of the original model and the preset action adjacency relationship.

[0074] In an optional embodiment of the present application, the joint point adjacency matrix of the original model can be obtained, and each element value in the row where each joint point is located in the joint point adjacency matrix represents the joint adjacency relationship between the joint point and other joint points. For each target joint point on the original model, the action semantics adjacent joint point of the target joint point is determined according to the preset action semantics adjacency relationship, and the element value of the action semantics adjacent joint point in the row where the target joint point is located in the joint point adjacency matrix is ​​updated to obtain the action semantic matrix of the target action.

[0075] The joint adjacency matrix represents the adjacency relationship between joints in the model, such as Figure 3C As shown, joint point 8 is adjacent to joint point 11, joint point 9, joint point 14, and joint point 7 respectively. Joint point 8 is not adjacent to any other joint points. If two joint points are adjacent, the corresponding element value in the matrix is ​​recorded as -1, otherwise it is 0. Figure 3D for Figure 3C The adjacency matrix of each joint point in is tabulated for easy identification. Figure 3D The first row and the first column are the joint point numbers. Take joint point 8 as an example. In the row where joint point 8 is located, since joint point 8 is adjacent to joint point 11, joint point 9, joint point 14, and joint point 7, the corresponding element value is -1. Joint point 8 has 4 adjacent joint points, so the element value in the row and column where joint point 8 is located is 4, and the element values ​​of the other non-adjacent joint points are 0. Figure 3D It can be seen that the diagonal line in the table is the number of joint points adjacent to each joint point.

[0076] The preset action semantic adjacency relationship represents the adjacency relationship between the joints in an action, that is, the semantics of the action is expressed by defining the adjacency relationship of the joints, such as Figure 3E As shown, for hand movements as an example, the adjacency relationship between the hand joint 16 and other joints can be predefined. In one example, the hand joint 16 can be defined as being adjacent to the head joint 10, the shoulder joint 14, the spine joint 7, the thigh joint 1, and the foot joint 3, a total of five joints. Then Figure 3D The element values ​​in the row where the middle node 16 is located are as follows:

[0077] Table 1

[0078] 0 -1 0 -1 0 0 0 -1 0 0 -1 0 0 0 -1 0 5

[0079] Since the element value is -1 when two joints are adjacent and 0 when they are not, the action semantic connection relationship of joint 16 can be seen from the row where the joint 16 is located. For the joints in the original model and the virtual character, the action semantic adjacency relationship of each joint can be pre-defined, such as Figure 3CIt is defined that the hand joint 16 is adjacent to the head joint 10, the shoulder joint 14, the spine joint 7, the thigh joint 1 and the foot joint 3, a total of five joints. More attention is paid to the hand movements, so the hand joint 16 is defined to be semantically adjacent to the other relatively fixed joints. If more attention is paid to the head movements, the head joint 10 can be defined to be semantically adjacent to the joint 8, joint 11, joint 14 and joint 7 respectively. The embodiment of the present application does not limit the semantic adjacency relationship of the joints.

[0080] Table 1 above is the representation of the semantic connection relationship of the action of joint point 16 on the matrix. Update the element value of Table 1 to Figure 3D In the matrix, and by analogy updating the element values ​​of the rows of other joints, the action semantic matrix of the target action can be obtained. The action semantic matrix represents the adjacency relationship of each joint in the action when the original model performs the target action, that is, Figure 3E The action adjacency relationship between the two joints connected by the dotted line. The entire action semantic matrix represents the action semantics of the action performed by the original model.

[0081] Furthermore, after setting the element value of the action semantics adjacent joint point in the row where the target joint point is located in the joint point adjacency matrix to a preset value, the distance between each action semantics adjacent joint point and the target joint point can also be calculated, and the weight of each action semantics adjacent joint point is calculated using the distance. The product of the weight and the preset value is calculated to obtain the weight, and the element value of the action semantics adjacent joint point in the row where the target joint point is located in the joint point adjacency matrix is ​​modified to be equal to the weight.

[0082] Specifically, if Figure 3D As shown, since joint 16 is adjacent to five joints in terms of action semantics, namely, joint 10 of the head, joint 14 of the shoulder, joint 7 of the spine, joint 1 of the thigh, and joint 3 of the foot, Figure 3D The element values ​​of joint point 10, shoulder joint point 14, spine joint point 7, thigh joint point 1, and foot joint point 3 in the row where the middle joint point 16 is located are changed to the values ​​in Table 1 above, that is, after being modified to the preset value -1, the distances between joint point 10, joint point 14, joint point 7, joint point 1, and joint point 3 in the original model and joint point 16 can be calculated. For example, the distance between the two joint points is calculated by the three-dimensional coordinates of the two joint points, and then the reciprocal of each distance is calculated, and the sum of the reciprocals of all reciprocals is calculated. For each joint point among joint point 10, joint point 14, joint point 7, joint point 1, and joint point 3, the ratio of the reciprocal of the distance of each joint point to the sum of the reciprocals is calculated as the weight of the joint point, as shown in the following formula:

[0083]

[0084] In the above formula, Distance_ij is the distance from joint point j to joint point i, which is adjacent to joint point i in action semantics, and w j The weight of joint point j, from which we can get that the greater the distance between joint point j and joint point i, the smaller the weight, such as Figure 3E As shown, the distance between the joint point 3 of the foot and the joint point 16 of the hand is the farthest. When the hand performs the target action, the semantics of the joint point 16 has little relationship with the foot, that is, the hand action has little relationship with the joint of the foot. On the contrary, the relationship with the joint point 14 of the shoulder is the greatest, so that the distance between the joint points with adjacent action semantics can be dynamically calculated during different actions to determine the weight, and the action semantics can be better explained through the weight.

[0085] After obtaining the weight of the joint point, the weight obtained by multiplying the weight by the element value corresponding to the joint point is used as the new element value. Taking the above Table 1 as an example, after calculation, assuming that the weights of joint point 16 and joint points 1, 3, 7, 10, and 14 are 0.1, 0.2, 0.2, 0.2, and 0.3 respectively, the above Table 1 is updated as follows:

[0086] 0 -0.1 0 -0.2 0 0 0 -0.2 0 0 -0.2 0 0 0 -0.3 0 1

[0087] After updating the weights of all joints, a weighted action semantic matrix is ​​obtained. In different actions, the distances between joints are different, and the weights are also different. The farther the distance, the smaller the weight, and vice versa. By dynamically allocating weights, the action semantic matrix can better explain the action semantics.

[0088] S306 , respectively calculating the product of the action semantic matrix with the original vector and the initial vector to obtain a first product and a second product.

[0089] Specifically, let the action semantic matrix of the target action in S305 be L, the vector formed by the positions of all joint points of the original model in S304 be srcPos3d, and the vector of all joint points of the virtual character be tarPos3d, then calculate the first product L×srcPos3d, and calculate the second product L×tarPos3d. The first product represents the measurement value of the adjacency relationship of each joint point in the action semantics in the original model, and the second product represents the measurement value of the adjacency relationship of each joint point in the action semantics in the virtual character.

[0090] S307: Calculate the distance between the first product and the second product as the objective function.

[0091] In an alternative embodiment, the objective function is as follows:

[0092] min 0.5×‖L×tarPos3d-L×srcPos3d‖ 2

[0093] Among them, L is the action semantic matrix, tarPos3d is the initial vector of the virtual character's joint point, srcPos3d is the original vector of the original model's joint point, ‖.‖ 2 is the two-norm distance. The smaller the objective function value is, the closer the action semantics of the virtual character is to that of the original model.

[0094] S308: Generate collision constraints between the skeleton joint points and the shape joint points on the virtual character, and generate length constraints between adjacent joint points with the distance between the skeleton joint points on the virtual character unchanged.

[0095] In an optional embodiment, for the length constraint, the distance between the two joint points of each bone on the virtual character can be calculated as the original length of the bone, and the distance between the vectors of the two joint points of each bone can be calculated to construct the length constraint as follows:

[0096] ‖tarPos3d[i]-tarPos3d[j]‖-resetLength=0

[0097] Among them, resetLength is the original length of the bone between the virtual character's joint point i and joint point j, tarPos3d[i] and tarPos3d[j] are the vectors of joint point i and joint point j respectively. This length constraint means that: in the process of solving the minimum value of the objective function, when the positions of joint point i and joint point j are continuously changed, the distance between the changed joint point i and joint point j is equal to the original length resetLength.

[0098] For collision constraints, the shape joints include preset collision points, and the skeleton joints include the joints subject to collision constraints. The collision constraints between the skeleton joints and the shape joints on the virtual character are generated as follows:

[0099] (tarPos3d[i]-collPos).dot(colldepth)≤0

[0100] Among them, collPos represents the vector of the collision point, tarPos3d[i] represents the vector of the joint point i subject to the collision constraint, tarPos3d[i]-collPos represents the vector from the joint point i subject to the collision constraint to the collision point, and the dot product of .dot(colldepth) represents the projection of the vector from the joint point i subject to the collision constraint to the collision point in the direction perpendicular to the outer contour of the virtual character. The collision constraint means that: in the process of solving the minimum value of the objective function, when the joint point i subject to the collision constraint is continuously changed, the distance of the projection of the vector of the collision point of the changed joint point i subject to the collision constraint in the direction perpendicular to the outer contour of the virtual character is less than or equal to 0.

[0101] The principle of collision constraint is as follows Figure 3F As shown, in Figure 3F In order to prevent the human body from putting hands on hips and penetrating into the body, an external joint point P2 is set on the body as a collision point, and the joint point P1 of the hand at the end of the lower arm is set as the joint point subject to collision constraint. The distance of the projection of the vector from the joint point P1 to the external joint point P2 in the direction perpendicular to the outer surface of the body is the distance from P1 to P3, that is, the collision depth. When the collision depth is less than or equal to 0, it means that the joint point P1 will not collide with the external joint point P2, that is, the hand will not penetrate into the body, thereby ensuring that the penetrating phenomenon will not occur.

[0102] S309: Using a sequential quadratic programming method or a Lagrangian method to solve the minimum distance value of the objective function under the length constraint and the collision constraint, to obtain the target action data of the virtual character.

[0103] Solving the target action data is to solve the following objective function:

[0104] min 0.5×‖L×tarPos3d-L×srcPos3d‖ 2

[0105]

[0106] That is, by constantly changing the position of the virtual character's joint points, the vector tarPos3d of the virtual character's joint points changes until ‖L×tarPos3d-L×srcPos3d‖ 2 When the minimum, the position of the joint point is the optimal position. In the process of changing the position of the joint point, it is necessary to ensure that the distance between the two joint points i and j that form the skeleton remains unchanged, and the joint point i subject to the collision constraint will not collide with the collision point.

[0107] In practical applications, the optimal solution of the objective function can be solved by sequential quadratic programming (SQP) or augmented Lagrange method (ALM), that is, the target position data of the virtual character's joints can be obtained. The solution methods of sequential quadratic programming (SQP) or augmented Lagrange method (ALM) can refer to the existing technology.

[0108] In an optional embodiment, for a target action, the action semantic matrix L is fixed and the collision constraint is that the collision depth is equal to 0, then the objective function can be simplified to a function of the equality constraint:

[0109] min 0.5×‖tarPos3d-srcPos3d‖ 2

[0110]

[0111] The solution process is as follows:

[0112] Assume C(tarPos3d)=‖tarPos3d[i]-tarPos3d[j]‖-resetLength

[0113] C(tarPos3d) is a quadratic nonlinear constraint. It can be obtained by Taylor expansion of C(tarPos3d) and performing a first-order linear transformation:

[0114] C(tarPos3d)=J×tarPos3d-b,

[0115] J is the Jacobian matrix of C(tarPos3d), b is the constant after Taylor expansion, and the specific Taylor expansion can be referred to the existing technology and will not be described in detail here.

[0116] Since the action semantic matrix remains unchanged after the action is determined, the Lagrangian function is constructed:

[0117] min 0.5×‖tarPos3d-srcPos3d‖+λ×(J×tarPos3d-b)

[0118] Assuming x = tarPos3d - srcPos3d, the Lagrangian function is transformed into:

[0119] L(x,λ)=0.5×‖x‖+transpose(λ)×(J×xb) (1)

[0120] transpose(.) is to get the transposed matrix.

[0121] Let the derivatives of formula (1) with respect to x and λ be equal to 0, then we can get the following equations:

[0122] x+transpose(J)×λ=0 (2)

[0123] J×x=b (3)

[0124] Transforming formula (2) yields:

[0125] x=-transpose(J)×λ (4)

[0126] Substituting formula (4) into formula (3) yields:

[0127] J×(-transpose(J)×λ)=b (5)

[0128] Solving for λ:

[0129]

[0130] Substitute formula (6) into formula (2) to solve for x:

[0131]

[0132] Use the Gauss-Seidel iteration method to iterate x until convergence to obtain the final x. Since x = tarPos3d - srcPos3d, srcPos3d is fixed, and tarPos3d, which is the optimal position of the virtual character's joint point, is obtained. The specific iteration process can refer to the iteration process of the Gauss-Seidel iteration method in the prior art and will not be described in detail here.

[0133] S310: Control each joint point of the virtual character to move to the position indicated by the target position data, so as to drive the virtual character to perform the target action.

[0134] After obtaining the target position data of each joint point of the virtual character, the various joint points on the virtual character's skeleton can be controlled to move to the position indicated by the target position data. After each joint point is located at the position indicated by the target position data, the action performed by the various skeletons composed of the joint points is the target action performed by the original model.

[0135] In the embodiment of the present application, after obtaining the original action data of the original model when executing the target action, the initial action data of the virtual character is determined according to the original action data, the original vectors of the joints of the original model are calculated using the original action data, and the initial vectors of the joints of the virtual character are calculated using the initial action data, an action semantic matrix of the target action is generated based on the skeletal structure of the original model and the preset action adjacency relationship, the product of the action semantic matrix with the original vector and the initial vector is calculated respectively to obtain a first product and a second product, the distance between the first product and the second product is calculated as the objective function, and the collision constraints between the skeletal joints and the shape joints on the virtual character are generated, and the distance between the skeletal joints on the virtual character is calculated based on the distance between the skeletal joints on the virtual character. The distance between them remains unchanged, and the length constraint between adjacent joint points is generated. The minimum distance value of the objective function under the length constraint and collision constraint is further solved to obtain the target action data of the virtual character. The target action data is the target position data of the joint points of the virtual character. Finally, the joint points of the virtual character are controlled to move to the position indicated by the target position data to drive the virtual character to perform the target action. On the one hand, the smaller the distance between the initial action data and the original action data, the closer the action of the virtual character is to the action of the original model, which ensures that the virtual character can accurately perform the target action made by the original model. On the other hand, the integrity of the action semantics can be guaranteed and model penetration can be avoided through the length constraint and collision constraint.

[0136] Example 3

[0137] Figure 4 This is a structural block diagram of a virtual character motion control device provided in Example 3 of the present application. Figure 4 As shown, the motion control device for a virtual character in an embodiment of the present application may specifically include the following modules:

[0138] The original motion data acquisition module 401 is used to acquire original motion data, wherein the original motion data is the position data of the joint points on the skeleton when the original model performs the target motion;

[0139] An initial motion data determining module 402 is configured to determine initial motion data of the virtual character based on the original motion data, wherein the initial motion data is initial position data of joints on the skeleton of the virtual character, and the joints of the original model and the virtual character include skeletal joints and shape joints;

[0140] An objective function generating module 403 is configured to construct an objective function using the initial motion data and the original motion data, wherein the objective function is configured to calculate the similarity between the initial motion data and the original motion data;

[0141] a constraint construction module 404 for generating collision constraints between the skeleton joint points and the shape joint points on the virtual character, and generating length constraints between adjacent joint points by keeping the distance between the skeleton joint points on the virtual character unchanged;

[0142] An objective function solving module 405 is configured to solve the objective function for a minimum distance value under the length constraint and the collision constraint, and obtain target motion data of the virtual character, wherein the target motion data is target position data of the joint points of the virtual character;

[0143] The virtual character control module 406 is configured to control each joint of the virtual character to move to a position indicated by the target position data, so as to drive the virtual character to perform the target action.

[0144] The virtual character motion control device provided in the embodiment of the present application can execute the virtual character motion control method provided in the first and second embodiments of the present application, and has the corresponding functional modules and beneficial effects of the execution method.

[0145] Example 4

[0146] Reference Figure 5 , shows a schematic diagram of the structure of a virtual character motion control device in an example of the present application. Figure 5As shown, the virtual character motion control device may specifically include: a processor 501, a storage device 502, a display screen 503 with a touch function, an input device 504, an output device 505, and a communication device 506. The number of processors 501 in the virtual character motion control device may be one or more. Figure 5 In the figure, a processor 501 is used as an example. The processor 501, storage device 502, display screen 503, input device 504, output device 505 and communication device 506 of the virtual character motion control device can be connected through a bus or other means. Figure 5 Taking the bus connection as an example, the virtual character motion control device is used to execute the virtual character motion control method provided in the embodiment of the present application.

[0147] Example 5

[0148] An embodiment of the present application provides a computer-readable storage medium, and when a computer program in the storage medium is executed by a processor, the method for controlling the action of a virtual character as described in the above method embodiment is implemented.

[0149] Example 6

[0150] An embodiment of the present application provides a computer program product. When instructions in the computer program product are executed by a processor, the method for controlling the motion of a virtual character described in the above method embodiment is implemented.

[0151] For the purposes of this application, a computer-readable storage medium can be any device that can contain, store, communicate, propagate, or transmit a program for use with an instruction execution system, device, or apparatus, or in conjunction with such instruction execution systems, devices, or apparatuses. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion having one or more wirings (electronic device), a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium can even be paper or other suitable medium on which the program is printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in other suitable ways as necessary, and then stored in a computer memory. A computer program product can be a product comprising a computer-readable storage medium, such that when the instructions in the computer-readable storage medium are executed by a processor, the method for controlling the action of the virtual character described in the above method embodiment is implemented.

[0152] It should be noted that, for the embodiments of the apparatus, device, storage medium, and computer program product, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

Claims

1. A method for controlling the motion of a virtual character, characterized in that: include: Acquire original motion data, where the original motion data is position data of joint points on the skeleton when the original model performs the target motion; Determining initial motion data of the virtual character based on the original motion data, wherein the initial motion data is initial position data of joints on the skeleton of the virtual character, wherein the joints of the original model and the virtual character include skeleton joints and outline joints, and the outline joints are virtual points set on the outlines of the original model and the virtual character; constructing an objective function using the initial motion data and the original motion data, wherein the objective function is used to calculate the distance between the initial motion data and the original motion data as similarity; generating collision constraints between the skeleton joint points and the shape joint points on the virtual character, and generating length constraints between adjacent joint points with the distance between the skeleton joint points on the virtual character unchanged; Solving the minimum distance value of the objective function under the length constraint and the collision constraint to obtain target action data of the virtual character, wherein the target action data is target position data of the joint points of the virtual character; Control each joint point of the virtual character to move to the position indicated by the target position data, so as to drive the virtual character to perform the target action.

2. The method for controlling the motion of a virtual character according to claim 1, wherein: Before getting the raw action data, it also includes: Joint points are set, and the joint points include skeleton joint points and shape joint points.

3. The method for controlling the motion of a virtual character according to claim 1, wherein: The obtaining of original motion data includes: Collect images of the original model; Joint point recognition is performed on the image to obtain position data of joint points of each skeleton on the original model as original action data.

4. The method for controlling the motion of a virtual character according to claim 1, wherein: The determining of the initial action data of the virtual character according to the original action data includes: Calculating the rotation data of the skeleton between two adjacent joint points according to the position data of two adjacent joint points of the original model in the original motion data; The rotation data of each bone in the original model is transplanted to the corresponding bone of the virtual character as the rotation data of the skeleton of the virtual character, thereby obtaining the initial action data of the virtual character.

5. The method for controlling the motion of a virtual character according to any one of claims 1 to 4, wherein: The constructing of the objective function by using the initial motion data and the original motion data includes: Calculating original vectors of joint points of the original model using the original motion data, and calculating initial vectors of joint points of the virtual character using the initial motion data; Generate an action semantic matrix of the target action based on the skeleton structure of the original model and the preset action adjacency relationship; Calculating the product of the action semantic matrix, the original vector, and the initial vector respectively to obtain a first product and a second product; A distance between the first product and the second product is calculated as an objective function.

6. The method for controlling the motion of a virtual character according to claim 5, wherein: The step of generating the action semantic matrix of the target action based on the skeleton structure of the original model and the preset action adjacency relationship includes: Obtaining a joint point adjacency matrix of the original model, wherein each element value in a row where each joint point is located in the joint point adjacency matrix represents a joint adjacency relationship between the joint point and other joint points; For each target joint point on the original model, determining the action semantics adjacent joint point of the target joint point according to a preset action semantics adjacency relationship; The element value of the action semantics adjacent joint point in the row where the target joint point is located in the joint point adjacency matrix is ​​updated to obtain the action semantics matrix of the target action.

7. The method for controlling the motion of a virtual character according to claim 6, wherein: The updating of the values ​​of the action semantics adjacent joint points in the row where the target joint point is located in the joint point adjacency matrix to preset values ​​to obtain the action semantic matrix of the target action includes: Setting the element value of the action semantic adjacent joint point in the row where the target joint point is located in the joint point adjacency matrix to a preset value; Calculating the distance between each of the action semantic adjacent joint points and the target joint point; Calculating the weight of each of the action semantic adjacent joint points using the distance; Calculate the product of the weight and the preset value to obtain a weight value; The element value of the action semantic adjacent joint point in the row where the target joint point is located in the joint point adjacency matrix is ​​updated to the weight value.

8. The method for controlling the motion of a virtual character according to claim 7, wherein: The step of using the distance to calculate the weight of each of the action semantic adjacent joint points includes: calculating the reciprocal of the distance; calculating the sum of the reciprocals of the distances; For each of the action semantics adjacent joint points, a ratio of the reciprocal to the sum of the reciprocals is calculated as the weight of the action semantics adjacent joint point.

9. The method for controlling the motion of a virtual character according to claim 5, wherein: The distance between the first product and the second product is calculated as the objective function using the following formula: min 0.5×‖L×tarPos3d-L×srcPos3d‖ 2 Among them, L is the action semantic matrix, tarPos3d is the initial vector of the virtual character's joint point, srcPos3d is the original vector of the original model's joint point, ‖.‖ 2 is the two-norm distance.

10. The method for controlling the motion of a virtual character according to claim 5, wherein: The step of generating length constraints between adjacent skeletal joint points on the virtual character by keeping the distance between adjacent skeletal joint points unchanged includes: Calculating the distance between two joint points of each bone of the virtual character as the original length of the bone; Calculate the distance between the vectors of the two joint points of each bone; The construction length constraints are as follows: ‖tarPos3d[i]-tarPos3d[j]‖-resetLength=0 Among them, resetLength is the original length of the bone between joint point i and joint point j of the virtual character, tarPos3d[i] and tarPos3d[j] are the vectors of joint point i and joint point j respectively.

11. The method for controlling the motion of a virtual character according to claim 5, wherein: The outer shape joints include preset collision points, and the skeletal joints include collision-constrained joints. The collision constraints between the skeletal joints and the outer shape joints on the virtual character are generated as follows: (tarPos3d[i]-collPos).dot(colldepth)≤0 Among them, collPos represents the vector of the collision point, tarPos3d[i] represents the vector of the joint point i subject to the collision constraint, tarPos3d[i]-collPos represents the vector from the joint point i subject to the collision constraint to the collision point, and the .dot(colldepth) dot product represents the projection of the vector from the joint point i subject to the collision constraint to the collision point in the direction perpendicular to the outer contour of the virtual character.

12. The method for controlling the motion of a virtual character according to any one of claims 1 to 4, wherein: Solving the minimum distance value of the objective function under the length constraint and the collision constraint to obtain target action data of the virtual character includes: A sequential quadratic programming method or a Lagrangian method is used to solve the minimum distance value of the objective function under the length constraint and the collision constraint to obtain the target action data of the virtual character.

13. A motion control device for a virtual character, characterized in that: include: An original motion data acquisition module is used to acquire original motion data, wherein the original motion data is position data of joint points on the skeleton when the original model performs the target motion; an initial motion data determining module, configured to determine initial motion data of the virtual character based on the original motion data, wherein the initial motion data is initial position data of joints on the skeleton of the virtual character, wherein the joints of the original model and the virtual character include skeletal joints and outline joints, wherein the outline joints are virtual points set on the outlines of the original model and the virtual character; an objective function generating module, configured to construct an objective function using the initial motion data and the original motion data, wherein the objective function is configured to calculate a distance between the initial motion data and the original motion data as similarity; a constraint construction module, configured to generate collision constraints between the skeletal joint points and the shape joint points on the virtual character, and to generate length constraints between adjacent joint points by keeping the distances between the skeletal joint points on the virtual character unchanged; an objective function solving module, configured to solve the minimum distance value of the objective function under the length constraint and the collision constraint, and obtain target action data of the virtual character, wherein the target action data is target position data of the joint points of the virtual character; The virtual character control module is used to control each joint point of the virtual character to move to the position indicated by the target position data, so as to drive the virtual character to perform the target action.

14. A motion control device for a virtual character, characterized in that: The motion control device of the virtual character includes: one or more processors; a storage device for storing one or more computer programs, When the one or more computer programs are executed by the one or more processors, the one or more processors implement the method for controlling the action of a virtual character according to any one of claims 1 to 12.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for controlling the action of a virtual character according to any one of claims 1 to 12 is implemented.

16. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor, the method for controlling the action of a virtual character according to any one of claims 1 to 12 is implemented.

Images