Method, apparatus, device and storage medium for controlling the movement of a virtual character

Abstract

In the embodiments of the present application, a method, an apparatus, a device, and a storage medium for controlling the movement of a virtual character are disclosed, and include the following: acquiring original movement data, the original movement data being position data of a plurality of skeletal joint points of an original model when the original model executes a target movement, determining initial position data of a plurality of skeletal joint points of a virtual character based on the original movement data, constructing a data target function using the initial position data and the original movement, generating collision constraints between the plurality of skeletal joint points of the virtual character and an external joint point of a virtual character, generating a length constraint so that the distance between adjacent skeletal joint points among the plurality of skeletal joint points of the virtual character does not change, solving the minimum distance of the target function under the length constraint and the collision constraint, obtaining target position data of the plurality of skeletal joint points of the virtual character, and controlling the plurality of skeletal joint points of the virtual character to move to positions indicated by the target position data, thereby driving the virtual character to execute the target movement.

Description

[Technical field]

[0001] This application claims priority to Chinese Patent Application No. 202210313961.2, filed with the State Intellectual Property Office of the People's Republic of China on March 28, 2022, the entire contents of which are incorporated herein by reference.

[0002] The present application relates to the technical field of virtual interaction, for example, to a method, an apparatus, a device, and a storage medium for controlling the movement of a virtual character. [Background technology]

[0003] With the emergence of virtual characters such as virtual newscasters, virtual idols, and virtual employees, there are an increasing number of applications in which the movements of virtual characters are controlled based on captured motion data, and there is also an increasing need to multiplex the captured motion data in real time onto multiple virtual characters with different external shapes (such as body fat, height, limb length, large heads, pompom skirts, etc.).

[0004] There are two main methods for controlling the movement of a virtual character. One method involves scaling the skeleton of the original model to the skeleton of the virtual character at the same ratio to obtain the amount of translational movement, correcting the movement data according to the amount of translational movement, and having the movement data corrected by the translational momentum act on the virtual character. The other method involves directly applying the rotational data of the skeleton of the original model to a virtual character of a different ratio using forward kinematics.

[0005] The above method of scaling at the same rate does not take into account the external shape of the virtual character when controlling the virtual character, which causes the loss or collapse of action semantics and even the phenomenon of model penetration. The method of controlling a virtual character using forward kinematics causes the loss or collapse of semantics as shown in Figure 1, where the left side of Figure 1 is a schematic diagram of the original model performing a salute action, and the center is the action performed by the virtual character after controlling the virtual character with forward kinematics. As can be seen from this, there are defects such as loss or collapse in the salute action performed by the virtual character in the schematic diagram in the center and the salute action performed by the predicted virtual character in the schematic diagram on the right. Summary of the Invention [Problem to be solved by the invention]

[0006] The embodiments of the present application provide a method, apparatus, device, and storage medium for controlling the movement of a virtual character, which solve the problems that exist in controlling the movement of a virtual character in the related art, such as loss of action semantics, collapse of action semantics, and model penetration. [Means for solving the problem]

[0007] An embodiment of the present application provides a method for controlling a movement of a virtual character, the method for controlling the movement of the virtual character includes: acquiring original movement data, the original movement data being position data of a plurality of skeletal joint points of the original model when the original model executes a target movement; determining initial movement data of a virtual character based on the original movement data, the initial movement data being initial position data of a plurality of skeletal joint points of the virtual character; constructing an objective function using the initial movement data and the original movement data, the objective function being used to calculate a similarity between the initial movement data and the original movement data; generating collision constraints between the plurality of skeletal joint points of a virtual character and the exterior joint points of the virtual character, and generating length constraints between adjacent skeletal joint points among the plurality of skeletal joint points of the virtual character such that a distance between the adjacent skeletal joint points among the plurality of skeletal joint points of the virtual character does not change; solving a minimum distance value under the length constraint and the collision constraint of the objective function to obtain target motion data of the virtual character, the target motion data being target position data of the plurality of skeletal joint points of the virtual character; and controlling the plurality of skeletal joint points of the virtual character to move to positions indicated by the target position data, thereby driving the virtual character to perform the target motion.

[0008] In an embodiment of the present application, a control device for a movement of a virtual character is provided, the control device for the movement of the virtual character includes an original movement data acquisition module, an initial movement data determination module, an objective function generation module, a constraint construction module, an objective function solving module, and a virtual character control module, the original movement data acquisition module is configured to acquire original movement data, the original movement data being position data of a plurality of skeletal joint points of the original model when the original model executes a target movement, the initial movement data determination module is configured to determine initial movement data of a virtual character based on the original movement data, the initial movement data being initial position data of a plurality of skeletal joint points of the virtual character, the objective function generation module is configured to construct an objective function using the initial movement data and the original movement data, an objective function is used to calculate a similarity between the initial motion data and the original motion data, the constraint construction module is configured to generate collision constraints between the multiple skeletal joint points of the virtual character and exterior joint points of the virtual character, and to generate length constraints between adjacent skeletal joint points among the multiple skeletal joint points of the virtual character such that a distance between the adjacent skeletal joint points among the multiple skeletal joint points of the virtual character does not change, the objective function solving module is configured to solve a minimum distance value under the length constraint and the collision constraint of the objective function to obtain target motion data of the virtual character, the target motion data being target position data of the multiple skeletal joint points of the virtual character, and the virtual character control module is configured to drive the virtual character to perform the target motion by controlling the multiple skeletal joint points of the virtual character to move to positions indicated by the target position data.

[0009] In an embodiment of the present application, a device for controlling movement of a virtual character is provided, the device for controlling movement of the virtual character includes at least one processor and a storage device, the storage device is configured to store at least one computer program, the at least one computer program being executed by the at least one processor to cause the at least one processor to realize the method for controlling movement of a virtual character described in the present application.

[0010] An embodiment of the present application provides a computer-readable storage medium having a computer program stored therein, the computer program implementing the method for controlling the movement of a virtual character described in the present application when executed by a processor.

[0011] In an embodiment of the present application, a computer program product is provided, the instructions of which, when executed by a processor, implement the method of controlling the movement of a virtual character. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a schematic diagram illustrating a breakdown in action semantics in controlling the movement of a virtual character in the related art. [Diagram 2] 1 is a flowchart of a method for controlling the movement of a virtual character provided in Example 1 of the present application. [Figure 3A] 11 is a flowchart of a method for controlling the movement of a virtual character provided in Example 2 of the present application. [Figure 3B] FIG. 1 is a schematic diagram of an example of adding contour articulation points in the present application. [Figure 3C] FIG. 2 is a schematic diagram of skeletal articulation points in an embodiment of the present application. [Figure 3D] FIG. 2 is a schematic diagram illustrating a graph of a joint point adjacency matrix in an embodiment of the present application. [Figure 3E] FIG. 2 is a schematic diagram of operational adjacency relationships in an embodiment of the present application. [Figure 3F]FIG. 2 is a schematic diagram of a collision constraint in an embodiment of the present application. [Figure 4] FIG. 11 is a structural block diagram of a control device for a virtual character's movement provided in the third embodiment of the present application; [Diagram 5] FIG. 11 is a structural block diagram of a control device for the movement of a virtual character provided in Example 4 of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The present application will be described below in conjunction with the drawings and examples. It is understood that the examples described herein are merely for the purpose of explaining the present application, and are not intended to limit the present application. For the sake of convenience, only partial structures related to the present application are shown in the drawings.

[0014] Example 1

[0015] 2 is a flowchart of a method for controlling the movement of a virtual character provided in the first embodiment of the present application, the embodiment of the present application can be applied to a situation where the movement of an original model is captured to control the movement of a simulation of a virtual character, the method is performed by a control device for a movement of a virtual character in the embodiment of the present application, the control device for a movement of a virtual character can be realized by hardware or software, and can be integrated into the control of the movement of a virtual character provided in the embodiment of the present application. As shown in FIG. 2, the method for controlling the movement of a virtual character in the embodiment of the present application can include the following steps:

[0016] S201, original motion data is obtained. The original motion data is position data of a plurality of skeletal joint points of an original model when the original model executes a target motion.

[0017] In the embodiment of the present application, the original model may be a model that performs a target action, and the virtual character may be a character that imitates the target action performed by the original model. In one example, the original model may be a human body model, and the virtual character may be a digital human. Exemplarily, the original model may be a real human body in real life, and the virtual character may be a virtual anchor, virtual presenter, virtual doll, robot, etc. The original model may be an animal, and the virtual character may be a virtual animal. The original model and the virtual character have the same skeleton structure. For convenience of explanation, the embodiment of the present application will explain a method for controlling the action of a virtual character by taking as an example an original model that is a human in real life and a virtual character that is a virtual human.

[0018] In one application scenario, when the motion of a caster is captured to control a virtual doll in a network, the caster is an original model, the virtual doll is a virtual character, and the virtual doll needs to perform the same motion as the caster, when obtaining the original motion data, at least one image of the caster is collected by a camera, and joint point identification is performed by the at least one image, and position data of multiple skeletal joint points when the caster performs a target motion can be obtained as the original motion data. Optionally, at least one image can be input into a pre-trained human body joint point identification network to obtain position data of multiple skeletal joint points of the caster as the original motion data.

[0019] S202, initial motion data of a virtual character is determined based on the original motion data, and the initial motion data is initial position data of a plurality of skeletal joint points of the virtual character.

[0020] The motion data may be rotation data of the skeleton in the model, and the execution of the target motion of the original model may be regarded as the movement of the skeleton of the original model, and the skeleton may be configured by connecting multiple skeletons, and the multiple skeletons may have a parent-child connection relationship, and the rotational motion of the child skeleton relative to the parent skeleton may be recorded in a matrix. In the skeleton of the human body, the root skeleton may be the pelvis, and a child skeleton connected upward to the pelvis, a child skeleton of the child skeleton, etc., are in a transmission relationship, and for example, the pelvis goes upward through the spine, upper arm, and forearm to reach the hand, and the motion data of the hand can be obtained by sequentially multiplying the rotation matrices of all the skeletons passing from the pelvis to the hand. Since the skeletal joint points are nodes connecting between skeletons, the position data of multiple skeletal joint points can be determined by obtaining the rotation data of the skeleton, or the rotation data of the skeleton constituted by the skeletal joint points can be obtained by obtaining the position data of multiple skeletal joint points.

[0021] In an embodiment of the present application, the original motion data indicates a target motion to be performed by the original model, and the original motion data can be assigned to the skeletons of a virtual character; that is, the multiple skeletons of the virtual character are initialized so that the multiple skeletons of the virtual character and the corresponding skeletons in the original model have the same rotation data; that is, the position data of the multiple skeletal joint points of the virtual character are initialized, thereby solving the optimal positions of the joint points within a range near the position data, reducing the difficulty of finding a solution and improving the efficiency of finding the solution, thereby realizing the control of the motion of the virtual character in real time.

[0022] In another alternative embodiment, the joint points may include skeleton joint points and contour joint points, where the skeleton joint points are nodes connecting skeletons, and the contour joint points may be virtual points set on the contours of the original model and the virtual character to avoid penetrating the models. In one example, one frame of original motion data may be applied to virtual characters with the same skeleton and different contours. For example, some virtual dolls are fat and some are thin, or some virtual humans have large heads or different contours due to wearing skirts, etc. Therefore, one or more virtual points may be set as contour joint points on the contours of the original model and the virtual character that need to avoid penetrating the models.

[0023] S203, constructing an objective function using the initial motion data and the original motion data.

[0024] The original motion data indicates the motion performed by the original model, and the initial motion data indicates the initialization motion of the virtual character; the initial motion data and the original motion data can be substituted into an objective function, and the objective function is used to calculate the similarity between the initial motion data and the original motion data, i.e., to calculate the similarity between the motion performed by the original model and the motion of the virtual character.

[0025] In one example, the similarity can be represented by the distance between the initial motion data and the original motion data, and the smaller the similarity, the closer the motion of the virtual character is to the motion of the original model. In the objective function, the original motion data is a fixed numerical value, the initial motion data is a variable, and the objective function value is a dependent variable, and the objective function value can be minimized by repeatedly updating the initial motion data. In one example, the objective function can be a function that calculates the distance between two numerical values, such as an L2 norm (Euclidean) distance function, a Chebyshev distance function, etc., and the embodiment of the present application is not limited to the objective function that calculates the similarity.

[0026] S204: generating collision constraints between the plurality of skeletal joint points of the virtual character and the external joint points of the virtual character, and generating length constraints between adjacent skeletal joint points so that the distance between adjacent skeletal joint points among the plurality of skeletal joint points of the virtual character does not change.

[0027] For a bone at the same position in the skeleton, the length of the original model of the bone may not match that of the virtual character, and the bone lengths of different virtual characters may not be the same, for example, virtual character A has long arms and virtual character B has short arms, and since the length of each bone is fixed and unchanging for the same virtual character, the length of each bone in the virtual character can be restricted not to change. For example, the length of each bone of the virtual character can be calculated, and the distance between two adjacent bone joint points constituting each bone in the initial motion data can be calculated, and the difference between this distance and the original distance of the bone is used as the length restriction, that is, the position of the bone joint point is changed while ensuring that the length of the bone composed of the two bone joint points does not change.

[0028] At the same time, skeletal joint points and collision points (external joint points) that are pre-installed on the virtual character and that need to be subject to collision constraints are determined, and the distance between the skeletal joint points that are subject to the collision constraints and the collision points is calculated as the collision depth. By constraining the collision depth to be 0 or less, it is possible to ensure that the skeletal joint points that are subject to the collision constraints do not collide with the collision points, and the movement of the virtual character does not penetrate the model.

[0029] S205, solving the minimum distance value of the objective function under the length constraint and the collision constraint to obtain target motion data of the virtual character, where the target motion data is target position data of a plurality of skeletal joint points of the virtual character.

[0030] The process of solving the minimum distance value of the objective function under the length constraint and the collision constraint is as follows: When it is ensured that the skeleton length does not change and that the skeleton joint points subject to the collision constraint do not collide with the collision points, the initial motion data is repeatedly changed so that the function value of the objective function is minimized, that is, the positions of the multiple skeleton joint points of the virtual character are repeatedly changed so that the objective function value is minimized, thereby obtaining the position data of the multiple skeleton joint points of the virtual character; in practical applications, the target position data of the skeleton joint points of the virtual character can be obtained by solving the optimal solution of the objective function by sequential quadratic programming (SQP) or the augmented Lagrange method (ALM); the solution method of sequential quadratic programming (SQP) or the augmented Lagrange method (ALM) can be referred to the related art, and a detailed description will be omitted here.

[0031] S206: A plurality of skeletal joint points of the virtual character are controlled to move to positions designated by the target position data, thereby driving the virtual character to execute the target action.

[0032] After solving the target position data of the multiple skeletal joint points of the virtual character, the multiple skeletal joint points in the skeleton of the virtual character can be controlled to move to the positions indicated by the target position data, and after the multiple skeletal joint points are positioned at the positions indicated by the target position data, the motions indicated by the multiple skeletons composed of the skeletal joint points are target motions to be executed by the original model.

[0033] In the embodiment of the present application, after obtaining original motion data when an original model executes a target motion, initial motion data of a virtual character is determined based on the original motion data, an objective function is constructed using the initial motion data and the original motion data, the objective function is used to calculate the similarity between the initial motion data and the original motion data, collision constraints are generated between a plurality of skeletal joint points of the virtual character and the external joint points of the virtual character, and a length constraint is generated between adjacent skeletal joint points of the plurality of skeletal joint points of the virtual character so that the distance between adjacent skeletal joint points among the plurality of skeletal joint points of the virtual character does not change.Furthermore, the minimum distance value under the length constraint and collision constraint of the objective function is solved to obtain target motion data of the virtual character, the target motion data is target position data of the joint points of the virtual character, and finally, the plurality of skeletal joint points of the virtual character are controlled to move to the positions indicated by the target position data, thereby driving the virtual character to execute the target motion. On the one hand, the smaller the distance between the initial action data and the original action data, the closer the action of the virtual character is to the action of the original model, ensuring that the virtual character can accurately execute the target action performed by the original model; on the other hand, the length constraint and collision constraint ensure the completeness of action semantics and avoid model penetration.

[0034] Example 2

[0035] 3A is a flowchart of a method for controlling the movement of a virtual character provided in Example 2 of the present application, and the embodiment of the present application will be described based on the above-mentioned Example 1. As shown in FIG. 3A, the method for controlling the movement of a virtual character in the embodiment of the present application can include the following steps: S301, obtain original movement data, where the original movement data is position data of multiple skeletal joint points of the original model when the original model performs a target movement;

[0036] In an embodiment of the present application, before acquiring original motion data, joint points of an original model and a virtual character can be set in advance, and the set joint points include skeleton joint points and contour joint points, where the skeleton joint points are two nodes constituting each skeleton, the contour joint points are virtual joint points set based on the contour of the virtual character to avoid penetration of the model when the virtual character moves, and the contour joint points can be set based on different contours of the virtual character.

[0037] For a virtual doll as shown in FIG. 3B, the head of the virtual doll is relatively large and the outline of the skirt worn by the virtual doll is large. In order to prevent the hands from entering the head or into the skirt or penetrating the model when moving the hands, outline joint points can be added to the head and the skirt. In one example, outline joint points (e.g., the black squares P1, P2, P3 on the head in FIG. 3B) are added to the head, and outline joint points (e.g., the black squares P4, P5, P6, P7 in FIG. 3B) are added to the skirt. Of course, in actual applications, different outline joint points can be preset according to different virtual characters, and the embodiments of the present application are not limited thereto.

[0038] In one alternative embodiment, when acquiring original motion data, an image of an original model is collected, and joint point identification is performed on the image, and position data of multiple skeletal joint points in the original model can be obtained as original motion data. For example, if the original model is a caster, at least one frame of image of the caster is collected by a camera and input into a pre-trained human body joint point identification network, and position data of the skeletal joint points of the caster is obtained as original motion data.

[0039] FIG. 3C is a schematic diagram of a human skeleton, which is composed of a number of skeletal structures, each of which has a skeletal joint point at both ends. In the embodiment of the present application, the model motion is represented by the position data of 17 skeletal joint points (point 0-point 16). In one example of the present application, the original motion data may be rotation data of the skeletons in the skeleton, and the pelvis may be the root skeleton, and the other skeletons may be child skeletons or secondary skeletons of the pelvis. The rotation data is the position data of each skeleton relative to the parent skeleton. For example, as shown in FIG. 3C, Assuming that the rotation data of skeleton P07 relative to skeleton joint point 0 is D07, and the rotation data of skeleton P78 relative to skeleton P07 is D78, the rotation data of skeleton P78 relative to skeleton joint point 0 is D07 x D78, and similarly, the rotation data of each skeleton relative to skeleton joint point 0 can be obtained. Furthermore, since the skeleton joint points are nodes that connect skeletons, obtaining the rotation data of a skeleton makes it possible to determine the position data of a plurality of skeletal joint points, or obtaining the position data of a plurality of skeletal joint points can also obtain the rotation data of a skeleton made up of the skeletal joint points.

[0040] S302: Calculating rotation data of the skeleton between every two adjacent skeleton joint points based on position data for every two adjacent skeleton joint points among a plurality of skeleton joint points of an original model in the original motion data.

[0041] In an optional embodiment of the present application, the position data may be the three-dimensional coordinates of skeletal joint points, and the identification of the human body joint points is the identification of the three-dimensional coordinates of multiple joint points relative to the human body coordinate system. For example, when the coordinate system origin is the skeletal joint point 0 as shown in FIG. 3C, when the human body joint points are identified, the three-dimensional coordinates of multiple skeletal joint points from skeletal joint point 1 to skeletal joint point 16 relative to skeletal joint point 0 can be obtained, and thus the rotation data of each skeleton relative to the parent skeleton can be calculated based on the coordinates of the two skeletal joint points constituting each skeleton.

[0042] As shown in FIG. 3C, the pelvis skeletal joint point 0 is set as the coordinate origin, and the three-dimensional coordinates of skeletal joint point 0 and skeletal joint point 7 are obtained. Then, rotation data of skeleton P07 can be calculated based on the three-dimensional coordinates of skeletal joint point 7 and skeletal joint point 0. Then, rotation data of skeleton P78 relative to its parent skeleton P07 is calculated based on the three-dimensional coordinates of skeletal joint point 8 and skeletal joint point 7. By multiplying the rotation data of skeleton P07 and the rotation data of skeleton P78, the rotation data of skeleton P78 relative to skeletal joint point 0 can be obtained. In the same manner, rotation data of multiple skeletons relative to skeletal joint point 0 is obtained.

[0043] S303: Rotation data of each skeleton in the original model is transferred to the corresponding skeleton of the virtual character as rotation data of the skeleton of the virtual character, thereby obtaining initial movement data of the virtual character.

[0044] The original motion data indicates a target motion to be performed by the original model, and the original motion data can be assigned to the skeletons of a virtual character; that is, the multiple skeletons of the virtual character are initialized so that the multiple skeletons of the virtual character and the corresponding skeletons in the original model have the same rotation data; that is, by initializing the position data of the multiple skeletal joint points of the virtual character, optimal positions of the skeletal joint points are solved within a range near the position data, reducing the difficulty of finding a solution and improving the efficiency of finding the solution, thereby enabling the motion of the virtual character to be controlled in real time.

[0045] Similarly, for a virtual character, the rotation data of each skeleton of the virtual character and the rotation data of the corresponding skeleton in the original model are set in the same way, with the pelvis skeletal joint point 0 as the origin, and the rotation data of each skeleton is set sequentially to become the initial position of each skeletal joint point of the virtual character, thereby solving the optimal position of the skeletal joint point within a range near the initial position, reducing the difficulty of finding a solution and improving the efficiency of finding the solution, thereby realizing the control of the movement of the virtual character in real time.

[0046] S304, calculating original vectors of a plurality of skeletal joint points of an original model using the original motion data, and calculating initial vectors of a plurality of skeletal joint points of a virtual character using the initial motion data.

[0047] For example, the three-dimensional coordinates of multiple skeletal joint points can be calculated according to the rotation data of the skeleton, and the three-dimensional coordinates of the multiple skeletal joint points can be connected as one vector, which is the vector of all the skeletal joint points of the model. As shown in FIG. 3C, there are 17 skeletal joint points in total, and each skeletal joint point has three-dimensional coordinate values ​​of x, y, and z. All the skeletal joint points of the model can be written as one vector x∈R^51.

[0048] S305, an action semantics matrix of a target action is generated based on the skeletal structure of the original model and the preset action semantics adjacency relationship.

[0049] In an optional embodiment of the present application, a joint point adjacency matrix of an original model can be obtained, and each element value in the row where each skeleton joint point is located in the joint point adjacency matrix indicates a joint adjacency relationship between the skeleton joint point and other skeleton joint points, and for each target skeleton joint point in the original model, an action semantics adjacent joint point of the target skeleton joint point is determined based on a preset action semantics adjacent relationship, and the element value of the action semantics adjacent joint point in the row where the target skeleton joint point is located in the joint point adjacency matrix is ​​updated to obtain an action semantics matrix of a target motion. The target skeleton joint point can be defined as a skeleton joint point of a more noteworthy motion of the original model, and is generally a skeleton joint point used to convey motion information in the skeleton of the original model, such as an arm or hand.

[0050] The joint point adjacency matrix indicates the adjacency relationship between the skeletal joint points in the model. As shown in FIG. 3C, the skeletal joint point 8 is adjacent to the skeletal joint point 11, the skeletal joint point 9, the skeletal joint point 14, and the skeletal joint point 7, respectively. The skeletal joint point 8 is not adjacent to any other skeletal joint point. When two skeletal joint points are adjacent, the element value of the corresponding element in the joint point adjacency matrix is ​​represented as -1. When two skeletal joint points are not adjacent, the element value of the corresponding element in the joint point adjacency matrix is ​​represented as 0. FIG. 3D is the joint point adjacency matrix of the multiple skeletal joint points in FIG. 3C. For ease of notation, the matrix is ​​tabulated. In FIG. 3D , the first row and the first column are the numbers of the skeletal joint points. Take skeletal joint point 8 as an example. In the row where skeletal joint point 8 is located, since skeletal joint point 8 is adjacent to skeletal joint point 11, skeletal joint point 9, skeletal joint point 14, and skeletal joint point 7, the corresponding element value is −1. If skeletal joint point 8 has four adjacent skeletal joint points, the corresponding element value in the row and column where skeletal joint point 8 is located is 4, and the element values ​​corresponding to the other non-adjacent skeletal joint points are 0. As can be seen from FIG. 3D , the diagonal line in the joint point adjacency matrix represents the number of adjacent skeletal joint points for each skeletal joint point.

[0051] The predefined action semantics adjacency indicates the adjacency between skeletal joint points in one action, i.e., the semantics of the action is indicated by defining the adjacency of skeletal joint points, and as shown in FIG. 3E, taking hand action as an example, the adjacency of the hand skeletal joint point 16 with other skeletal joint points can be predefined, and in one example, the adjacency of the hand skeletal joint point 16 with five skeletal joint points, namely, the head skeletal joint point 10, the shoulder skeletal joint point 14, the spine skeletal joint point 7, the thigh skeletal joint point 1, and the foot skeletal joint point 3, can be defined. Similar to the joint point adjacency matrix, the predefined action semantics adjacency can also be graphed, and the element value of the row in which the skeletal joint point 16 is located in the table corresponding to the predefined action semantics adjacency is shown in Table 1.

[0052] [Table 1]

[0053] When two skeletal joint points are adjacent, the element value is -1, and when two skeletal joint points are not adjacent, the element value is 0, so the action semantics connection relationship of the skeletal joint point 16 can be known from the row in which the above skeletal joint point 16 is located. For the skeletal joint points in the original model and the virtual character, the action semantics adjacency relationship of each skeletal joint point can be defined in advance, and as shown in Fig. 3C, the adjacency between the hand skeletal joint point 16 and five skeletal joint points, namely, the head skeletal joint point 10, the shoulder skeletal joint point 14, the spine skeletal joint point 7, the thigh skeletal joint point 1, and the foot skeletal joint point 3, is defined, and more attention is paid to the hand action, so the action semantics adjacency between the hand skeletal joint point 16 and other relatively fixed skeletal joint points is defined. If more attention is focused on head movement, action semantics adjacency can be defined between skeletal joint point 10 of the head and skeletal joint point 8, skeletal joint point 11, skeletal joint point 14, and skeletal joint point 7, and the embodiments of the present application do not place any restrictions on the action semantics adjacency relationship of skeletal joint points.

[0054] The above Table 1 is a representation of the action semantics connection relationship of the skeletal joint points 16 in a matrix. By updating the element values ​​of Table 1 to a matrix as shown in FIG. 3D, and similarly updating the element values ​​in the rows in which other skeletal joint points are located, an action semantics matrix of a target motion can be obtained. This action semantics matrix indicates the adjacent relationship of each skeletal joint point in the motion when the original model executes the target motion, that is, it indicates the action adjacent relationship of the two skeletal joint points connected by the dashed line in FIG. 3E, and the entire action semantics matrix indicates the action semantics of the motion performed by the original model.

[0055] After setting the element value of the action semantics adjacent joint point in the row in which the target skeleton joint point is located in the joint point adjacency matrix to a preset value, distances between the multiple action semantics adjacent joint points and the target skeleton joint point are calculated, and a weight of each action semantics adjacent joint point is calculated using the distance, and a weight value is obtained by calculating a product of the weight of each action semantics adjacent joint point and the preset value, and the element value of the action semantics adjacent joint point in the row in which the target skeleton joint point is located in the joint point adjacency matrix is ​​modified to be equal to the weight value.

[0056] As shown in FIG. 3D, the skeletal joint point 16 is adjacent to five skeletal joint points, namely, the head skeletal joint point 10, the shoulder skeletal joint point 14, the spine skeletal joint point 7, the thigh skeletal joint point 1, and the foot skeletal joint point 3, in the action semantics. Therefore, the element values ​​of the skeletal joint point 10, the shoulder skeletal joint point 14, the spine skeletal joint point 7, the thigh skeletal joint point 1, and the foot skeletal joint point 3 in the row where the skeletal joint point 16 is located in FIG. 3D are changed to the values ​​in Table 1, i.e., the preset value of -1, and then the skeletal joint points in the original model are changed to the values ​​in Table 1. The distance between each of the joint points 10, 14, 7, 1, and 3 and the skeleton joint point 16 can be calculated. For example, the distance between two skeletal joint points is calculated according to the three-dimensional coordinates of the two skeletal joint points, and the reciprocal of each distance is calculated, and the sum of the reciprocals of all the reciprocals is calculated. For each of the joint points 10, 14, 7, 1, and 3, the ratio of the reciprocal of the distance to the sum of the reciprocals is calculated as the weight of the skeletal joint point, which is shown in the following formula:

number

[0057] In the above formula, Distance_ij is the distance from the jth joint point adjacent to the ith joint point in the action semantics to the ith joint point, and w jis the weight of the skeletal joint point j, and it can be seen that the greater the distance between the skeletal joint point j and the skeletal joint point i, the smaller the weight. As shown in FIG. 3E, the distance between the foot skeletal joint point 3 and the hand skeletal joint point 16 is the farthest, and when the hand performs the target motion, the relationship between the semantics of the skeletal joint point 16 and the foot is small, that is, the relationship between the hand motion and the foot joint is not large. Conversely, the relationship between the skeletal joint point 16 and the shoulder skeletal joint point 14 is the largest. Thus, when realizing different motions, the weight of the skeletal joint point is determined by dynamically calculating the distance between adjacent skeletal joint points in the action semantics, and the action semantics is better interpreted according to the weight.

[0058] After obtaining the weight of the skeleton joint point, the weight value is multiplied by the element value corresponding to the skeleton joint point to obtain a new element value. Taking the above Table 1 as an example, assuming that the weights of the skeleton joint point 16 and the skeleton joint points 1, 3, 7, 10, and 14 are 0.1, 0.2, 0.2, 0.2, and 0.3, respectively, after calculation, the above Table 1 is updated as follows:

[0059] [Table 2]

[0060] After updating the weight values ​​of all the skeletal joint points, a weighted action semantics matrix is ​​obtained, and for different actions, the distances between the skeletal joint points are different, and the weights are also different, the farther the distance, the smaller the weight, and the closer the distance, the larger the weight. By dynamically allocating weights, the action semantics matrix can better interpret the action semantics.

[0061] S306, multiplying the action semantics matrix and the original vector to obtain a first product, and multiplying the action semantics matrix and the initial vector to obtain a second product.

[0062] Let L be the action semantics matrix of the target action in S305, srcPos3d be the vector formed by the positions of all the skeletal joint points of the original model in S304, and tarPos3d be the vector formed by the positions of all the skeletal joint points of the virtual character.

number

number

[0063] S307: Calculate the distance between the first product and the second product as the target function.

[0064] In one alternative embodiment, the objective function is:

number

[0065] L is the action semantics matrix of the target action, tarPos3d is the initial vector of multiple skeletal joint points of the virtual character, srcPos3d is the original vector of multiple skeletal joint points of the original model,

number

[0066] S308: generating collision constraints between the plurality of skeletal joint points of the virtual character and the external joint points of the virtual character, and generating length constraints between adjacent skeletal joint points so that the distance between adjacent skeletal joint points among the plurality of skeletal joint points of the virtual character does not change.

[0067] In one optional embodiment, for the length constraint, the distance between the two skeleton joint points of each skeleton in the virtual character can be calculated as the original length of the skeleton, and the vector distance between the two skeleton joint points of each skeleton is calculated, and the length constraint is constructed as follows:

number

[0068] resetLength is the original length of the skeleton between skeletal joint point i and skeletal joint point j of the virtual character, tarPos3d[i] and tarPos3d[j] are the vectors of skeletal joint point i and skeletal joint point j, respectively. This length constraint indicates that when the positions of skeletal joint point i and skeletal joint point j are repeatedly changed in the process of finding a solution for the minimum value of the objective function, the distance between skeletal joint point i and skeletal joint point j after the change is equal to the original length resetLength, where i and j are both integers greater than or equal to zero.

[0069] Regarding collision constraints, the exterior joint points include collision points that are set in advance, and the skeleton joint points include joint points that are subject to collision constraints, and collision constraints between the multiple skeleton joint points of the virtual character and the exterior joint points of the virtual character are generated as follows:

number

[0070] collPos indicates the vector of the collision point, tarPos3d[i] indicates the vector of the joint point i that is subject to the collision constraint,

number

number

[0071] The principle of collision constraint is shown in Fig. 3F. In Fig. 3F, when the human body places both hands on the hips, in order to prevent the phenomenon that the human body's hands penetrate the body and penetrate the model, the contour joint point P2 is set on the body as a collision point, and the skeleton joint point P1 of the hand at the end of the forearm is set as a skeleton joint point (collision constraint joint point) that is subject to collision constraint, and the distance of the vector from the skeleton joint point P1 to the contour joint point P2 projected in the direction perpendicular to the outer surface of the body is the distance from P1 to P3, that is, the collision depth, and when the collision depth is less than or equal to 0, it indicates that the skeleton joint point P1 does not collide with the contour joint point P2, that is, the hand does not penetrate the body, thereby ensuring that the phenomenon of penetrating the model does not occur.

[0072] S309, using sequential quadratic programming or Lagrange algorithm, solve the minimum distance value of the objective function under the length constraint and the collision constraint to obtain the target motion data of the virtual character.

[0073] Solving for the target motion data essentially involves solving the following objective function:

number

[0074] That is, by repeatedly changing the positions of the skeletal joint points of the virtual character,

number

[0075] In practical applications, the optimal solution of the objective function can be solved by sequential quadratic programming (SQP) or the augmented Lagrange method (ALM), and then the target position data of the skeletal joint points of the virtual character can be obtained. For the solution methods by sequential quadratic programming (SQP) or the augmented Lagrange method (ALM), please refer to the related art.

[0076] In an alternative embodiment, for a target action, if the action semantics matrix L is fixed and invariant, and the collision constraint is collision depth equal to zero, the objective function can be simplified to a function of the equality constraints.

number

[0077] The process to find the solution is as follows:

[0078]

number

number

[0079] J is the Jacobian matrix of C(tarPos3d), and b is a constant after Taylor expansion. For the Taylor expansion, refer to the related art, and a detailed description will be omitted here.

[0080] After determining the action, we construct the Lagrangian function since the action semantics matrix is ​​fixed and invariant.

number

number

number

[0081] By setting the derivatives of equation (1) with respect to x and λ equal to 0, we obtain the following equation:

number

number

number

number

number

[0082] Until the final x is obtained by converging, the Gauss-Seidel method is used to iterate on x, and since x=tarPos3d-srcPos3d is fixed and invariant, tarPos3d is obtained, that is, the optimal position of the bone joint point of the virtual character is obtained. The iterative process can be referred to the iterative process of the Gauss-Seidel method in the related art, and a detailed description is omitted here.

[0083] S310 controls a plurality of skeletal joint points of the virtual character to move to positions designated by the target position data, thereby driving the virtual character to execute a target action.

[0084] After determining the target position data of the multiple skeletal joint points of the virtual character, the multiple skeletal joint points in the skeleton of the virtual character can be controlled to move to the positions indicated by the target position data, and after the multiple skeletal joint points are positioned at the positions indicated by the target position data, the motion expressed by the multiple skeletons constituted by the skeletal joint points is the target motion to be executed by the original model.

[0085] In an embodiment of the present application, after obtaining original motion data when an original model performs a target motion, initial motion data of a virtual character is determined based on the original motion data, original vectors of skeletal joint points of the original model are calculated using the original motion data, and initial vectors of skeletal joint points of the virtual character are calculated using the initial motion data, an action semantics matrix of the target motion is generated based on the skeletal structure of the original model and a preset action semantics adjacency relationship, a product of the action semantics matrix and the original vector and a product of the action semantics matrix and the initial vector are respectively calculated to obtain a first product and a second product, and a distance between the first product and the second product is calculated as a target function. Furthermore, generate collision constraints between the multiple skeletal joint points of the virtual character and the external joint points of the virtual character, generate length constraints between adjacent joint points so that the distance between the multiple skeletal joint points of the virtual character does not change, and solve the minimum distance value of the objective function under the length constraint and collision constraint to obtain target motion data of the virtual character, the target motion data being target position data of the multiple skeletal joint points of the virtual character, and finally control the multiple skeletal joint points of the virtual character to move to positions indicated by the target position data, thereby driving the virtual character to perform the target motion. On the one hand, the smaller the distance between the initial motion data and the original motion data, the closer the motion of the virtual character is to the motion of the original model, ensuring that the virtual character can accurately perform the target motion performed by the original model, and on the other hand, the length constraint and collision constraint ensure the completeness of the action semantics and avoid penetrating the model.

[0086] Example 3

[0087] FIG. 4 is a structural block diagram of a virtual character motion control device provided in the third embodiment of the present application. As shown in FIG. 4, the virtual character motion control device of the embodiment of the present application may specifically include the following modules: The original motion data acquisition module 401 is configured to acquire original motion data, and the original motion data is position data of a plurality of skeletal joint points of the original model when the original model executes a target motion. The initial motion data determination module 402 is configured to determine initial motion data of a virtual character based on the original motion data, and the initial motion data is initial position data of a plurality of skeletal joint points of the virtual character. The objective function generation module 403 is configured to construct an objective function using the initial motion data and the original motion data, and the objective function is used to calculate the similarity between the initial motion data and the original motion data. The constraint construction module 404 is configured to generate a collision constraint between the plurality of skeletal joint points of the virtual character and the external joint points of the virtual character, and generate a length constraint between adjacent skeletal joint points so that the distance between adjacent skeletal joint points among the plurality of skeletal joint points of the virtual character does not change. The objective function solving module 405 is configured to solve the minimum distance value of the objective function under the length constraint and the collision constraint to obtain target motion data of the virtual character, the target motion data being target position data of a plurality of skeletal joint points of the virtual character. The virtual character control module 406 is configured to control the plurality of skeletal joint points of the virtual character to move to positions indicated by the target position data, thereby driving the virtual character to perform the target motion.

[0088] The virtual character movement control device provided in the embodiments of the present application can execute the virtual character movement control method provided in the embodiments 1 and 2 of the present application, and the virtual character movement control device includes a corresponding functional module for executing the method.

[0089] Example 4

[0090] Referring to FIG. 5, a structural schematic diagram of a control device for a virtual character's movement in one embodiment of the present application is shown. As shown in FIG. 5, the control device for a virtual character's movement can include a processor 501, a storage device 502, a display 503 with a touch function, an input device 504, an output device 505, and a communication device 506. The number of processors 501 in the control device for a virtual character's movement can be one or more, and one processor 501 is shown in FIG. 5 as an example. The processor 501, the storage device 502, the display 503, the input device 504, the output device 505, and the communication device 506 of the control device for a virtual character's movement can be connected by a bus or other manner, and the processor 501, the storage device 502, the display 503, the input device 504, the output device 505, and the communication device 506 are connected by a bus in FIG. 5 as an example, and the control device for a virtual character's movement is used to execute the control method for a virtual character's movement provided in the embodiment of the present application.

[0091] Example 5

[0092] In an embodiment of the present application, a computer-readable storage medium is provided, and a computer program in the storage medium, when executed by a processor, realizes the method for controlling the movement of a virtual character as described in the above method embodiment.

[0093] Example 6

[0094] In an embodiment of the present application, a computer program product is provided, the instructions in the computer program product, when executed by a processor, realizing the method for controlling the movement of a virtual character as described in the above method embodiment.

[0095] For the purposes of this application, a computer-readable storage medium may be any instruction-executing system, apparatus or device that contains, stores, communicates, propagates or transmits a program, or may be used in conjunction with such an instruction-executing system, apparatus or device. Examples of computer-readable media (non-exhaustive list) include electrical connections with one or more wires (electronic devices), portable computer disks (magnetic devices), Random Access Memory (RAM), Read-Only Memory (ROM), Erasable Programmable Read-Only Memory (EPROM) or Flash memory, fiber optic devices, and portable optical disk read-only memory (CD-ROM). A computer-readable medium may also be paper or other suitable medium on which the program can be printed, for example, optically scanned on the paper or other medium, and edited, interpreted or otherwise processed as necessary in an appropriate manner to obtain the program in electronic form, which can then be stored in a computer memory. The computer program product may be a product including a computer-readable storage medium, which, when instructions in the computer-readable storage medium are executed by a processor, realizes the method of controlling the movement of a virtual character as described in the method embodiment above.

[0096] In addition, the embodiments of the apparatus, device, storage medium, and computer program product are basically similar to the embodiments of the method, so they have been described relatively briefly. However, for relevant points, reference may be made to the description of some of the embodiments of the method. [Explanation of symbols]

[0097] 601 Processor 602 Memory 603 Display 604 Input Device 605 Output Device 606 Communication Equipment

Claims

1. 1. A method for controlling a movement of a virtual character, comprising: Acquire original motion data, the original motion data being position data of a plurality of skeletal joint points of the original model when the original model executes a target motion; determining initial motion data of a virtual character based on the original motion data, the initial motion data being initial position data of a plurality of skeletal joint points of the virtual character; constructing an objective function using the initial motion data and the original motion data, and using the objective function to calculate a similarity between the initial motion data and the original motion data; generating collision constraints between the plurality of skeletal joint points of the virtual character and an external joint point of the virtual character, and generating length constraints between adjacent skeletal joint points among the plurality of skeletal joint points of the virtual character such that a distance between the adjacent skeletal joint points does not change; solving the minimum distance value of the objective function under the length constraint and the collision constraint to obtain target motion data of the virtual character, the target motion data being target position data of the plurality of skeletal joint points of the virtual character; and controlling the plurality of skeletal joint points of the virtual character to move to positions indicated by the target position data, thereby driving the virtual character to execute the target action. method.

2. Before obtaining the original operation data, setting joint points, the joint points including a plurality of skeleton joint points of the original model, and an outer joint point and a plurality of skeleton joint points of the virtual character; The method of claim 1.

3. Obtaining original motion data is acquiring an image of the original model; performing joint point identification on the image to obtain position data of a plurality of skeletal joint points in the original model as original motion data; The method of claim 1.

4. Determining initial motion data of a virtual character based on the original motion data includes: calculating rotation data of a skeleton between two adjacent skeletal joint points of the original model in the original motion data based on position data of the two adjacent skeletal joint points; Transferring rotation data of each skeleton in the original model to a corresponding skeleton of the virtual character as rotation data of the skeleton of the virtual character, thereby obtaining initial movement data of the virtual character. The method of claim 1.

5. Constructing an objective function using the initial motion data and the original motion data includes: Calculating original vectors of a plurality of skeletal joint points of the original model using the original motion data, and calculating initial vectors of a plurality of skeletal joint points of the virtual character using the initial motion data; generating an action semantics matrix of the target action based on a skeleton structure of the original model and a preset action semantics adjacency relationship; multiplying the action semantics matrix and the original vector to obtain a first product, and multiplying the action semantics matrix and the initial vector to obtain a second product; calculating a distance between the first product and the second product as an objective function; 5. The method according to any one of claims 1 to 4.

6. generating an action semantics matrix of the target action based on a skeleton structure of the original model and a preset semantics action adjacency relationship, obtaining a joint point adjacency matrix of the original model, and each element value in a row in which each skeleton joint point is located in the joint point adjacency matrix indicates a joint adjacency relationship between each skeleton joint point and other skeleton joint points; According to a preset action semantics adjacency relationship of each target skeletal joint point of the original model, determine an action semantics adjacency relationship of each target skeletal joint point, and the preset action semantics adjacency relationship of each target skeletal joint point includes a predefined adjacency relationship between the target skeletal joint point and other skeletal joint points; updating element values ​​of the action semantics adjacent joint points in a row in which each of the target skeleton joint points is located in the joint point adjacency matrix to obtain an action semantics matrix of the target action; The method according to claim 5.

7. The number of action semantics adjacent joint points of each target skeletal joint point is multiple, Updating element values ​​of the action semantics adjacent joint points in a row in which each of the target skeleton joint points is located in the joint point adjacency matrix to obtain an action semantics matrix of the target action, Setting an element value of each action semantics adjacent joint point in the row in which each of the target skeleton joint points is located in the joint point adjacency matrix to a preset value; Calculating a distance between each of the target skeletal joint points and each of the action semantics adjacent joint points in a row in which each of the target skeletal joint points is located; Calculating a weight of each action semantics adjacent joint point in a row in which each target skeletal joint point is located using the distance; Calculating the product of the weight of each action semantics adjacent joint point in the row where each target skeleton joint point is located and the preset value to obtain a weight value; updating an element value of each of the action semantics adjacent joint points in a row in which each of the target skeleton joint points is located in the joint point adjacency matrix to the weight value; The method according to claim 6.

8. Calculating a weight of each action semantics adjacent joint point in a row in which each target skeletal joint point is located using the distance, Calculating the inverse of a distance between each of the target skeletal joint points and each of the action semantics adjacent joint points in a row in which each of the target skeletal joint points is located; Calculating the sum of all reciprocals corresponding to all action semantics adjacent joint points in the row in which each target skeletal joint point is located; Calculating a ratio value between the sum and an inverse number corresponding to each action semantics adjacent joint point in a row in which each target skeleton joint point is located as a weight of each of the action semantics adjacent joint points, The method according to claim 7.

9. Calculating a distance between the first product and the second product as a target function using the following formula: [0010] where L is the action semantics matrix, tarPos3d is an initial vector of a plurality of skeletal joint points of the virtual character, and srcPos3d is an original vector of a plurality of skeletal joint points of the original model. [0025] is the binorm distance, The method according to claim 5.

10. generating a length constraint between adjacent skeletal joint points among the plurality of skeletal joint points of the virtual character so that a distance between the adjacent skeletal joint points does not change, Calculating a distance between two skeletal joint points of each skeleton of the virtual character as an original length of each skeleton; Calculating a vector distance of the two skeletal joint points of each of the skeletons; and constructing a length constraint as follows: [0030] where resetLength is the original length of the skeleton between two adjacent skeleton joint points i and j of the virtual character, tarPos3d[i] and tarPos3d[j] are the vectors of the skeleton joint points i and j, respectively, where i and j are integers equal to or greater than zero; The method according to claim 5.

11. The outer joint points include collision points set in advance, and the skeleton joint points include joint points that are subject to collision constraints. A collision constraint is generated between the plurality of skeleton joint points of the virtual character and the outer joint points of the virtual character as follows: [0045] Here, collPos indicates the vector of the pre-established collision point, tarPos3d[i] indicates the vector of the joint point i that is subject to the collision constraint, [0050] indicates a vector from the joint point i subject to the collision constraint to the pre-established collision point, [006] The dot product indicates a projection of a vector from the joint point i that is subject to the collision constraint to the pre-established collision point in a direction perpendicular to the outer contour of the virtual character, where i is an integer equal to or greater than zero. The method according to claim 5.

12. Solving the objective function for a minimum distance value under the length constraint and the collision constraint to obtain target motion data for the virtual character includes: solving the objective function for a minimum distance value under the length constraint and the collision constraint using a sequential quadratic programming or Lagrangian method to obtain target motion data for the virtual character; 5. The method according to any one of claims 1 to 4.

13. The method includes: an original motion data acquisition module; an initial motion data determination module; an objective function generation module; a constraint construction module; an objective function solving module; and a virtual character control module; The original motion data acquisition module is configured to acquire original motion data, the original motion data being position data of a plurality of skeletal joint points of the original model when the original model executes a target motion; The initial motion data determination module is configured to determine initial motion data of a virtual character based on the original motion data, the initial motion data being initial position data of a plurality of skeletal joint points of the virtual character; The objective function generation module is configured to construct an objective function using the initial motion data and the original motion data, and the objective function is used to calculate a similarity between the initial motion data and the original motion data; the constraint construction module is configured to generate collision constraints between the plurality of skeletal joint points of the virtual character and an outer joint point of the virtual character, and to generate length constraints between adjacent adjacent joint points of the plurality of skeletal joint points of the virtual character such that a distance between the adjacent adjacent joint points does not change; the objective function solving module is configured to solve a minimum distance value of the objective function under the length constraint and the collision constraint to obtain target motion data of the virtual character, the target motion data being target position data of the plurality of skeletal joint points of the virtual character; the virtual character control module is configured to control the plurality of skeletal joint points of the virtual character to move to positions indicated by the target position data, thereby driving the virtual character to execute the target action; A device for controlling the movement of a virtual character.

14. at least one processor and a storage device; the storage device is configured to store at least one computer program; The at least one computer program, when executed by the at least one processor, causes the at least one processor to implement a method according to any one of claims 1 to 12. A control device for the movement of a virtual character.

15. A computer-readable storage medium having a computer program stored thereon, The computer program, when executed by a processor, implements the method according to any one of claims 1 to 12. A computer-readable storage medium.

16. A computer program comprising: The instructions in the computer program, when executed by a processor, implement the method according to any one of claims 1 to 12. Computer program.

Images