A method, apparatus and related equipment for driving virtual humans

Abstract

This invention provides a method, apparatus, and related equipment for driving a virtual human. The method includes: receiving a video to be identified; identifying the video using a target 2D pose estimation model and a target 3D pose estimation model to obtain the coordinates of multiple human body 3D key points; calculating the rotation angles of all frames in the video based on the coordinates of the multiple human body 3D key points; storing the coordinates of the multiple human body 3D key points and the rotation angles as a preset format file according to the parent-child relationship between the coordinates of the multiple human body 3D key points; and inputting the preset format file into a preset engine to drive a virtual human model in the preset engine. Based on the pose estimation algorithm, the optimized target 2D pose estimation model and target 3D pose estimation model are used to extract the coordinates of multiple human body 3D key points in the video to be identified, improving the speed and accuracy of pose estimation. Using the pose estimation algorithm to drive the virtual human model reduces production costs and facilitates the expansion of application scope.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus, and related equipment for driving a virtual human. Background Technology

[0002] With the development of virtual reality technology, virtual digital character driving technology has broad application prospects and a huge market potential in many fields. Currently, the main method for driving virtual digital characters is motion capture, which involves acquiring motion data from real-world characters through a motion capture system and using that data to drive virtual characters on the same platform. However, motion capture methods are costly to produce, making large-scale application difficult and limiting their applicability. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a method, apparatus and related equipment for driving virtual humans, so as to solve the problem of high production costs for driving virtual data characters.

[0004] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0005] The first aspect of this invention discloses a method for driving a virtual human, the method comprising:

[0006] Receive the video to be identified;

[0007] The target two-dimensional pose estimation model is used to identify the video to be identified, and the coordinates of multiple human two-dimensional key points are obtained. The target two-dimensional pose estimation model is pre-built based on the two-dimensional pose estimation model.

[0008] The coordinates of the multiple two-dimensional key points of the human body are input into the target three-dimensional pose estimation model to obtain the coordinates of the multiple three-dimensional key points of the human body. The target three-dimensional pose estimation model is pre-optimized based on the three-dimensional pose estimation model.

[0009] The rotation angle of all frames in the video to be identified is calculated based on the coordinates of the multiple three-dimensional key points of the human body.

[0010] Based on the parent-child relationship between the coordinates of the multiple human body 3D key points, the coordinates of the multiple human body 3D key points and the rotation angle of all frames in the video to be identified are stored as a preset format file;

[0011] The preset format file is input into the preset engine to drive the virtual human model in the preset engine.

[0012] Preferably, the process of constructing a target two-dimensional attitude estimation model based on the two-dimensional attitude estimation model includes:

[0013] Obtain a two-dimensional attitude estimation model;

[0014] The two-dimensional attitude estimation model is compressed to obtain a compressed two-dimensional attitude estimation model;

[0015] The target two-dimensional pose estimation model is constructed based on the compressed two-dimensional pose estimation model, the preset convolutional block, and the preset dejitter module.

[0016] Preferably, the process of optimizing the three-dimensional pose estimation model to obtain the target three-dimensional pose estimation model includes:

[0017] Obtain a 3D pose estimation model;

[0018] A refinement module is added to the three-dimensional attitude estimation model to obtain the target three-dimensional attitude estimation model.

[0019] Preferably, the step of calculating the rotation angle of all frames in the video to be identified based on the coordinates of the plurality of human body 3D key points includes:

[0020] Calculate the rotation matrix based on the coordinates of the multiple three-dimensional key points of the human body;

[0021] Convert the rotation matrix into a quaternion;

[0022] The quaternion is converted into Euler angles to obtain the rotation angles of all frames in the video to be identified.

[0023] A second aspect of this invention discloses an apparatus for driving a virtual human, the apparatus comprising:

[0024] The receiving unit is used to receive the video to be identified;

[0025] The recognition unit is used to recognize the video to be recognized using a target two-dimensional pose estimation model to obtain the coordinates of multiple human two-dimensional key points. The target two-dimensional pose estimation model is pre-constructed based on the two-dimensional pose estimation model.

[0026] The input unit is used to input the coordinates of the multiple human body two-dimensional key points into the target three-dimensional pose estimation model to obtain the coordinates of the multiple human body three-dimensional key points. The target three-dimensional pose estimation model is pre-optimized based on the three-dimensional pose estimation model.

[0027] The calculation unit is used to calculate the rotation angle of all frames in the video to be identified based on the coordinates of the multiple human body three-dimensional key points;

[0028] The storage unit is used to store the coordinates of the multiple human body three-dimensional key points and the rotation angle of all frames in the video to be identified as a preset format file according to the parent-child relationship between the coordinates of the multiple human body three-dimensional key points;

[0029] The driving unit is used to input the preset format file into the preset engine to drive the virtual human model in the preset engine.

[0030] Preferably, the device further includes:

[0031] The first acquisition unit is used to acquire the two-dimensional attitude estimation model;

[0032] A compression unit is used to compress the two-dimensional attitude estimation model to obtain a compressed two-dimensional attitude estimation model;

[0033] The construction unit is used to construct the target two-dimensional pose estimation model based on the compressed two-dimensional pose estimation model, the preset convolutional block, and the preset dejitter module.

[0034] Preferably, the device further includes:

[0035] The second acquisition unit is used to acquire the three-dimensional pose estimation model;

[0036] An additional unit is added to the three-dimensional attitude estimation model to add a refinement module, thereby obtaining the target three-dimensional attitude estimation model.

[0037] Preferably, the computing unit includes:

[0038] The calculation module is used to calculate the rotation matrix based on the coordinates of the multiple human body 3D key points;

[0039] The first conversion module is used to convert the rotation matrix into a quaternion;

[0040] The second conversion module is used to convert the quaternion into Euler angles to obtain the rotation angles of all frames in the video to be identified.

[0041] A third aspect of this invention discloses an electronic device, comprising: a processor and a memory, the processor and the memory being connected via a communication bus; wherein, the processor is used to call and execute a program stored in the memory; the memory is used to store the program, the program being used to implement the method for driving a virtual human as described in any one of the preceding embodiments.

[0042] A fourth aspect of the present invention discloses a computer-readable storage medium storing computer-executable instructions for implementing the method of driving a virtual human as described in any one of the preceding embodiments.

[0043] Based on the above embodiments of the present invention, a method, apparatus, and related equipment for driving a virtual human are provided. The method includes: receiving a video to be identified; identifying the video to be identified using a target two-dimensional pose estimation model to obtain the coordinates of multiple human two-dimensional key points; inputting the coordinates of the multiple human two-dimensional key points into a target three-dimensional pose estimation model to obtain the coordinates of multiple human three-dimensional key points; calculating the rotation angle of all frames in the video to be identified based on the coordinates of the multiple human three-dimensional key points; storing the coordinates of the multiple human three-dimensional key points and the rotation angle of all frames in the video to be identified as a preset format file according to the parent-child relationship between the coordinates of the multiple human three-dimensional key points; and inputting the preset format file into a preset engine to drive a virtual human model in the preset engine. Based on the pose estimation algorithm, the optimized target two-dimensional pose estimation model and target three-dimensional pose estimation model are used to extract the coordinates of multiple human three-dimensional key points in the video to be identified, which improves the speed and accuracy of pose estimation. Using the pose estimation algorithm to drive the virtual human model reduces the production cost and facilitates the expansion of the application scope. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0045] Figure 1 A flowchart illustrating a method for driving a virtual human, as provided in an embodiment of the present invention;

[0046] Figure 2 This is a schematic diagram illustrating the process of a target two-dimensional pose estimation model identifying a video to be identified, as provided in an embodiment of the present invention.

[0047] Figure 3 This is a schematic diagram illustrating module compression of the HRNet model provided in an embodiment of the present invention;

[0048] Figure 4 This is a schematic diagram illustrating the improvement of the residual structure of the HRNet model provided in an embodiment of the present invention;

[0049] Figure 5 This is a schematic diagram of the network structure of the three-dimensional pose estimation model provided in an embodiment of the present invention;

[0050] Figure 6 This is a schematic diagram of the network structure of the target three-dimensional pose estimation model provided in an embodiment of the present invention;

[0051] Figure 7 A schematic diagram illustrating the process of driving a virtual human as provided in an embodiment of the present invention;

[0052] Figure 8 This is a structural block diagram of a device for driving a virtual human, provided in an embodiment of the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0055] As the background technology indicates, with the development of virtual reality technology, virtual digital character driving technology has a very broad application prospect and a huge market space in many fields. Currently, the main methods for driving virtual digital characters are AI-driven and motion capture-driven. AI-driven methods are limited by specific scenarios and lack flexibility, while motion capture-driven methods specifically acquire motion data of real-world characters from a motion capture system and use this data to drive virtual characters on the same platform. However, motion capture-driven methods are costly to produce, making large-scale application difficult and limiting their scope of use.

[0056] Therefore, embodiments of the present invention provide a method, apparatus, and related equipment for driving a virtual human. A target two-dimensional pose estimation model is constructed based on a compressed two-dimensional pose estimation model, a preset convolutional block, and a preset dejittering method, reducing the computational load and improving the model's calculation speed. A refinement module is added to the three-dimensional pose estimation model to obtain the target three-dimensional pose estimation model, thereby improving the accuracy of the model's calculation results. Using a pose estimation algorithm to drive the virtual human model reduces production costs and facilitates wider application.

[0057] It should be noted that the method for driving the virtual human uses a pose estimation algorithm, which includes key technologies such as object detection, human skeletal key point detection, and segmentation. It mainly uses human information in a given image or video to locate and determine the skeletal key points of various parts of the human body, thereby estimating the human pose.

[0058] See Figure 1 The diagram illustrates a flowchart of a method for driving a virtual human according to an embodiment of the present invention. The method for driving a virtual human includes:

[0059] Step S101: Receive the video to be identified.

[0060] It should be noted that the video to be identified can be a full-body video of a single person.

[0061] Step S102: Use the target two-dimensional pose estimation model to identify the video to be identified and obtain the coordinates of multiple human two-dimensional key points.

[0062] In the specific implementation step S102, the video to be identified is input into the target two-dimensional pose estimation model for identification, and multiple human two-dimensional key point coordinates are obtained (such as 17 human two-dimensional key point coordinates).

[0063] It should be noted that, see Figure 2 The specific steps for identifying the video using a target 2D pose estimation model are as follows: A compressed 2D pose estimation model is invoked to extract features from the input video. A first pre-defined convolutional block is invoked to calculate the extracted features, yielding joint prediction results and coarse limb connection results for local features. This first pre-defined convolutional block consists of two 3x3 and one 1x1 convolutional block. Dilated convolutions are used to enhance the receptive field. The joint prediction results, coarse limb connection results, and the results after dilated convolutions are fused and input into a second pre-defined convolutional block for correction, resulting in a corrected version. The result is as follows: the second preset convolutional block consists of two 3*3 and one 1*1 convolutional blocks; the corrected result is then de-jittered using a preset de-jitter module to obtain the coordinates of multiple two-dimensional human body key points. The preset de-jitter module has two de-jitter methods: one is to use a low-pass filter, such as a Euro filter, which requires adjusting the two parameters of the low-pass filter, the minimum cutoff frequency and the velocity coefficient, until the two parameters reach a dynamic balance; the other is an exponential moving average, which uses a recursive algorithm to save only the calculation result of the previous frame each time, and the calculation result of each subsequent frame is calculated based on the previous frame.

[0064] It should be noted that the commonly used two-dimensional pose estimation model (HRNet model) has a relatively slow processing speed. The embodiments of the present invention construct a target two-dimensional pose estimation model (SFNet model) based on the two-dimensional pose estimation model (HRNet model), aiming to simplify the two-dimensional pose estimation model, reduce model parameters, and add a preset de-jitter method to improve the quality of model recognition results.

[0065] Specifically, a two-dimensional pose estimation model is obtained; the two-dimensional pose estimation model is compressed to obtain a compressed two-dimensional pose estimation model; and a target two-dimensional pose estimation model is constructed based on the compressed two-dimensional pose estimation model, a preset convolutional block, and a preset de-jittering method.

[0066] Understandably, the HRNet model consists of four stages. The first stage uses a high-resolution sub-network, and each subsequent stage incorporates a parallel low-resolution sub-network, fusing the high-resolution and low-resolution sub-networks. Each stage contains four block residual modules and one fusion module, such as... Figure 3 As shown, the number of block residual modules is reduced to two. Figure 4 As shown in Figure 401, the residual networks in stages 2 to 4 of the HRNet model are changed to bottleneck residual structures (e.g., ...). Figure 4 As shown in Figure 402, the batch normalization layer and activation function layer are placed before the convolutional layer to improve the network's generalization ability. This completes the compression of the two-dimensional pose estimation model, reducing the number of parameters in the HRNet model, thereby reducing the computational load and improving the model's computational speed.

[0067] Step S103: Input the coordinates of multiple human body 2D key points into the target 3D pose estimation model to obtain the coordinates of multiple human body 3D key points.

[0068] In the specific implementation step S103, the coordinates of multiple human body two-dimensional key points are elevated to three-dimensional space using the target three-dimensional pose estimation model, thereby obtaining the coordinates of multiple human body three-dimensional key points.

[0069] It should be noted that the target 3D pose estimation model (Re-Simplebaseline3D model) is pre-optimized based on the 3D pose estimation model (Simplebaseline3D model). Specifically, the 3D pose estimation model is obtained, where the network structure of the 3D pose estimation model is as follows: Figure 5 As shown, it consists of two residual blocks; a refinement module is added to the 3D pose estimation model, which consists of four fully connected network layers (its inputs are 2D and 3D keypoint coordinates, and its output is the optimized 3D keypoint coordinates), resulting in the target 3D pose estimation model (network structure as shown). Figure 6 (as shown), thereby improving the accuracy of the model's calculation results.

[0070] Step S104: Calculate the rotation angle of all frames in the video to be identified based on the coordinates of multiple human body 3D key points.

[0071] In the specific implementation step S104, the rotation matrix is ​​calculated based on the coordinates of multiple human body 3D key points; the rotation matrix is ​​converted into quaternions; the quaternions are converted into Euler angles to obtain the rotation angles of all frames in the video to be recognized.

[0072] It should be noted that the specific calculation process for calculating the rotation angle of all frames in the video to be identified based on the coordinates of multiple human body 3D key points is common knowledge to those in the field, and will not be elaborated here.

[0073] Step S105: Based on the parent-child relationship between multiple human body 3D key point coordinates, store the multiple human body 3D key point coordinates and the rotation angle of all frames in the video to be identified as a preset format file.

[0074] It should be noted that the default format file can be a BVH file, which consists of a header and a data section. The header begins with the keyword "HIERARCHY," followed by the root node, nodes under the root node "JOINT," and "CHANNEL," which specifies the number and names of channels. The data section begins with the keyword "MOTION," with each line representing the data for each frame. Data is stored sequentially according to the order of the head joints and the number of channels. This part includes the displacement of the root node and the rotation angles of each joint.

[0075] Step S106: Input the preset format file into the preset engine to drive the virtual human model in the preset engine.

[0076] In the specific implementation step S106, the preset format file (bvh file) is input into the preset engine (such as the Unreal Engine) through a preset plugin (such as the PerceptionNeuron plugin) to drive the virtual human model in the preset engine.

[0077] Specifically, import the 3D model (FBX model) into Unreal Engine, select the Skeletal Mesh in Unreal Engine, create an Animation Blueprint, and create a NewPoseCalc node (belonging to the Perception Neuron plugin) within the Animation Blueprint. Connect this node to Outputpose; perform bone repositioning through the NewPoseCalc node settings. Create a character Blueprint and add a motion capture component (PerceptionNeuron component) and a Skeletal Mesh component to it; in the Skeletal Mesh component settings, set the Anim Class to the aforementioned Animation Blueprint and the Skeletal Mesh to the imported model's Skeletal Mesh. Place the character Blueprint in the scene and set the BVH file as the BVH Asset of the motion capture component (PerceptionNeuron component).

[0078] In this embodiment of the invention, a target two-dimensional pose estimation model is constructed using a compressed two-dimensional pose estimation model, which simplifies the model, reduces the number of model parameters, and thus improves the speed of model recognition. The three-dimensional pose estimation model is optimized to obtain the target three-dimensional pose estimation model, thereby improving the accuracy of the model calculation results. The pose estimation algorithm is used to drive the virtual human, which effectively simplifies the steps of driving the virtual human, saves production costs, and makes driving the virtual human more flexible and convenient.

[0079] To better understand Figure 1 The method of driving virtual humans in the text will be explained and illustrated. See [link / reference]. Figure 7 The diagram illustrates the process of driving a virtual human according to an embodiment of the present invention.

[0080] Receive video 100; input video 100 into the target 2D pose estimation model (SFNet model) 200 to obtain the coordinates (x, y) of multiple human 2D keypoints corresponding to video 100; input the coordinates (x, y) of multiple human 2D keypoints into the target 3D pose estimation model (Re-Simplebaseline3D model) 300 to obtain the coordinates (x, y, z) of multiple human 3D keypoints corresponding to video 100; calculate the coordinates of multiple human 3D keypoints in data processing 400 to obtain the rotation angles of all frames corresponding to video 100, and store the coordinates of multiple human 3D keypoints and the rotation angles of all frames as a bvh file; use the bvh file as the input data source for the Perception Neuron plugin 500 to drive the virtual human model in the Unreal Engine 600.

[0081] In this embodiment of the invention, a pose estimation algorithm is used to drive the virtual human model, which reduces the production cost compared to wearable motion capture devices and makes driving the virtual human simpler and more convenient.

[0082] Corresponding to the method for driving a virtual human provided in the above embodiments of the present invention, see also... Figure 8 The diagram illustrates a structural block diagram of a device for driving a virtual human according to an embodiment of the present invention. The device includes: a receiving unit 801, a recognition unit 802, an input unit 803, a calculation unit 804, a storage unit 805, and a driving unit 806.

[0083] The receiving unit 801 is used to receive the video to be identified.

[0084] The recognition unit 802 is used to recognize the video to be recognized using the target two-dimensional pose estimation model, and obtain the coordinates of multiple human two-dimensional key points. The target two-dimensional pose estimation model is pre-built based on the two-dimensional pose estimation model.

[0085] The input unit 803 is used to input the coordinates of multiple human body two-dimensional key points into the target three-dimensional pose estimation model to obtain the coordinates of multiple human body three-dimensional key points. The target three-dimensional pose estimation model is pre-optimized based on the three-dimensional pose estimation model.

[0086] The calculation unit 804 is used to calculate the rotation angle of all frames in the video to be identified based on the coordinates of multiple human body 3D key points.

[0087] Storage unit 805 is used to store the coordinates of multiple human body 3D key points and the rotation angles of all frames in the video to be identified as a preset format file based on the parent-child relationship between the coordinates of multiple human body 3D key points.

[0088] The driving unit 806 is used to input a preset format file into the preset engine to drive the virtual human model in the preset engine.

[0089] In this embodiment of the invention, a pose estimation algorithm is used to drive the virtual human model, which reduces the production cost compared to wearable motion capture devices and makes driving the virtual human simpler and more convenient.

[0090] Preferred, combined Figure 8 As shown, the device for driving the virtual human also includes: a first acquisition unit, a compression unit, and a construction unit.

[0091] The first acquisition unit is used to acquire the two-dimensional attitude estimation model.

[0092] The compression unit is used to compress the two-dimensional attitude estimation model to obtain the compressed two-dimensional attitude estimation model.

[0093] The building unit is used to construct the target two-dimensional pose estimation model based on the compressed two-dimensional pose estimation model, the preset convolutional block, and the preset dejitter module.

[0094] Preferred, combined Figure 8 As shown, the device for driving the virtual human also includes a second acquisition unit and an addition unit.

[0095] The second acquisition unit is used to acquire the three-dimensional attitude estimation model.

[0096] Adding units is used to add refinement modules to the 3D attitude estimation model to obtain the target 3D attitude estimation model.

[0097] Preferred, combined Figure 8 The calculation unit 804, as shown, includes a calculation module, a first conversion module, and a second conversion module. The implementation principles of each module are as follows:

[0098] The calculation module is used to calculate the rotation matrix based on the coordinates of multiple 3D key points of the human body.

[0099] The first transformation module is used to convert rotation matrices into quaternions.

[0100] The second conversion module is used to convert quaternions into Euler angles to obtain the rotation angles of all frames in the video to be identified.

[0101] In summary, the embodiments of the present invention provide a method, apparatus, and related equipment for driving a virtual human. A target two-dimensional pose estimation model is constructed using a compressed two-dimensional pose estimation model, simplifying the model, reducing the number of model parameters, and thus improving the model recognition speed. The three-dimensional pose estimation model is optimized to obtain the target three-dimensional pose estimation model, thereby improving the accuracy of the model calculation results. The pose estimation algorithm is used to drive the virtual human, effectively simplifying the virtual human driving steps, saving production costs, and making the driving of the virtual human more flexible and convenient.

[0102] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0103] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0104] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method of driving a virtual person, characterized by, The method comprises: receiving a video to be identified; identifying the video to be identified by using a target two-dimensional pose estimation model to obtain a plurality of two-dimensional human key point coordinates, the target two-dimensional pose estimation model being previously constructed based on a two-dimensional pose estimation model; inputting the plurality of two-dimensional human key point coordinates into a target three-dimensional pose estimation model to obtain a plurality of three-dimensional human key point coordinates, the target three-dimensional pose estimation model being previously optimized based on a three-dimensional pose estimation model; calculating rotation angles of all frames in the video to be identified according to the plurality of three-dimensional human key point coordinates; storing the plurality of three-dimensional human key point coordinates and the rotation angles of all frames in the video to be identified as a preset format file according to a parent-child relationship among the plurality of three-dimensional human key point coordinates; inputting the preset format file into a preset engine to drive a virtual human model in the preset engine; wherein the process of constructing the target two-dimensional pose estimation model based on the two-dimensional pose estimation model comprises: obtaining a two-dimensional pose estimation model; halving the number of residual modules in the second to fourth stages of the two-dimensional pose estimation model and changing the residual modules into neck residual modules to obtain a preliminarily compressed two-dimensional pose estimation model; placing the batch normalization layer and the activation function layer of the preliminarily compressed two-dimensional pose estimation model before the convolution layer to obtain a compressed two-dimensional pose estimation model; constructing the target two-dimensional pose estimation model based on the compressed two-dimensional pose estimation model, a preset convolution block and a preset de-bouncing module.

2. The method of claim 1, wherein, The process of optimizing the target three-dimensional pose estimation model based on the three-dimensional pose estimation model comprises: obtaining a three-dimensional pose estimation model; adding a refining module to the three-dimensional pose estimation model to obtain the target three-dimensional pose estimation model.

3. The method of claim 1, wherein, The process of calculating the rotation angles of all frames in the video to be identified according to the plurality of three-dimensional human key point coordinates comprises: calculating a rotation matrix according to the plurality of three-dimensional human key point coordinates; converting the rotation matrix into a quaternion; converting the quaternion into Euler angles to obtain the rotation angles of all frames in the video to be identified.

4. An apparatus for driving a virtual person, characterized by comprising: The device comprises: a receiving unit configured to receive a video to be identified; an identifying unit configured to identify the video to be identified by using a target two-dimensional pose estimation model to obtain a plurality of two-dimensional human key point coordinates, the target two-dimensional pose estimation model being previously constructed based on a two-dimensional pose estimation model; an input unit configured to input the plurality of two-dimensional human key point coordinates into a target three-dimensional pose estimation model to obtain a plurality of three-dimensional human key point coordinates, the target three-dimensional pose estimation model being previously optimized based on a three-dimensional pose estimation model; a calculation unit configured to calculate rotation angles of all frames in the video to be identified according to the plurality of three-dimensional human key point coordinates; a storage unit configured to store the plurality of three-dimensional human key point coordinates and the rotation angles of all frames in the video to be identified as a preset format file according to a parent-child relationship among the plurality of three-dimensional human key point coordinates; a driving unit configured to input the preset format file into a preset engine to drive a virtual human model in the preset engine; wherein the device further comprises: The first obtaining unit is configured to obtain a two-dimensional pose estimation model; The compression unit is configured to halve the number of residual modules in the second to fourth stages of the two-dimensional pose estimation model and change the residual modules into neck residual modules to obtain a preliminarily compressed two-dimensional pose estimation model; and the compression unit is further configured to place batch normalization layers and activation function layers of the preliminarily compressed two-dimensional pose estimation model before convolution layers to obtain a compressed two-dimensional pose estimation model; The construction unit is configured to construct a target two-dimensional pose estimation model based on the compressed two-dimensional pose estimation model, a preset convolution block and a preset de-bouncing module.

5. The apparatus of claim 4, wherein, The device further includes: The second obtaining unit is configured to obtain a three-dimensional pose estimation model; The increasing unit is configured to add a refining module to the three-dimensional pose estimation model to obtain a target three-dimensional pose estimation model.

6. The apparatus of claim 4, wherein, The computing unit includes: The computing module is configured to calculate a rotation matrix according to the plurality of three-dimensional key point coordinates of the human body; The first conversion module is configured to convert the rotation matrix into a quaternion; The second conversion module is configured to convert the quaternion into Euler angles to obtain rotation angles of all frames in the video to be recognized.

7. An electronic device, comprising: The device includes: A processor and a memory connected through a communication bus; wherein the processor is configured to call and execute a program stored in the memory; The memory is configured to store a program, and the program is configured to implement the method for driving a virtual person according to any one of claims 1-3.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer executable instructions, and the computer executable instructions are used to execute the method for driving a virtual person according to any one of claims 1-3. The computer readable storage medium stores computer executable instructions, and the computer executable instructions are used to execute the method for driving a virtual person according to any one of claims 1-3.

Images