Target object animation method and related devices and equipment

Abstract

The present application discloses an animation method of a target object and related devices and equipment, wherein the animation method of the target object includes: obtaining a frontal photo of the target object and constructing a three-dimensional model of the target object based on the frontal photo; obtaining the data to be animated of the target object; determining the target features of the target object from the data to be animated; and constraining and aligning the features of the three-dimensional model of the target object with the target features in the data to be animated, so as to realize the animation of the data to be animated through the three-dimensional model. The above scheme can realize the animation of the target features of the target object based only on the frontal photo of the target object, simplifies the conditions for the animation of the target features, and improves the accuracy of the animation of the target features.

Description

Technical Field

[0001] The present application relates to the technical field of computer vision, and in particular to an animation method of a target object and related devices and equipment. Background Art

[0002] With the rapid development of Internet technology, the functions of Internet technology are becoming more and more diversified and efficient. Among them, users have an increasing demand for the diversification of applications in the process of using applications.

[0003] In the past, when animators created an expression animation, they had to manually design the corresponding animation based on the semantics of the expression, such as moving the target object based on experience. However, this process is a very tedious operation. For a long video, such as a video with tens of thousands of frames, it takes a lot of time and energy for animators to set the animation of the target object frame by frame. In addition, the animation of the target object manually designed by the animator cannot accurately reflect the actual movement law of the target object.

[0004] Therefore, the current way of animating target objects needs to be improved. Summary of the invention

[0005] The present application provides a target object animation method and related devices and equipment to solve the problem of complicated animation steps in the prior art.

[0006] The present application provides a method for animating a target object, the method comprising: obtaining a frontal photo of the target object, and constructing a three-dimensional model of the target object based on the frontal photo; obtaining data to be animated of the target object; determining target features of the target object from the data to be animated; and constraining and aligning features of the three-dimensional model of the target object with target features in the data to be animated, so as to realize animation of the data to be animated through the three-dimensional model.

[0007] Among them, after obtaining a frontal photo of the target object and constructing a three-dimensional model of the target object based on the frontal photo, the features of the three-dimensional model of the target object are constrained and aligned with the target features in the data to be animated, so as to realize the animation of the data to be animated through the three-dimensional model. Before the step of animating the data to be animated through the three-dimensional model, it also includes: adjusting the three-dimensional model to determine a preset number of standard expression bases of the target objects; the step of animating the data to be animated through the three-dimensional model includes: animating the data to be animated through the three-dimensional model based on the standard expression base.

[0008] Among them, the step of constraining and aligning the features of the three-dimensional model of the target object with the target features in the data to be animated to realize animation of the data to be animated through the three-dimensional model includes: rigidly transforming the three-dimensional model to obtain a transformed image corresponding to the target object in the data to be animated in the current frame; determining the features of the target object from the transformed image; adjusting the features based on the target features so that the features correspond to the target features in the same way; obtaining the expression base coefficients of the target object in the data to be animated in the current frame using the features in the adjusted three-dimensional model based on the standard expression base; and realizing animation of the data to be animated in the current frame based on the expression base coefficients.

[0009] Among them, the step of rigidly transforming the three-dimensional model to obtain a transformed image corresponding to the target object in the data to be animated in the current frame includes: translating and rotating the three-dimensional model of the target object, and projecting the three-dimensional model onto a two-dimensional plane to obtain a transformed image in the form of a two-dimensional plane; wherein the angle and position of the target object in the transformed image are the same as those of the target object in the data to be animated in the current frame.

[0010] Among them, the step of determining the features of the target object from the transformed image includes: determining two relative edge points of the target object and their coordinates from the transformed image; adding and equally dividing the coordinates of the two edge points to obtain the coordinates of the transformation center point; and performing image division on the transformed image to obtain the area that needs to be transformed in the transformed image; obtaining the pixel value of the transformed area based on the area that needs to be transformed in the transformed image; and determining the coordinates of the transformation center point and the pixel value of the transformed area as features.

[0011] Among them, the step of determining the target features of the target object from the data to be animated includes: determining the current center point coordinates and current area pixel values ​​of the target object from the data to be animated in the current frame, and using the current center point coordinates and current area pixel values ​​of the target object as the target features.

[0012] Among them, based on the target feature, the step of adjusting the feature so that the feature corresponds to the target feature includes: taking the current center point coordinates and the current area pixel values ​​as labels, and adjusting the transformation center point coordinates and the transformation area pixel values ​​through the L2 norm until the transformation center point coordinates and the transformation area pixel values ​​correspond to the current center point coordinates and the current area pixel values.

[0013] Among them, the step of animating the data to be animated in the current frame based on the expression base coefficients includes: importing the expression base coefficients into the three-dimensional virtual image corresponding to the three-dimensional model, so that the three-dimensional virtual image makes the same expression as the target object of the data to be animated in the current frame based on the expression base coefficients.

[0014] The present application also provides an animation device for a target object, the animation device for a target object comprising: a first acquisition module, used to acquire a frontal photo of the target object and construct a three-dimensional model of the target object based on the frontal photo; a second acquisition module, used to acquire data to be animated of the target object; a determination module, used to determine target features of the target object from the data to be animated; and a constraint module, used to constrain and align features of the three-dimensional model of the target object with target features in the data to be animated, so as to realize animation of the data to be animated through the three-dimensional model.

[0015] The present application also provides an electronic device, including a memory and a processor coupled to each other, wherein the processor is used to execute program instructions stored in the memory to implement any of the above-mentioned target object animation methods.

[0016] The present application also provides a computer-readable storage medium on which program instructions are stored. When the program instructions are executed by a processor, the animation method of any target object described above is implemented.

[0017] The above scheme, the present application constructs a three-dimensional model of the target object based on the frontal photo first, then obtains the data to be animated of the target object, determines the target features of the target object from the data to be animated, and constrains and aligns the features of the three-dimensional model of the target object with the target features in the data to be animated, so as to realize the animation of the data to be animated through the three-dimensional model. The present invention can realize the animation of the target features of the target object based only on the frontal photo of the target object, simplifies the conditions for animation of the target features, and improves the accuracy of animation of the target features. Moreover, by constraining and aligning the features of the three-dimensional model of the target object with the target features in the data to be animated, it avoids manual setting based on experience, improves the animation accuracy of the target object, and reflects the actual movement law of the target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flowchart of an embodiment of an animation method for a target object of the present application;

[0019] Figure 2 It is a flowchart of another embodiment of the animation method of the target object of the present application;

[0020] Figure 3 yes Figure 2 Schematic diagram of 9 standard expression bases in the embodiment;

[0021] Figure 4 It is a schematic diagram of a framework of an embodiment of an animation device of the target object of the present application;

[0022] Figure 5 It is a schematic diagram of the framework of an embodiment of the electronic device of the present application;

[0023] Figure 6 A schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0024] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0025] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0026] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of the associated objects. There can be three kinds of relationships. For example, A and / or B can be: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally means that the associated objects before and after are in an "or" relationship. In addition, "more" in this article means two or more than two.

[0027] See also Figure 1 , Figure 1 It is a flowchart of an embodiment of an animation method for a target object of the present application.

[0028] Step S11: obtaining a front view of the target object, and constructing a three-dimensional model of the target object based on the front view.

[0029] First, a frontal photo of the target object is obtained. The target object in this embodiment includes a human face, a human eye, a human hand, a human body or part of a human limb, an animal limb or the whole animal, and a movable object, etc. The specific target object is not limited here.

[0030] The front view of the target object is obtained by shooting based on the specific type of the target object. In a specific application scenario, when the target object is a human face, the front view of the target object can be obtained by shooting based on the front of the face, and the front view includes the facial contour and facial features of the face. In another specific application scenario, when the target object is a human eye, the front view of the target object can be obtained by shooting based on the front of the face or eye, and the front view includes the eye and eye socket. In another specific application scenario, when the target object is a human hand, the front view of the target object can be obtained by shooting based on the front of the hand, and the front view includes the palm and fingers.

[0031] The front view of the target object can be obtained by photographing the front view of the target object using a camera or a sensor.

[0032] After obtaining the front view of the target object, a three-dimensional model of the target object can be constructed based on the front view of the target object, wherein the three-dimensional model of the target object corresponds to the front view of the target object.

[0033] In a specific application scenario, when the target object is a human face, the 3D model is a 3D virtual model of the human face. In another specific application scenario, when the target object is a human hand, the 3D model is a 3D virtual model of the human hand.

[0034] Step S12: Acquire the data to be animated of the target object.

[0035] Obtain the data to be animated of the target object, wherein the data to be animated includes images to be animated and videos to be animated.

[0036] The data to be animated in this step can be obtained by taking images or recording videos of the target object using an ordinary camera, a camera built into a smart device, or other sensors.

[0037] Step S13: determining target features of the target object from the data to be animated.

[0038] Determine the target features of the target object from the data to be animated. The target features are feature data that need to be animated in the data to be animated. In a specific application scenario, when the target object is a human hand, the target features are the motion trajectories of various parts of the human hand and the texture data of various parts of the human hand.

[0039] When the data to be animated is a video to be animated, the video to be animated is divided into frames to obtain multiple frames of data to be animated, and the target features of the target object in each frame are determined from the data to be animated in each frame.

[0040] Step S14: Constraint-align the features of the three-dimensional model of the target object with the target features in the data to be animated, so as to realize animation of the data to be animated through the three-dimensional model.

[0041] The features corresponding to the target features in the target object in the three-dimensional model are constrained and aligned with the target features in the data to be animated, so that the three-dimensional model can simulate the target features in the data to be animated, and the target features in the data to be animated are animated using the three-dimensional model.

[0042] In a specific application scenario, the features of the 3D model of the target object can be constrained and aligned with the target features in the data to be animated through a loss function. In another specific application scenario, the features of the 3D model of the target object can also be constrained and aligned with the target features in the data to be animated through a deep neural network.

[0043] Through the above steps, the animation method of the target object of the present embodiment first constructs a three-dimensional model of the target object based on the frontal photo, then obtains the data to be animated of the target object, determines the target features of the target object from the data to be animated, and constrains and aligns the features of the three-dimensional model of the target object with the target features in the data to be animated, so as to realize the animation of the data to be animated through the three-dimensional model. The present embodiment can realize the animation of the target features of the target object based only on the frontal photo of the target object, simplifies the conditions for the animation of the target features, and improves the accuracy of the animation of the target features.

[0044] See also Figure 2 , Figure 2 1 is a flow chart of another embodiment of the target object animation method of the present application, wherein the target object in this embodiment is a human eye, and the eyeball movement and related texture of the human eye are animated.

[0045] Step S21: Obtain a frontal photo of the target object, construct a three-dimensional model based on the frontal photo, adjust the three-dimensional model, and determine a preset number of standard expression bases of the target object.

[0046] A frontal photo of the target object is obtained. If the target object in this embodiment is a human eye, a frontal photo of a human eye or a human face can be obtained. When the target object is photographed, the left and right eyeballs are centered, that is, in a natural state without any expression.

[0047] The three-dimensional model of the target object is constructed based on the frontal photo of the target object. The three-dimensional model of the target object can be constructed based on the frontal photo of the target object using a SfM (Structure from motion) three-dimensional reconstruction algorithm or other three-dimensional reconstruction algorithms.

[0048] Among them, since both eyeballs of the target object in the frontal photo are centered, both eyeballs of the three-dimensional model established based on this are also centered. By adjusting the two eyeballs in the three-dimensional model, the standard expression bases of a preset number of target objects are determined.

[0049] In a specific application scenario, there can be 9 standard expression bases. Specifically, the 9 standard expression bases are left eye centered and right eye centered, left eye centered and right eye upward, left eye centered and right eye downward, left eye centered and right eye to the left, left eye centered and right eye to the right, left eye upward and right eye centered, left eye downward and right eye centered, left eye to the left and right eye centered, left eye to the right and right eye centered.

[0050] Among them, the eyeballs upward, downward, leftward and rightward in the standard expression base in this embodiment are all extreme positions that the eyeballs can move to corresponding directions.

[0051] See also Figure 3 , Figure 3 yes Figure 2 Schematic diagram of 9 standard expression bases in the embodiment.

[0052] The 9 standard expression bases of this embodiment are standard expression base 1-left eye centered and right eye centered, standard expression base 2-left eye centered and right eye upward, standard expression base 3-left eye centered and right eye downward, standard expression base 4-left eye centered and right eye to the left, standard expression base 5-left eye centered and right eye to the right, standard expression base 6-left eye upward and right eye centered, standard expression base 7-left eye downward and right eye centered, standard expression base 8-left eye to the left and right eye centered, and standard expression base 9-left eye to the right and right eye centered.

[0053] Among them, the standard expression base 1 can be directly obtained through the three-dimensional model, and the other 8 standard expression bases can be obtained by moving the eyeballs in the standard expression base 1 in the three-dimensional model through artificial or deep neural networks, thereby obtaining a preset number of standard expression bases.

[0054] After establishing a preset number of standard expression bases, the corresponding expression base coefficient system can be constructed based on the standard expression bases at each extreme position. The range of the expression base coefficient is [0,1]. For example, when the right eye is centered, its corresponding expression base coefficient is 0, and when the right eye is upward and at an extreme position, its corresponding expression base coefficient is 1. When the right eye is upward but not at an extreme position, its corresponding expression base coefficient can be obtained based on the expression base coefficient system.

[0055] The preset number in this embodiment is 9, and in other embodiments, it can also be set to 17, 33, etc. according to the requirements of the standard expression base. The specific number is not limited here.

[0056] Step S22: Acquire the data to be animated of the target object.

[0057] This embodiment is described by taking the data to be animated as a video to be animated as an example. When the data to be animated is an image to be animated, the processing steps are similar to those of this embodiment and will not be repeated here.

[0058] The target object is recorded by video to obtain the data to be animated within a period of time.

[0059] Step S23: Determine the current center point coordinates and current area pixel values ​​of the target object from the data to be animated in the current frame.

[0060] The data to be animated is divided into frames to obtain multiple frames of data to be animated, and the target features of the target object of the current frame, i.e., the current center point coordinates and the current regional pixel values, are determined from the data to be animated in the current frame. The center point of this embodiment refers to the center point of the eyeball, and the regional pixel values ​​refer to the pixel values ​​of all pixels in the eye socket.

[0061] In a specific application scenario, the specific steps of determining the current center point coordinates of the target object and the pixel values ​​of the current area may include: determining the two edge points and their coordinates relative to each eyeball from the data to be animated in the current frame, and then adding and evenly dividing the coordinates of the two edge points to obtain the current center point coordinates. The two relative edge points and the center point are located on the same straight line. Then, the image is divided for the data to be animated in the current frame, and the target area in the data to be animated in the current frame, that is, the eye socket area, is obtained, and then the pixel value of the eye socket area is obtained to obtain the pixel value of the current area.

[0062] Among them, the method for determining the current center point coordinates of the target object is applicable when the target object is centrally symmetrical. When the target object is not centrally symmetrical, it can be determined by manually setting the center point or setting its center of gravity as the center point.

[0063] The image segmentation of the data to be animated in the current frame and the acquisition of the pixel value of the current area can be performed through segmentation and extraction by a deep neural network.

[0064] Among them, the center point of the target object can reflect the moving trajectory of the target object, and the regional pixel value can reflect the texture data such as RGB color and lighting of the target area.

[0065] Step S24: rigidly transform the three-dimensional model to obtain a transformed image corresponding to the target object in the data to be animated in the current frame, and determine the transformation center point coordinates and transformation area pixel values ​​of the target object from the transformed image.

[0066] Among them, since the data to be animated is two-dimensional data, and the three-dimensional model is three-dimensional data, it is necessary to convert the three-dimensional model so that the two correspond to each other.

[0067] Specifically, a rigid transformation is performed on the three-dimensional model to obtain a transformation image corresponding to the target object in the data to be animated in the current frame, and the transformation center point coordinates and transformation area pixel values ​​of the target object are determined from the transformation image.

[0068] In a specific application scenario, the three-dimensional model of the target object is translated and rotated so that the angle and position of the target object in the three-dimensional model correspond to the target object in the data to be animated in the current frame, and then the translated and rotated three-dimensional model is projected onto a two-dimensional plane to obtain a transformation image in the form of a two-dimensional plane. Among them, the angle and position of the target object in the transformation image are the same as the target object in the data to be animated in the current frame, so that it is convenient to constrain the features in the transformation image and the features of the target object in the data to be animated in the current frame.

[0069] Then determine the coordinates of the transformation center point and the pixel values ​​of the transformation area of ​​the target object from the transformation image. Specifically, first determine the two edge points and their coordinates of the target object from the transformation image, then add and evenly divide the coordinates of the two edge points to obtain the coordinates of the transformation center point; and perform image division on the transformation image to obtain the area that needs to be transformed in the transformation image; obtain the pixel values ​​of the transformation area based on the area that needs to be transformed in the transformation image. Among them, the specific method of determining the coordinates of the transformation center point and the pixel values ​​of the transformation area of ​​the target object from the transformation image is the same as the corresponding method of determining the current center point coordinates and the current area pixel values ​​of the target object from the data to be animated in the current frame in the aforementioned steps.

[0070] Step S25: Based on the current center point coordinates and the current region pixel values, the transformation center point coordinates and the transformation region pixel values ​​are adjusted so that the transformation center point coordinates and the transformation region pixel values ​​correspond to the current center point coordinates and the current region pixel values.

[0071] Based on the current center point coordinates and the current area pixel values, the transformation center point coordinates and the transformation area pixel values ​​in the transformation image are adjusted so that the transformation center point coordinates and the transformation area pixel values ​​correspond to the current center point coordinates and the current area pixel values.

[0072] Specifically, the current center point coordinates and the current area pixel values ​​are used as labels, and the transformation center point coordinates and the transformation area pixel values ​​are adjusted and constrained through the L2 norm until the transformation center point coordinates and the transformation area pixel values ​​correspond to the current center point coordinates and the current area pixel values, that is, the constraints between the transformation center point coordinates and the transformation area pixel values ​​and the current center point coordinates and the current area pixel values ​​converge.

[0073] In a specific application scenario, the current center point coordinates can be used as labels, and the coordinate difference between the current center point coordinates and the transformed center point coordinates can be calculated for learning, and the coordinates converge when the coordinate difference meets the preset difference. The preset difference can be 0 or other values, which are set based on actual needs. When the preset difference is 0, the animation accuracy and authenticity are the highest.

[0074] In a specific application scenario, the pixel value of the current region can be used as a label, the transformed image can be rendered, and then the pixel difference between the pixel value of the current region and the pixel value of the rendered transformed region can be calculated for learning, and the pixels converge when the pixel difference meets the preset pixel difference. The preset pixel difference can be 0 or other values, which are set based on actual needs. When the preset pixel difference is 0, the animation accuracy and authenticity are the highest.

[0075] In other embodiments, other loss functions, such as edit distance, etc., may be used to constrain the current center point coordinates and the transformed center point coordinates, and the current region pixel values ​​and the transformed region pixel values.

[0076] Step S26: Based on the standard expression base, the expression base coefficients of the target object in the data to be animated in the current frame are obtained by using the coordinates of the transformation center point and the pixel values ​​of the transformation area in the adjusted three-dimensional model, and animation of the data to be animated in the current frame is realized based on the expression base coefficients.

[0077] After the constraints between the transformation center point coordinates and the transformation area pixel values ​​and the current center point coordinates and the current area pixel values ​​converge, the transformation center point coordinates and the transformation area pixel values ​​in the adjusted three-dimensional model are used to obtain the expression base coefficients of the target object in the current frame of the data to be animated based on the standard expression base, and then the animation of the current frame of the data to be animated is realized based on the expression base coefficients. Specifically, each expression base coefficient can be obtained by the expression base coefficient system of a preset number of standard expression bases.

[0078] In a specific application scenario, the expression base coefficients can be imported into the three-dimensional virtual image corresponding to the three-dimensional model, so that the three-dimensional virtual image makes the same expression as the target object of the data to be animated in the current frame based on the expression base coefficients. The three-dimensional virtual image and the three-dimensional model can be constructed in the same way to facilitate the import of the expression base coefficients, and the three-dimensional virtual image can be a three-dimensional virtual image of any image, such as a kitten, a puppy, etc., which is not specifically limited here.

[0079] By performing the above processing on each frame of the data to be animated in turn, the linearized multi-frame expression base coefficients can be obtained, so as to realize the feature simulation of the target object in the data to be animated through the three-dimensional virtual image, truly restore the motion trajectory of the target object, and then realize the animation of the target object in the data to be animated.

[0080] Through the above steps, the animation method of the target object of the present embodiment constructs a three-dimensional model based on the frontal photo, determines a preset number of standard expression bases based on the three-dimensional model, obtains the target object's data to be animated, determines the current center point coordinates and the current area pixel values ​​of the target object from the data to be animated in the current frame, performs a rigid transformation on the three-dimensional model, obtains a transformation image corresponding to the target object in the data to be animated in the current frame, thereby improving the constraint efficiency, and then constrains it with the current center point coordinates and the current area pixel values, so that the transformation center point coordinates and the transformation area pixel values ​​correspond to the current center point coordinates and the current area pixel values, and finally uses the transformation center point coordinates and the transformation area pixel values ​​in the adjusted three-dimensional model to obtain the expression base coefficients of the target object in the data to be animated in the current frame, and realizes the animation of the data to be animated in the current frame based on the expression base coefficients. In the present embodiment, by making the transformation center point coordinates and the transformation area pixel values ​​of the transformation image correspond to the current center point coordinates and the current area pixel values, the corresponding expression base coefficients are obtained, and finally animation is performed based on the expression base coefficients, so that the animation data can be converted into data to improve the precision and accuracy of the animation. Furthermore, this embodiment can realize the animation of the target features of the target object based only on the front view of the target object, thereby simplifying the conditions for the animation of the target features and improving the accuracy of the animation of the target features.

[0081] See also Figure 4 , Figure 4 It is a schematic diagram of the framework of an embodiment of an animation device for a target object of the present application. The animation device 40 for a target object includes a first acquisition module 41, a second acquisition module 42, a determination module 43, and a constraint module 44. The first acquisition module 41 is used to acquire a frontal photo of the target object and construct a three-dimensional model of the target object based on the frontal photo; the second acquisition module 42 is used to acquire the data to be animated of the target object; the determination module 43 is used to determine the target features of the target object from the data to be animated; the constraint module 44 is used to constrain and align the features of the three-dimensional model of the target object with the target features in the data to be animated, so as to realize the animation of the data to be animated through the three-dimensional model.

[0082] The first acquisition module 41 is also used to adjust the three-dimensional model and determine a preset number of standard expression bases of target objects. The constraint module 44 is also used to animate the data to be animated through the three-dimensional model based on the standard expression bases.

[0083] The constraint module 44 is also used to perform a rigid transformation on the three-dimensional model to obtain a transformed image corresponding to the target object in the data to be animated in the current frame; determine the characteristics of the target object from the transformed image; adjust the characteristics based on the target characteristics so that the characteristics correspond to the target characteristics; obtain the expression base coefficient of the target object in the data to be animated in the current frame based on the standard expression base using the characteristics in the adjusted three-dimensional model; and realize the animation of the data to be animated in the current frame based on the expression base coefficient.

[0084] The constraint module 44 is also used to translate and rotate the three-dimensional model of the target object, and project the three-dimensional model onto a two-dimensional plane to obtain a transformed image in the form of a two-dimensional plane; wherein the angle and position of the target object in the transformed image are the same as those of the target object in the data to be animated in the current frame.

[0085] The constraint module 44 is also used to determine the characteristics of the target object from the transformed image, including the following steps: determining two relative edge points of the target object and their coordinates from the transformed image; adding and equally dividing the coordinates of the two edge points to obtain the coordinates of the transformation center point; and performing image division on the transformed image to obtain the area in the transformed image that needs to be transformed; and obtaining the pixel value of the transformed area based on the area in the transformed image that needs to be transformed.

[0086] The determination module 43 is further used to determine the current center point coordinates and current region pixel values ​​of the target object from the data to be animated in the current frame, and use the current center point coordinates and current region pixel values ​​of the target object as target features.

[0087] The constraint module 44 is also used to use the current center point coordinates and the current area pixel values ​​as labels, and adjust the transformation center point coordinates and the transformation area pixel values ​​through the L2 norm until the transformation center point coordinates and the transformation area pixel values ​​correspond to the current center point coordinates and the current area pixel values.

[0088] The constraint module 44 is also used to import the expression base coefficients into the three-dimensional virtual image corresponding to the three-dimensional model, so that the three-dimensional virtual image makes the same expression as the target object of the data to be animated in the current frame based on the expression base coefficients.

[0089] The above scheme can realize the animation of the target features of the target object based only on the front view of the target object, simplifies the conditions for the animation of the target features, and improves the accuracy of the animation of the target features.

[0090] See also Figure 5 , Figure 51 is a schematic diagram of a framework of an embodiment of an electronic device of the present application. The electronic device 50 includes a memory 51 and a processor 52 coupled to each other, and the processor 52 is used to execute program instructions stored in the memory 51 to implement the steps of the target object animation method of any of the above embodiments. In a specific implementation scenario, the electronic device 50 may include but is not limited to: a microcomputer, a server, and in addition, the electronic device 50 may also include a mobile device such as a laptop computer and a tablet computer, which is not limited here.

[0091] Specifically, the processor 52 is used to control itself and the memory 51 to implement the steps of the above-mentioned animation method embodiment of any target object. The processor 52 can also be called a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip with signal processing capabilities. The processor 52 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 52 can be implemented by an integrated circuit chip.

[0092] The above scheme can realize the animation of the target features of the target object based only on the front view of the target object, simplifies the conditions for the animation of the target features, and improves the accuracy of the animation of the target features.

[0093] See also Figure 6 , Figure 6 The computer-readable storage medium 60 stores program instructions 601 that can be executed by a processor, and the program instructions 601 are used to implement the steps of the target object animation method of any of the above embodiments.

[0094] The above scheme can realize the animation of the target features of the target object based only on the front view of the target object, simplifies the conditions for the animation of the target features, and improves the accuracy of the animation of the target features.

[0095] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0096] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0097] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0098] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

Claims

1. A method for animating a target object, characterized in that: The animation method of the target object includes: Acquire a front view of the target object, and construct a three-dimensional model of the target object based on the front view; Acquire the data to be animated of the target object; Determining a target feature of the target object from the data to be animated; Constraining and aligning the features of the three-dimensional model of the target object with the target features in the data to be animated, so as to realize animation of the data to be animated through the three-dimensional model; The step of constraining and aligning the features of the three-dimensional model of the target object with the target features in the data to be animated so as to animate the data to be animated through the three-dimensional model comprises: Perform a rigid transformation on the three-dimensional model to obtain a transformation image corresponding to the target object in the data to be animated in the current frame; determine the characteristics of the target object from the transformation image; adjust the characteristics based on the target characteristics so that the characteristics correspond to the target characteristics; obtain the expression base coefficient of the target object in the data to be animated in the current frame based on the standard expression base using the characteristics in the adjusted three-dimensional model; and animate the data to be animated in the current frame based on the expression base coefficient.

2. The target object animation method according to claim 1, characterized in that: After the step of acquiring a frontal photo of the target object and constructing a three-dimensional model of the target object based on the frontal photo, before the step of constraining and aligning features of the three-dimensional model of the target object with target features in the data to be animated so as to animate the data to be animated through the three-dimensional model, the method further includes: Adjusting the three-dimensional model to determine a preset number of standard expression bases of the target object; The step of animating the data to be animated by using the three-dimensional model comprises: The animation of the data to be animated is realized through the three-dimensional model based on the standard expression base.

3. The target object animation method according to claim 1, characterized in that: The step of rigidly transforming the three-dimensional model to obtain a transformed image corresponding to the target object in the data to be animated in the current frame comprises: The three-dimensional model of the target object is translated and rotated, and the three-dimensional model is projected onto a two-dimensional plane to obtain the transformed image in a two-dimensional plane form; wherein the angle and position of the target object in the transformed image are the same as those of the target object in the data to be animated in the current frame.

4. The target object animation method according to claim 1, characterized in that: The step of determining the features of the target object from the transformed image comprises: Determine two opposite edge points of the target object and their coordinates from the transformed image; Adding and evenly dividing the coordinates of the two edge points to obtain the coordinates of the transformation center point; and Performing image segmentation on the transformed image to obtain an area in the transformed image that needs to be transformed; Acquiring pixel values ​​of a transformed region based on a region to be transformed in the transformed image; The transformation center point coordinates and the transformation area pixel values ​​are determined as the features.

5. The target object animation method according to claim 4, characterized in that: The step of determining the target feature of the target object from the data to be animated comprises: The current center point coordinates and current area pixel values ​​of the target object are determined from the data to be animated in the current frame, and the current center point coordinates and current area pixel values ​​of the target object are used as the target features.

6. The target object animation method according to claim 5, characterized in that: The step of adjusting the feature based on the target feature so that the feature corresponds to the same target feature includes: The current center point coordinates and the current area pixel values ​​are used as labels, and the transformed center point coordinates and the transformed area pixel values ​​are adjusted and constrained by the L2 norm until the transformed center point coordinates and the transformed area pixel values ​​correspond to the current center point coordinates and the current area pixel values.

7. The target object animation method according to claim 1, characterized in that: The step of animating the data to be animated in the current frame based on the expression base coefficient comprises: The expression base coefficients are imported into a three-dimensional virtual image corresponding to the three-dimensional model, so that the three-dimensional virtual image makes the same expression as the target object of the data to be animated in the current frame based on the expression base coefficients.

8. An animation device for a target object, characterized in that: The animation device of the target object comprises: A first acquisition module, used to acquire a front view of a target object and construct a three-dimensional model of the target object based on the front view; A second acquisition module is used to acquire the to-be-animated data of the target object; A determination module, used for determining a target feature of the target object from the data to be animated; A constraint module is used to constrain and align the features of the three-dimensional model of the target object with the target features in the data to be animated, so as to realize animation of the data to be animated through the three-dimensional model, wherein the three-dimensional model is rigidly transformed to obtain a transformed image corresponding to the target object in the data to be animated in the current frame; the features of the target object are determined from the transformed image; based on the target features, the features are adjusted so that the features correspond to the target features in the same way; based on the standard expression base, the expression base coefficients of the target object in the data to be animated in the current frame are obtained using the features in the adjusted three-dimensional model; and animation of the data to be animated in the current frame is realized based on the expression base coefficients.

9. An electronic device, characterized in that: It comprises a memory and a processor coupled to each other, wherein the processor is used to execute program instructions stored in the memory to implement the target object animation method according to any one of claims 1 to 7.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the target object animation method according to any one of claims 1 to 7 is implemented.

Images