Virtual puppet generation method and system for animation image

Abstract

The invention discloses a virtual puppet generation method and system for an animation image. The method comprises the following steps: acquiring an image of a pre-drawn role; acquiring a joint relationship corresponding to the role, wherein the joint relationship is used for indicating the position of a joint on the role when the role is a puppet; marking each joint on the role on an image corresponding to the role according to the joint relationship; and generating a virtual puppet according to each joint of the role and the image, generating a virtual three-dimensional image according to the position of each joint, and mapping the image of the role to the three-dimensional image to generate the virtual puppet. According to the method and the device, the problem that the shooting efficiency is influenced due to the fact that a lot of time and a complicated process are needed to manufacture a role when the puppet animation is shot in the related technology is solved, and then the virtual puppet can be generated according to the pre-drawn role image, so that the shooting efficiency is improved to a certain extent.

Description

Technical Field

[0001] This application relates to the field of image processing. Specifically, it relates to a method and system for generating virtual puppets for animated images. Background Art

[0002] Puppet animation is an animation style developed on the basis of puppet shows. Different from traditional two-dimensional hand-drawn animations and three-dimensional computer animations, it has its own unique three-dimensional space. When shooting, the continuous movements of the puppet are first decomposed into successive fixed postures, and then frame-by-frame shooting is carried out to make the puppet move on the screen.

[0003] The production process of puppet animation involves a variety of technologies and techniques. First of all, the production of puppets itself requires exquisite handicraft skills, including the design, carving, assembly, and painting of puppets. Usually, the characters of puppets are mainly made of wood, supplemented by rubber, plastic, sponge, and wire, etc. All movable parts (including eyes and mouths) are made of silver wire or metal joints for easy manipulation.

[0004] Producing multiple puppets is required for shooting animated cartoons. This way of making puppets and then shooting is time-consuming and laborious, and the efficiency is relatively low. Summary of the Invention

[0005] The embodiments of this application provide a method and system for generating virtual puppets for animated images, so as to at least solve the problem in the related art that a large amount of time and complicated processes are required to produce characters when shooting puppet animations, thus affecting the shooting efficiency.

[0006] According to one aspect of this application, a method for generating virtual puppets for animated images is provided. The method is applied in software, and the software is used to execute the method. The method includes the following steps: obtaining an image of a pre-drawn character, where the character includes at least one of the following: human, animal; obtaining a joint relationship corresponding to the character, where the joint relationship is pre-configured and is used to indicate the positions of the joints of the character when it is a puppet on the character; marking each joint on the character on the image corresponding to the character according to the joint relationship; generating a virtual puppet according to each joint of the character and the image, where a virtual three-dimensional image is generated according to the position of each joint, and the image of the character is mapped onto the three-dimensional image to generate the virtual puppet.

[0007] Further, the three-dimensional image is divided into different parts according to the positions of each joint of the character; the image is split into pictures corresponding to the parts according to the positions of each joint; the pictures corresponding to each part are attached to the corresponding part and rendered to obtain the virtual puppet.

[0008] Further, it further includes: obtaining a three-dimensional image of a pre-drawn character, wherein the three-dimensional image displays the character from multiple angles; or, obtaining multiple two-dimensional images of the pre-drawn character, wherein each two-dimensional image is of the character drawn from a different angle, and generating a three-dimensional image of the character based on the multiple two-dimensional images.

[0009] Further, according to the correspondence between the three-dimensional image and the three-dimensional image, attaching the three-dimensional image to the three-dimensional image.

[0010] According to another aspect of the present application, there is also provided a system for generating virtual puppets for animated images, which is applied to software. The software includes the following modules: an acquisition module for acquiring a pre-drawn image of a character; wherein the character includes at least one of the following: a person, an animal; a determination module for obtaining a joint relationship corresponding to the character, wherein the joint relationship is pre-configured and is used to indicate the position of the joints of the character when the character is a puppet on the character; a display module for displaying the joint relationship and marking each joint on the character on the image corresponding to the character; a processing module for generating a virtual puppet from each joint of the character and the image, wherein a virtual three-dimensional image is generated according to the position of each joint, and the image of the character is mapped onto the three-dimensional image to generate the virtual puppet; Further, the processing module is used for: dividing the three-dimensional image into different parts according to the position of each joint of the character; splitting the image into pictures corresponding to the parts according to the position of each joint; attaching the pictures corresponding to each part to the corresponding part and performing rendering to obtain the virtual puppet.

[0011] Further, the acquisition module is used for: obtaining a three-dimensional image of a pre-drawn character, wherein the three-dimensional image displays the character from multiple angles; or, obtaining multiple two-dimensional images of the pre-drawn character, wherein each two-dimensional image is of the character drawn from a different angle, and generating a three-dimensional image of the character based on the multiple two-dimensional images.

[0012] Further, the determination module is further used for: attaching the three-dimensional image to the three-dimensional image according to the correspondence between the three-dimensional image and the three-dimensional image.

[0013] According to another aspect of the present application, there is also provided an electronic device, including a memory and a processor; wherein, the memory is used for storing one or more computer instructions, and the one or more computer instructions are executed by the processor to implement the above method steps.

[0014] According to another aspect of the present application, there is also provided a readable storage medium storing computer instructions thereon, wherein when the computer instructions are executed by a processor, the above method steps are implemented.

[0015] In an embodiment of the present application, an image of a pre-drawn character is acquired, wherein the character includes at least one of the following: a person, an animal; a joint relationship corresponding to the character is acquired, wherein the joint relationship is pre-configured and is used to indicate the positions of the joints on the character when the character is a puppet; each joint on the character is marked on the image corresponding to the character according to the joint relationship; a virtual puppet is generated according to each joint of the character and the image, wherein a virtual three-dimensional image is generated according to the position of each joint, and the image of the character is mapped onto the three-dimensional image to generate the virtual puppet. By means of the present application, the problem in the related art that a large amount of time and complicated processes are required to produce a character when shooting a puppet animation, thus affecting the shooting efficiency, is solved. Furthermore, a virtual puppet can be generated according to a pre-drawn character image, thereby improving the efficiency to a certain extent for shooting. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The schematic embodiments and descriptions thereof of the present application are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings: Figure 1 is a flowchart of a method for generating a virtual puppet for an animated image according to an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the drawings and in conjunction with the embodiments.

[0018] It should be noted that the steps shown in the flowchart of the drawings may be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than here.

[0019] Producing puppet animations requires making puppets. For example, the following method can be used for production: When assembling humanoid or animal puppets according to one's own wishes, one can combine and install pre-prepared bones, artificial skin, artificial muscles, a face, hands, and feet according to one's preferences to create the desired body shape. The replicated puppet consists of a face, hands, and feet, and includes a plurality of connecting components that connect the face, hands, and feet together to form bones; the connecting components are detachably connected to each other, and the face, hands, feet, and the connecting components are detachably connected to form a bent portion at the joint of the bones; and an artificial skin that can be separately covered on the connecting components having the bent portion to form the arms, legs, and torso. A body shape-adjusting artificial muscle can be detachably installed on the arms and legs of the connecting components. By installing the artificial muscle, when assembling the desired puppet, one can create a muscular body shape, a fat body shape, and a thin body shape according to one's preferences, and cover the body shape-adjusting artificial muscle with the artificial skin. The body shape-adjusting artificial muscle is formed of an elastic and soft material into a hollow cylinder with open ends, is applicable to the arms and legs, and has a cut portion formed along the longitudinal direction for easy installation on the arms and legs. Embedding holes for detachably embedding eyebrows, a nose, eyes, and a mouth are provided on the face. The bent portion includes spherical portions formed at the ends of the face, hands, and feet and at both ends of the connecting components, and connecting components having receiving grooves formed at both ends thereof for detachably and slidably receiving the spherical portions. The artificial skin is made of an elastic and soft material such as silicon. The artificial skin is formed in any color such as white, yellow, and black. The hand consists of fingers formed by bent portions that are detachably connected between the connecting components and the connecting components, and a palm connected by the fingers. The surfaces of the fingers and the palm are covered with the artificial skin. The foot consists of toes formed by bent portions that are detachably connected between the connecting components and the connecting components, and a sole connected by the toes. The surfaces of the toes and the sole are covered with the artificial skin. The above can produce others or oneself as puppets. Moreover, by covering the bones with artificial skin, a very realistic and lifelike feeling can be obtained. In addition, by adjusting the length of the joints, it is possible to simply create puppets resembling adults or children.

[0020] Although the above puppets can be replicated, it is still time-consuming and laborious.

[0021] In this embodiment, a virtual puppet can be generated according to a pre-drawn character image. This method can be called a virtual puppet generation method for animated images. Figure 1 It is a flowchart of a virtual puppet generation method for animated images according to an embodiment of the present application, as Figure 1 shown. The steps involved in the process in Figure 1 will be described below.

[0022] Step S102: Obtain the pre-drawn image of the character, where the character includes at least one of the following: human, animal.

[0023] As an optional implementation, a three-dimensional image of the pre-drawn character can be obtained, where the three-dimensional image displays the character from multiple angles; or, multiple two-dimensional images of the pre-drawn character can be obtained, where each two-dimensional image is of the character drawn from a different angle, and a three-dimensional image of the character is generated based on the multiple two-dimensional images. According to the correspondence between the three-dimensional image and the three-dimensional image, attach the three-dimensional image to the three-dimensional image.

[0024] In this step, if the image or picture content is relatively complex, for example, there is a complex background in addition to the puppet, the following steps can be used for processing. The mobile phone scans the entire picture to be processed, and based on the scanning result, selects the area in the picture that matches the area that can be a puppet saved in the mobile phone database as the area that can be a puppet in the picture and saves it; the mobile phone reads the focus of the user operation and compares the focus with the area that can be a puppet in the saved picture. When the focus is within the area that can be a puppet in the picture, perform edge tracing display on the edge of this area; the mobile phone forms all the picture content within the area with edge tracing display into a new layer; after the mobile phone reads and completes the user's operation on the new layer and saves the operation result in the new layer, synthesize the new layer with the picture. For landscape pictures, the area that can be a puppet saved in the mobile phone database is all areas. For human pictures, the area that can be a puppet saved in the mobile phone database is: the areas corresponding to each movable part divided according to human anatomy. After the mobile phone compares the focus with the area that can be a puppet in the saved picture, when it is determined that the focus is not within the area that can be a puppet in the picture, prompt the user that the current area is not operable. The operations on the new layer include: one or any combination of rotation of the new layer content, change of size, change of transparency, and displacement operation. After the new layer and the picture are synthesized, use the pixels around the new layer to supplement the blank parts in the synthesized picture.

[0025] Step S104: Obtain the joint relationship corresponding to the character, where the joint relationship is pre-configured and is used to indicate the positions of the joints of the character when the character is a puppet on the character.

[0026] Step S106: Mark each joint on the character on the image corresponding to the character according to the joint relationship.

[0027] Step S108: Generate a virtual puppet based on each joint of the character and the image, where a virtual three-dimensional image is generated according to the position of each joint.

[0028] As an optional implementation manner, the virtual puppet can be generated according to each joint of the character and the image, where the three-dimensional image is divided into different parts according to the position of each joint of the character; the image is split into pictures corresponding to the parts according to the position of each joint; the pictures corresponding to each part are attached to the corresponding part and rendered to obtain the virtual puppet.

[0029] Step S112: Map the image of the character onto the three-dimensional image to generate the virtual puppet.

[0030] In an optional implementation manner, obtain multiple pictures of at least one puppet from the already taken puppet images; for each puppet, extract the parts that have undergone relative position changes in the pictures of the puppet according to different actions of the puppet in the multiple pictures, where the relative position change is the position change of each part of the puppet relative to the puppet body; determine the joints on the puppet according to the parts that have undergone the relative position change; save the relative position relationship of each joint on the puppet on the puppet, where the relative position relationship is used as the joint relationship. Obtain multiple puppets of the same type as the character and having the joint relationship saved; obtain the text description of the character, and obtain the puppet closest to the character among the multiple puppets according to the text description; use the joint relationship of the closest puppet as the corresponding relationship with the character.

[0031] In an optional implementation manner, after the number of groups of saved training data exceeds the threshold, input the saved multiple groups of training data into a neural network model for training, where the multiple pictures are used as the input data of the neural network model, and the joint relationship is used as the output data of the neural network model; after the neural network model training converges, obtain the trained neural network model. Input the pre-drawn image of the character into the trained neural network model to obtain the joint relationship output by the neural network model.

[0032] Through the above implementation manner, the problem in the related art that a large amount of time and complicated processes are required to produce a character when shooting puppet animations, thus affecting the shooting efficiency, is solved.

[0033] In the above implementation manner, a neural network model is used. The construction of the neural network model will be described below.

[0034] A neural network model can be two neural network models. For example, a first neural network model and a second neural network model can be trained. The weights in the second model are set based on the corresponding weights in the first model; the second model is trained on a first dataset, where the training includes updating the weights in the second model; and the corresponding weights in the first model are adjusted based on the updated weights in the second model. As described above, when training two neural network models to classify similar types of data (e.g., the same type of images), even if the outputs of the models are different (e.g., one model can be trained to detect the presence of a specific object in an image, while the other model is trained to measure the length of a specific type of object in the image), the weights of some layers of the models are usually very similar and can even converge to the same values in the case of training the models on a database of sufficiently large training data. Therefore, if the second model has been trained, the updated weights from that training can be used to improve the weights of the first model without applying any other training directly to the first model. In this way, the repeated training of similar models can be reduced, thereby making the training process more efficient, and thus making the training converge (e.g., making the weights of the model move towards the optimal value for each weight) much faster than other possible cases. Additionally, less training data is required for each model (e.g., the remote database used to train the second model does not have to be directly used for the first model) and computing power is saved because only one of the two models has to process each new batch of training data, rather than each model having to independently process each new batch of training data.

[0035] An artificial neural network (or simply referred to as a neural network) is familiar to those skilled in the art. Briefly, a neural network is a model that can be used to classify data (e.g., classify or identify the content of image data). The structure of a neural network is inspired by the human brain. A neural network includes various layers, and each layer includes multiple neurons. Each neuron includes mathematical operations. During the process of classifying a portion of data, the mathematical operations of each neuron are performed on that portion of data to produce a numerical output, and the output of each layer in the neural network is sequentially fed into the next layer. Typically, the mathematical operations associated with each neuron include one or more weights that are tuned during the training process (e.g., updating the values of the weights during the training process to tune the model to produce more accurate classifications). For example, in a neural network model used to classify the content of an image, each neuron in the neural network can include a mathematical operation that includes a weighted linear sum of pixel (or voxel in three dimensions) values in the image, followed by a non-linear transformation. Examples of non-linear transformations used in neural networks include the sigmoid function, the hyperbolic tangent function, and the rectified linear function. Neurons in each layer of a neural network typically include different weighted combinations of a single type of transformation (e.g., the same type of transformation, such as the sigmoid function, but with different weights).

[0036] In some layers, each neuron can apply the same weights in a linear sum; this applies, for example, in the case of convolutional layers. The weights associated with each neuron may cause certain features to be more dominant (or conversely less dominant) than other features during the classification process. Thus, adjusting the weights of the neurons during training trains the neural network to increase the salience of specific features when classifying images. Generally, a neural network can have weights associated with neurons and / or weights between neurons (which, for example, modify the data values passed between neurons). As described above, in some neural networks (e.g., convolutional neural networks), the lower layers (e.g., the input layer or hidden layer in a neural network) (i.e., the layers towards the beginning of a series of layers in a neural network) are activated by small features or patterns in that portion of the data (i.e., the output of the lower layers depends on the small features or patterns in that portion of the data), while the higher layers (i.e., the layers towards the end of a series of layers in a neural network) are activated by increasingly larger features in that portion of the data being classified. As an example, in the case where the data includes an image, the lower layers in the neural network are activated by small features (e.g., edge patterns in the image), the middle layers are activated by features such as larger shapes and forms in the image, and the layers closest to the output (e.g., the upper layers) are activated by the entire object in the image. Generally, the weights of the last layer (referred to as the output layer) of a neural network model depend most strongly on the specific classification problem that the neural network is solving. For example, the weights of the outer layer may strongly depend on whether the classification problem is a localization problem or a detection problem. The weights of the lower layers (e.g., the input layer and / or hidden layer) tend to depend on the content (e.g., features) of the data being classified. Thus, the present embodiment has recognized that, with sufficient training, the weights in the input layer and hidden layer of neural networks that process the same type of data can converge towards the same values over time, even if the outer layers of the model are tuned to solve different classification problems.

[0037] Generally, the systems and methods described in this embodiment relate to training a first neural network model and a second neural network model (referred to herein as the first model and the second model, respectively). The first model and the second model can include weights (e.g., parameters) that are updated (e.g., adjusted) as part of the training process of the first model and the second model. Generally, the first neural network model and the second neural network model can include feedforward models (e.g., convolutional neural networks, autoencoder neural network models, probabilistic neural network models, and time-delay neural network models), radial basis function network models, recurrent neural network models (e.g., fully recurrent models, Hopfield models, or Boltzmann machine models), or any other type of neural network model that includes weights.

[0038] The first model and the second model can be used to classify data. The data can be any type of data, e.g., data including images (e.g., image data), data including text such as documents or records, audio data, or any other type of data that can be classified by the first neural network model and the second neural network model. In some embodiments, the data includes medical data, e.g., medical images (e.g., X-ray images, ultrasound images, etc.) or medical records. In some embodiments, the first model and the second model can be trained to produce one or more classes (e.g., labels) for the data. In some embodiments, the first model and the second model are trained to classify the same type of data (e.g., process and produce labels). For example, both the first model and the second model can be used to classify imaging data (e.g., medical imaging data). In some embodiments, the first model and the second model can be used to classify the same type of imaging data. For example, both the first model and the second model can be used to classify medical imaging data of a specific anatomical structure such as the vasculature, the heart, or any other anatomical structure. In some embodiments, the first model and the second model can produce the same type of classes (e.g., both the first model and the second model can annotate data in the same way or be used to solve the same problem). In some embodiments, the first model can be used to produce different classes (e.g., the first model can be used to produce annotations of a different type than the second model or be used to solve different problems).

[0039] During the shooting process, stop-motion shooting techniques need to be utilized to capture every subtle movement of the puppet. This requires the shooter to have a high degree of patience and precise control ability to ensure the quality of each frame. In addition, to create a more realistic animation effect, delicate lighting and scene setting for the puppet are also needed. In the traditional animation production method, although it has a unique artistic style and expressiveness, there are certain limitations in terms of production efficiency and the smoothness of character movements.

[0040] To solve this problem, in another embodiment, a method for processing puppet actions in puppet animation is also provided. The method includes the following steps: obtaining a first shape and a second shape of a virtual puppet, where the virtual puppet is a puppet image generated in software, multiple joints are configured on the virtual puppet, and the first shape and the second shape are different forms of the virtual puppet after the software adjusts the multiple joints; marking the same joints in the first shape and the second shape; obtaining a first position of each same joint in the first shape and a second position of the same joint in the second shape; setting a path between the first position and the second position, where the path is the movement route of the corresponding joint from the first shape to the second shape; moving the joints with paths set in the virtual puppet of the first shape along the path from the first position to the second position, and generating an animation of the movement process.

[0041] There can be many ways to set the path. As an additional embodiment, setting a path between the first position and the second position includes: connecting the first position and the second position to generate a line and displaying the generated line; receiving an adjustment by the user to the line, and using the adjusted line as the path.

[0042] In another additional embodiment, connecting the first position and the second position to generate a line includes: obtaining the names of the joints at the first position and the second position, and looking up a line corresponding to the names of the joints from the lines pre - saved according to the names of the joints. For each joint name, a corresponding line is pre - saved, and the line is a curve or a straight line; generating a line connecting the first position and the second position according to the shape of the found line.

[0043] In another embodiment, connecting the first position and the second position to generate a line includes: when there are multiple lines corresponding to the names of the joints pre - saved, obtaining at least one of the following information: the name of the first shape, the name of the second shape, the name used to describe the process from the first shape to the second shape; looking up a line corresponding to the information from the multiple lines; generating a line connecting the first position and the second position according to the shape of the found line.

[0044] An animated cartoon can be made according to the above - mentioned animation. The steps for making an animated cartoon can be as follows: 1. Compile a human body model, 2. Input the actual actions of a person, 3. Analyze the input actions, 4. Design actions, 5. Apply dynamics, 6. Apply constraint conditions, 7. Apply inverse dynamics, 8. Display the results of each stage.

[0045] In the first stage of "constructing a human body model", the human body is decomposed into parts that are the smallest units forming movements. According to the inherent properties of these parts, their mutual relationships, and constraint conditions such as the range of joint movements, a human body model is constructed and input into a computer as a database.

[0046] In the second stage of "inputting the actual movements of a person", the movements to be calculated are input in units of frames of a television image or units of aberration of a film image. In this case, if images taken simultaneously from multiple directions are used, the calculations in the next stage can be performed more specifically.

[0047] In the third stage of "analyzing the input movements", inverse dynamics is applied to calculate the movements input in the second stage, and the center of gravity of each part, the forces and torques acting on each joint, the overall center of gravity, and the forces and torques acting on this center of gravity are analyzed.

[0048] When only analyzing the input movements, the center of gravity of each part, the forces and torques acting on each joint, the overall movement and center of gravity, the forces and torques acting on the overall center of gravity obtained in the third stage are used, such as with arrows, and overlapped with the input actual movements and represented on the image.

[0049] The movement analysis is carried out in this way. Next, an explanation is given for calculating a new movement using the above analysis results. To design a new movement, the human body model data in the first stage, the data of the actual movements of a person obtained in the second stage, and the analysis result data obtained in the third stage are input into the data rate in advance.

[0050] In the action design of the fourth stage, the producer initially selects basic actions from the database. The forces occurring at each joint of the human body are represented on three vertical axes of X, Y, and Z. In addition, for two forces occurring at the same joint, of course, their magnitudes are equal and their directions are opposite. In addition, complex actions can be represented by multiple control line diagrams. For example, the control line diagram representing the action of a person walking after standing up from a chair is formed by synthesizing successive actions. In addition, for the line diagrams of other parts, they are designed in the same way as the example of the left elbow. Next, for all parts of the human body where forces occur, overall changes including physical variable changes such as enlargement and reduction of the horizontal and vertical axes of the control line diagram are carried out, as well as partial changes that cause physical variable changes such as the force occurring in a certain part of the human body. In the fifth stage of "applied dynamics", the actions of each part are calculated according to the forces specified by the producer and the dynamic equations governing the actions of each part. In this case, originally, each part of the human body is in a mutually connected relationship. However, in order to reduce the amount of calculation, each part of the body is separated from other parts, and the constraint conditions regarding the mutual connection relationship and joint movement range of each part of the human body are temporarily ignored. In order to calculate the actions of each part, in the action analysis of this embodiment, the linear acceleration of the center of gravity is obtained using Newton's equation, and the angular acceleration of the center of gravity is obtained using Euler's equation. After obtaining the linear acceleration and angular acceleration, they are integrated to obtain the velocity, and then integrated again to obtain the position.

[0051] In the sixth stage of "applied constraint conditions", for the calculation results of the actions of each part, two physical constraint conditions regarding the mutual connection relationship of each part of the human body and the joint movement range are verified. This process starts from the basic part, and each lower part is verified in order of position to direction. Here, two conditions are also verified: whether the lower part is always connected to the upper part and whether the movement of each joint exceeds the determined range. As a result, in the case where the lower part is not connected to the upper part, the lower part is translated parallel to connect it to the upper part. In the case where the movement of each joint exceeds a certain range, by adjusting the rotation, the movement of the joint is made to fall within the range, thereby correcting it to a natural posture. In the seventh stage of "applied inverse dynamics", the Lagrangian equation representing the relationship between force and action is used to calculate the forces generated at each joint of the body. When designing a new action and not yet obtaining a satisfactory result, the process from the fifth stage to the seventh stage is repeated, so as to design a new action in a dialogue form. In the eighth stage of "displaying the action", the new action in the middle of design or at the end of design is displayed on the image. At this time, the position representing the center of gravity of the human body, the direction of the occurring force, and the synthesis of the human body can be displayed, so that the represented action can be displayed more specifically.

[0052] A reasonable and perfect combination of forces can be obtained based on inverse dynamics. In addition, without relying on inverse dynamics, it is impossible for the designer to find a perfect force design. In this embodiment, the arrangement direction of each part of the body changes as its joints exceed the limit, so that the row positions of each part of the body match the physical constraint conditions of the body. Since the human body movements obtained in this way are natural human body movements that can always connect the lower part and the upper part of the human body and can ensure that the movements of each joint do not exceed the determined range, the human body movements can be stereoscopically enriched and realistically displayed in this way.

[0053] In summary, the method for producing an animated cartoon applying dynamics according to this embodiment includes two processes: analyzing the basic movements of an actual person and programming new movements. And the analysis of the basic human body movements is carried out in three stages: programming a human body model, inputting actual movements, and analyzing the input movements, and the programming of movements is carried out in three stages: dynamic constraints and inverse dynamics. In the dynamics stage, the human body is divided into 50 separately independent parts separated by joints, and the movements of each part are calculated using Newton's equations and Euler's equations or by separating the movements from other parts. In the constraint condition stage, the mutual combination relationship of the body parts and the joint movement range are verified. In the inverse dynamics stage, the forces generating new movements corrected by the constraint conditions are calculated.

[0054] Therefore, according to the method for producing an animated cartoon of this embodiment, the calculation problems in the production of animated cartoons applying dynamics so far can be solved, dynamics can be better applied to the actual animated cartoon production operation, and real-time feedback can be realized. And for calculating the movements of each part of the human body, since the linear acceleration of the center of gravity is obtained using Newton's equations and the acceleration of the center of gravity is obtained using Euler's equations or, the positions of the centers of gravity of each part of the body and the forces applied to these centers of gravity can be obtained and displayed, and the position of the overall center of gravity and the force applied to this center of gravity can also be obtained and displayed. In addition, it is possible to smoothly realize the programming of three-dimensional, rich and realistic rather than line-drawing human body movements. In addition, the producer can view the human body model on the display image from various viewpoints, and can translate the body parts or rotate them in a dialogue form. Therefore, the producer can correctly grasp the correspondence between the image and the human body model. For the basic data including human body movements and the knowledge of the constraint conditions specifying the movement ranges of each joint obtained by relying on the intuition of the producer in the traditional method for producing animated cartoons, the knowledge in the production of animated cartoons of this embodiment is actual motion parameters obtained by analyzing the actual movements of people. Therefore, scientific, reliable and realistic movements can be programmed using this knowledge.

[0055] In addition, recently, the goal-directed mode has been used in various fields. According to the goal-directed method, the user interface has become a direct operation mode. Since this direct operation mode makes the image displayed on the image react to the object, if the method for producing an animated cartoon according to this embodiment is used, the object can be directly processed or operated in the object space.

[0056] In this embodiment, an electronic device is provided, which includes a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the method in the above embodiment.

[0057] The above program can run in the processor, or can also be stored in the memory (or referred to as a computer-readable medium). The computer-readable medium includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0058] These computer programs can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate computer-implemented processing. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 The steps corresponding to different steps can be implemented by different modules.

[0059] In this embodiment, such a device or system is provided. The system is called a system for generating virtual puppets for animated images and is applied to software. The software includes the following modules: an acquisition module for acquiring an image of a pre-drawn character, where the character includes at least one of the following: a person, an animal; a determination module for acquiring a joint relationship corresponding to the character, where the joint relationship is pre-configured and is used to indicate the position of the joints of the character when the character is a puppet on the character; a display module for displaying the joint relationship and marking each joint on the character in the image corresponding to the character; a processing module for generating a virtual puppet from each joint of the character and the image, where a virtual three-dimensional image is generated according to the position of each joint, and the image of the character is mapped onto the three-dimensional image to generate the virtual puppet; Optionally, the processing module is configured to: divide the three-dimensional image into different parts according to the positions of each joint of the character; split the image into pictures corresponding to the parts according to the positions of each joint; attach the pictures corresponding to each part to the corresponding part and perform rendering to obtain the virtual puppet.

[0060] Optionally, the obtaining module is configured to: obtain a pre-drawn three-dimensional image of a character, where the three-dimensional image displays the character from multiple angles; or, obtain multiple pre-drawn two-dimensional images of the character, where each two-dimensional image is a drawing of the character from a different angle, and generate a three-dimensional image of the character according to the multiple two-dimensional images.

[0061] Optionally, the determining module is further configured to: attach the three-dimensional image to the three-dimensional image according to the correspondence between the three-dimensional image and the three-dimensional image.

[0062] In an alternative embodiment, the determining module is further configured to: obtain multiple pictures of at least one puppet from the already captured puppet images; for each puppet, extract the parts that have undergone relative position changes in the pictures of the puppet according to different actions of the puppet in the multiple pictures, where the relative position change is the position change of each part of the puppet relative to the puppet body; determine the joints on the puppet according to the parts that have undergone the relative position changes; save the relative position relationship of each joint on the puppet on the puppet, where the relative position relationship is used as the joint relationship. Obtain multiple puppets of the same type as the character and having the joint relationship saved; obtain a text description of the character, and obtain the puppet closest to the character among the multiple puppets according to the text description; use the joint relationship of the closest puppet as the correspondence with the character.

[0063] As another optional embodiment, the processing module is further configured to: after the number of groups of saved training data exceeds a threshold, input the saved multiple groups of training data into a neural network model for training, where the multiple pictures are used as the input data of the neural network model, and the joint relationship is used as the output data of the neural network model; after the neural network model converges during training, obtain a trained neural network model. Input the pre-drawn image of the character into the trained neural network model to obtain the joint relationship output by the neural network model.

[0064] The system or device is used to implement the functions of the methods in the above embodiments. Each module in the system or device corresponds to each step in the method, and those that have been described in the method will not be repeated here.

[0065] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A method for generating a virtual puppet for animated images, characterized in that, Including: Obtain an image of a pre-drawn character, where the character includes at least one of the following: a person, an animal; Obtain the joint relationship corresponding to the character, where the joint relationship is pre-configured and is used to indicate the positions of the joints on the character when the character is a puppet; Mark each joint on the character on the image corresponding to the character according to the joint relationship; Generate a virtual puppet based on each joint of the character and the image, where a virtual three-dimensional image is generated according to the position of each joint, and the image of the character is mapped onto the three-dimensional image to generate the virtual puppet.

2. The method according to claim 1, characterized in that, Generating the virtual puppet based on each joint of the character and the image includes: Dividing the three-dimensional image into different parts according to the positions of each joint of the character; Splitting the image into pictures corresponding to the parts according to the positions of each joint; Attaching the pictures corresponding to each part to the corresponding part and performing rendering to obtain the virtual puppet.

3. The method according to claim 2, wherein, Obtaining an image of a pre-drawn character includes: Obtain a three-dimensional image of a pre-drawn character, where the three-dimensional image displays the character from multiple angles; or, obtain multiple two-dimensional images of the pre-drawn character, where each two-dimensional image is of the character drawn from a different angle, and generate a three-dimensional image of the character based on the multiple two-dimensional images.

4. The method according to claim 3, wherein Generating the virtual puppet based on each joint of the character and the image includes: Attaching the three-dimensional image to the three-dimensional image according to the correspondence between the three-dimensional image and the three-dimensional image.

5. A system for generating virtual puppets for animated images, applied in software, characterized in that, The software includes the following modules: An acquisition module for obtaining an image of a pre-drawn character; where the character includes at least one of the following: a person, an animal; A determination module for obtaining the joint relationship corresponding to the character, where the joint relationship is pre-configured and is used to indicate the positions of the joints on the character when the character is a puppet; A display module for displaying each joint on the character marked on the image corresponding to the character according to the joint relationship; A processing module for generating a virtual puppet based on each joint of the character and the image, where a virtual three-dimensional image is generated according to the position of each joint, and the image of the character is mapped onto the three-dimensional image to generate the virtual puppet.

6. The system according to claim 5, wherein The processing module is used for: Dividing the three-dimensional image into different parts according to the positions of each joint of the character; Splitting the image into pictures corresponding to the parts according to the positions of each joint; Attaching the pictures corresponding to each part to the corresponding part and performing rendering to obtain the virtual puppet.

7. The system according to claim 6, wherein The acquisition module is further used for: Obtain a three-dimensional image of a pre-drawn character, where the three-dimensional image displays the character from multiple angles; or, obtain multiple two-dimensional images of the pre-drawn character, where each two-dimensional image is of the character drawn from a different angle, and generate a three-dimensional image of the character based on the multiple two-dimensional images.

8. The system according to any one of claims 5-7, characterized in that, The determination module is further used for: Attach the three-dimensional image to the three-dimensional avatar according to the correspondence between the three-dimensional image and the three-dimensional avatar.

9. An electronic device, comprising a memory and a processor; wherein, The memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method steps described in any one of claims 1 to 4.

10. A readable storage medium having computer instructions stored thereon, wherein, When the computer instructions are executed by the processor, the method steps described in any one of claims 1 to 4 are implemented.

Images