Method, system and non-transitory computer readable recording medium for making animation

Abstract

According to one embodiment of the present invention, there is provided a method for producing an animation, comprising the steps of: obtaining characteristic information relating to appearance and motion of a real-world object from an input video; determining a virtual object associated with the real-world object with reference to the characteristic information relating to appearance of the real-world object, and determining a virtual motion associated with the real-world object with reference to the characteristic information relating to motion of the real-world object; and producing an animation sequence based on the virtual object and the virtual motion.

Description

Technical Field

[0001] This invention relates to a method, system, and non-transitory computer-readable recording medium for producing animation. Background Technology

[0002] In recent years, there has been a growing demand for content that can be rendered in three dimensions on two-dimensional screens of digital devices such as personal computers and mobile terminals. Furthermore, with the rise of personal media platforms and content creator platforms, the number of general users who want to directly create and use three-dimensional content is also increasing.

[0003] However, in the past, content creation tools were used to create such 3D content, but the usual animation creation tools are complex and difficult to use, and generally only skilled experts can use them.

[0004] Therefore, the inventors propose a novel and advanced technology that can determine virtual objects and virtual motion based on feature information related to the appearance and motion of real-world objects determined from input images, and generate an animation sequence based on the virtual objects and virtual motion.

[0005] Existing technical documents

[0006] Patent documents

[0007] Patent Document 1: Korean Patent Publication No. 2014-0070915 (June 11, 2014) Summary of the Invention

[0008] The technical problem to be solved by the invention

[0009] The purpose of this invention is to solve all the technical problems of the aforementioned prior art.

[0010] Furthermore, another objective of this invention is to enable anyone to easily and conveniently create animations using photographic images.

[0011] Furthermore, another object of the present invention is to determine, from among the virtual objects purchased and held by the user on the animation production platform, a virtual object having an appearance that is at least as similar to the appearance of a real-world object; and to determine, from among the virtual motions purchased and held by the user on the animation production platform, a virtual motion that is at least as similar to the motion of a real-world object. Animation sequences are then generated using the determined virtual objects and virtual motions.

[0012] means for solving technical problems

[0013] The representative configuration of the present invention for achieving the stated objective is as follows.

[0014] According to one aspect of the present invention, a method for creating animation is provided, comprising the steps of: obtaining feature information related to the appearance and motion of a real-world object from an input image; determining a virtual object associated with the real-world object by referring to the feature information related to the appearance of the real-world object, and determining a virtual motion associated with the real-world object by referring to feature information related to the motion of the real-world object; and generating an animation sequence based on the virtual object and the virtual motion.

[0015] According to another aspect of the present invention, a system for producing animation is provided, comprising: a feature information acquisition unit for acquiring feature information related to the appearance and motion of a real-world object from an input image; a determination unit for determining a virtual object associated with the real-world object by referring to the feature information related to the appearance of the real-world object, and determining a virtual motion associated with the real-world object by referring to feature information related to the motion of the real-world object; and a sequence generation unit for generating an animation sequence based on the virtual object and the virtual motion.

[0016] In addition, other methods, other systems for implementing the present invention, and non-transitory computer-readable recording media for recording computer programs for performing said methods are also provided.

[0017] Invention Effects

[0018] According to the present invention, anyone can easily and simply create animations using photographic images.

[0019] Furthermore, according to the present invention, among virtual objects purchased and held by a user on an animation production platform, a virtual object with an appearance that is at least as similar to the appearance of a real-world object can be determined; among virtual motions purchased and held by a user on an animation production platform, a virtual motion that is at least as similar to the motion of a real-world object can be determined; and an animation sequence can be generated using the determined virtual object and virtual motion. Attached Figure Description

[0020] Figure 1 This is a diagram showing the general structure of an overall system for creating animations according to an embodiment of the present invention.

[0021] Figure 2 This is a diagram showing in detail the internal structure of an animation production system based on an embodiment of the present invention.

[0022] Figures 3 to 10 This is a diagram illustrating the process of creating an animation according to an embodiment of the present invention.

[0023] Symbol Explanation: 100 - Communication Network, 200 - Animation Production System, 210 - Feature Information Acquisition Unit, 220 - Decision Unit, 230 - Sequence Generation Unit, 240 - Shared Management Unit, 250 - Communication Unit, 260 - Control Unit, 300 - Device Detailed Implementation

[0024] The detailed description of the invention described below refers to the accompanying drawings, which illustrate specific embodiments in which the invention can be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. Various embodiments of the invention differ from one another, but it should be understood that these embodiments are not mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented by changing one embodiment to another without departing from the spirit and scope of the invention. Furthermore, it should be understood that the position or arrangement of individual constituent elements in various embodiments may also be changed without departing from the spirit and scope of the invention. Therefore, the detailed description described below is not intended to be limiting, but should be accepted as encompassing the scope claimed by the claims and all equivalents thereof. Similar reference numerals in the drawings denote the same or similar constituent elements on various sides.

[0025] Hereinafter, several preferred embodiments of the invention will be described in detail with reference to the accompanying drawings, so that those skilled in the art can readily implement the invention.

[0026] Overall system structure

[0027] Figure 1 This is a diagram showing the general structure of an overall system for creating animations according to an embodiment of the present invention.

[0028] like Figure 1 As shown, an overall system based on an embodiment of the present invention may be configured to include a communication network (100), an animation production system (200), and devices (300).

[0029] First, the communication network (100) based on an embodiment of the present invention can be constructed without distinguishing between wired or wireless communication methods, and can be composed of various communication networks such as a local area network (LAN), a metropolitan area network (MAN), and a wide area network (WAN). Preferably, the communication network (100) mentioned in this specification can be the well-known Internet or the World Wide Web (WWW). However, the communication network (100) is not necessarily limited to this, and may also include at least a portion of a well-known wired / wireless data communication network, a well-known telephone network, or a well-known wired / wireless television network.

[0030] For example, the communication network (100) may be a wireless data communication network, and at least a portion thereof implements conventional communication methods such as radio frequency (RF) communication, WiFi communication, cellular (LTE, etc.) communication, Bluetooth communication (more specifically, Bluetooth Low Energy (BLE) communication), infrared communication, and ultrasonic communication.

[0031] Next, an animation production system (200) based on an embodiment of the present invention can be a digital device having a memory unit and a microprocessor to provide computing power. For example, such an animation production system (200) can be a server system.

[0032] An animation production system (200) based on an embodiment of the present invention can perform the following functions: obtain feature information related to the appearance and movement of a real-world object from an input image, determine a virtual object related to the real-world object by referring to the feature information related to the appearance of the real-world object, determine a virtual movement related to the real-world object by referring to the feature information related to the movement of the real-world object, and generate an animation sequence based on the virtual object and the virtual movement.

[0033] The input image according to an embodiment of the present invention can refer to an image captured in real time or within a specified period by at least one camera module (e.g., an RGB camera, an infrared camera, an ultraviolet camera, a TOF (Time-of-Flight) camera, etc.) or at least one scanner module (e.g., a LiDAR sensor, a radar sensor, etc.). Furthermore, the input image as described above can include two-dimensional images or three-dimensional images. On the other hand, it should be understood that the input image according to an embodiment of the present invention is not necessarily limited to the camera module (or type of camera module) or scanner module (or type of scanner module) mentioned above, and can be modified in various ways within the scope of achieving the objectives of the present invention.

[0034] Furthermore, according to an embodiment of the present invention, a real-world object can be a concept that refers to all types of objects in the real world, such as people and things, that can be determined using at least one camera module or at least one scanner module.

[0035] The structure and function of the animation production system (200) based on the present invention will now be described in more detail. On the other hand, the animation production system (200) has been described as above, but such description is merely illustrative. At least some of the functions or components required by the animation production system (200) may be implemented within the device (300) or external system (not shown) as needed, or may be included in the device (300) or external system, as will be apparent to those skilled in the art.

[0036] Next, the device (300) based on an embodiment of the present invention is a digital device that has the function of communicating with the animation production system (200) via a communication network (100). Any portable digital device, such as a smartphone or tablet PC, that has a memory unit and is equipped with a microprocessor to provide computing power can be used as the device (300) based on the present invention. Furthermore, according to an embodiment of the present invention, the device (300) may also include at least one camera module or at least one scanner module for acquiring input images, and a display module (e.g., LCD, LED display, OLED display, etc.) for playing animations.

[0037] On the other hand, the device (300) according to an embodiment of the present invention may also include an application for supporting the animation production function based on the present invention. Such an application can be downloaded from the animation production system (200) or an external application distribution server (not shown).

[0038] Structure of an animation production system

[0039] The following section will explain the internal structure of the animation production system (200) that performs important functions in order to realize the present invention, as well as the functions of each component.

[0040] Figure 2 This is a diagram showing in detail the internal structure of an animation production system (200) based on an embodiment of the present invention.

[0041] like Figure 2 As shown, the system may be configured to include a feature information acquisition unit (210), a decision unit (220), a sequence generation unit (230), a sharing management unit (240), a communication unit (250), and a control unit (260). According to one embodiment of the present invention, at least a portion of the feature information acquisition unit (210), decision unit (220), sequence generation unit (230), sharing management unit (240), communication unit (250), and control unit (260) may be a program module that communicates with an external system. Such a program module may be included in the animation production system (200) as an operating system, application module, or other program module, and may be physically stored in various known storage devices. Furthermore, such a program module may also be stored in a remote storage device capable of communicating with the animation production system (200). On the other hand, such program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc., that perform specific business operations or specific abstract data types according to the present invention.

[0042] First, the feature information acquisition unit (210) according to an embodiment of the present invention is capable of performing the function of acquiring feature information related to the appearance and movement of a real-world object from an input image. The feature information related to the appearance of a real-world object according to an embodiment of the present invention may include information related to the size, height, width, shape, color, position, temperature, and connectivity of the real-world object (specifically, each region or part defined in the real-world object). The feature information related to the movement of a real-world object according to an embodiment of the present invention may include information related to the movement speed, movement path, movement angle, acceleration, and angular velocity of the real-world object (specifically, each region or part defined in the real-world object).

[0043] For example, the feature information acquisition unit (210) can determine the type of real-world object (e.g., person, animal, plant, thing, etc.) from the input image, and extract feature information related to the appearance and movement of the real-world object with reference to that type. More specifically, if the type of real-world object is determined to be a person, the feature information acquisition unit (210) can determine the attributes of each body part of a person with reference to a model related to the person's body structure (e.g., the position, shape, and connection relationship of the head, face, eyes, nose, mouth, arms, and legs) and movements (e.g., the range of motion and direction of the head, face, eyes, nose, mouth, arms, and legs), and can extract feature information related to the person's appearance (e.g., gender, height, hairstyle, clothing, position and size of each part of the face, etc.) and movements (e.g., the movement path, movement speed, movement direction, etc. of a person or each body part of a person) based on the determined attributes of each body part. Furthermore, if the type of real-world object is determined to be a bus, the feature information acquisition unit (210) can refer to a model related to the bus's structure (e.g., the position, shape, and orientation of the body, wheels, and windows) and movement (e.g., the range of motion and direction of the wheels) to determine the attributes of each component of the bus, and can extract feature information related to the appearance and movement of the bus based on the determined attributes of each component.

[0044] As another example, the feature information acquisition unit (210) analyzes real-world objects in the input image using an object analysis model for identifying real-world objects (or types of real-world objects) or for extracting features of real-world objects. This allows it to identify what kind of object or object the real-world object is, or to extract feature information related to the appearance and movement of the real-world object. Here, the object analysis model may be a model that has been learned using feature information related to the appearance (e.g., size, height, width, shape, color, position, temperature, connectivity, etc. of each region or part determined in each real-world object) and movement (e.g., the speed, path, angle, acceleration, angular velocity, etc. of each region or part determined in each real-world object). (In this case, learning feature information related to the appearance and movement of multiple real-world objects and the real-world objects may also be considered, taking into account at least one of the position of the real-world object determined in the input image, the direction of the real-world object, and the distance to the real-world object.) Such learning can be based on algorithms such as Naive Bayes classification, Support Vector Machine (SVM), Artificial Neural Network (ANN), and Hidden Markov Model (HMM). Furthermore, the feature information acquisition unit (210) can use an object analysis model to identify real-world objects (or types of real-world objects) or extract feature information related to the appearance and movement of the real-world object. However, it can also construct a first object analysis model for identifying real-world objects (or types of real-world objects), a second object analysis model for extracting feature information related to the appearance of real-world objects, and a third object analysis model for extracting feature information related to the movement of real-world objects, and use each of these models to identify what kind of object the real-world object is or to extract feature information related to the appearance and movement of the real-world object.

[0045] More specifically, the feature information acquisition unit (210) can use a first object analysis model for identifying real-world objects (or types of real-world objects) to identify what kind of object or type of real-world object is in the input image (in this case, the input image or data related to real-world objects in the input image can be input into the first object analysis model and detection, classification, segmentation, and recognition processes can be performed to identify the real-world object (or type of real-world object)). Furthermore, the feature information acquisition unit (210) can use a second object analysis model for extracting feature information related to the appearance of real-world objects to extract feature information related to the appearance of real-world objects (in this case, the input image or data related to real-world objects in the input image can be input into the second object analysis model and detection, classification, segmentation, and recognition processes can be performed to extract feature information related to the appearance of real-world objects). Furthermore, the feature information acquisition unit (210) can utilize a third object analysis model for extracting feature information related to the movement of real-world objects to extract feature information related to the movement of real-world objects (in this case, the input image or data related to real-world objects in the input image can be input into the third object analysis model and a detection, classification, subdivision, and recognition process can be performed to extract feature information related to the movement of real-world objects). On the other hand, such a third object analysis model may also include a learning model for detecting the skeleton (bones) of real-world objects and connecting skeleton points (e.g., joints when the real-world object is a person) (e.g., a learning model for performing skeletal operations), a learning model for detecting the movement of real-world objects based on the skeleton, a learning model for detecting conditions related to real-world objects, and so on.

[0046] Furthermore, the feature information acquisition unit (210) can also acquire feature information related to the emotion of a real-world object from the input image. Based on the present invention, the feature information related to the emotion of a real-world object may include information related to the type of emotion (e.g., happy, angry, sad, pleasant, blissful, annoyed, etc.) and its intensity (or strength).

[0047] For example, the feature information acquisition unit (210) can determine the type of real-world object (e.g., person, animal, plant, thing, etc.) from the input image, and obtain feature information related to the emotion of the real-world object by referring to that type. More specifically, if the type of real-world object is determined to be a person, the feature information acquisition unit (210) can determine the attributes of each body part of the person by referring to a model related to the person's body structure (e.g., the position, shape and connection relationship of the head, face, eyes, nose, mouth, arms and legs) and movements (e.g., the range of motion and direction of the head, face, eyes, nose, mouth, arms and legs), and can extract feature information related to the person's emotion based on the determined attributes of each body part.

[0048] As another example, the feature information acquisition unit (210) can analyze the real-world objects in the input image by referring to an object analysis model used to extract emotion-related features from real-world objects in order to extract emotion-related feature information of the real-world objects. At least part of the structure and manner of such an object analysis model can be similar to the object analysis models known above for extracting feature information related to appearance or motion.

[0049] As another example, the feature information acquisition unit (210) can extract feature information related to the emotion of a real-world object based on feature information related to at least one of the appearance and movement of the real-world object. More specifically, for example, when the real-world object is a person, the feature information acquisition unit (210) can determine the position of the eyes, the position, shape, and movement of the mouth based on feature information related to the person's appearance and movement, and determine the type of the person's emotion based on this; or, it can determine the movement, movement path, and movement speed of the hands and legs (or feet) based on feature information related to the person's appearance and movement, and determine the type of the person's emotion based on this (for example, when lingering in a specific location, it can be determined to be an anxious emotion). Furthermore, for example, when the real-world object is a person, the feature information acquisition unit (210) can determine the position of the eyes, the position, shape, and changes or duration of the mouth's movements based on feature information related to the person's appearance and movement, and determine the intensity of the person's emotion based on this; or, it can determine the movement, changes in movement path, changes in movement speed, and duration of the hands, legs, or feet based on feature information related to the person's movement, and determine the intensity of the person's emotion based on this.

[0050] On the other hand, it should be clarified that the feature information related to the emotion of a real-world object based on an embodiment of the present invention does not necessarily need to be distinguished from the feature information related to the movement of the real-world object, and can be included in the feature information related to the movement of the real-world object as needed.

[0051] Next, the decision unit (220) based on an embodiment of the present invention can perform the following functions: determine a virtual object related to a real-world object by referring to feature information related to the appearance of a real-world object, and determine a virtual motion related to a real-world object by referring to feature information related to the motion of a real-world object.

[0052] For example, among multiple virtual objects, the decision unit (220) can determine objects with an appearance that is at least as similar to the appearance of real-world objects as virtual objects based on feature information related to the appearance of real-world objects. Similarly, among multiple virtual movements, movements that are at least as similar to the movements of real-world objects are determined as virtual movements related to real-world objects based on feature information related to the movements of real-world objects. Here, the multiple virtual objects and multiple virtual movements can be set according to a database related to virtual objects and virtual movements (such a database can be contained in the animation production system (200) based on the present invention or exist in an external system), or purchased and held by a user on a platform related to animation production based on the present invention (in which case, the multiple virtual objects and multiple virtual movements can vary depending on the user).

[0053] On the other hand, the decision-making department (220) determines the manner of virtual movement that is similar to the movement of real-world objects, which may vary depending on the presence or absence of the skeleton of the real-world object.

[0054] For example, when a real-world object has a skeleton, a virtual movement with a specified level of similarity to the movement of that real-world object can be determined based on information related to changes in the skeleton of the real-world object (such information can belong to the feature information related to the movement of the real-world object). When a real-world object does not have a skeleton, a virtual movement with a specified level of similarity to the movement of that real-world object can be determined based on feature information related to the movement of the real-world object divided by unit time or unit area (i.e., multiple unit feature information related to the movement of the real-world object). On the other hand, it should be clarified that the method of determining a virtual movement similar to the movement of a real-world object according to an embodiment of the present invention is not necessarily limited to the above-described method, and can be arbitrarily modified within the scope of achieving the purpose of the present invention.

[0055] Furthermore, the decision unit (220) can also refer to the user's personal information to determine at least one of the virtual object and the virtual sport. According to an embodiment of the present invention, the user's personal information may include information related to the user's age, gender, ethnicity, region, preferred virtual object, preferred sport, etc.

[0056] For example, the decision unit (220) can determine the virtual objects and virtual movements preferred by the user based on the personal information of the user (or the holder of the device (300)) related to the animation production, and can assign a higher weighting value to the virtual objects and virtual movements preferred by the user among multiple virtual objects and virtual movements than other virtual objects and virtual movements. That is, when deciding on at least one virtual object and virtual movement, the user's preference for the determined virtual object and virtual movement can also be taken into consideration.

[0057] Next, the sequence generation unit (230) based on an embodiment of the present invention is able to perform the function of generating an animation sequence based on the virtual object and virtual motion.

[0058] For example, the sequence generation unit (230) can generate a virtual object to perform virtual movement for a predetermined time (e.g., the same time as the movement time of a previously known real-world object). More specifically, the sequence generation unit (230) can generate a virtual object to perform virtual movement while moving along a movement path that is the same as or similar to that of a real-world object.

[0059] Furthermore, the sequence generation unit (230) can combine the animation sequence generated based on the virtual object and virtual motion with real-world images to generate at least one of augmented reality (AR) images and mixed reality (MR) images.

[0060] For example, the sequence generation unit (230) can generate augmented reality images by combining an animation sequence with an input image as a background. More specifically, the sequence generation unit (230) can generate augmented reality images in which virtual objects move along the same or similar lines as real-world objects while performing virtual motion, with an image from which real-world objects have been removed as a background.

[0061] Next, the sharing management unit (240) based on an embodiment of the present invention enables multiple users to share animation sequences generated by the sequence generation unit (230) or images generated in combination with the animation sequences (e.g., augmented reality images or mixed reality images).

[0062] For example, the sharing management unit (240) can enable the animation sequence or the augmented reality image generated in combination with the animation sequence to be shared with other users (e.g., real-time sharing) through the social networking service (SNS), cloud, web server, etc. of the user (or the holder of the device (300)) related to the animation production.

[0063] On the other hand, the sharing management unit (240) can provide a user interface that allows other users to add or modify at least one of the virtual objects and virtual movements in the shared animation sequence, or to change them to other virtual objects or virtual movements. That is, the virtual objects or virtual movements in the shared animation sequence can be added, modified, or changed by other users.

[0064] Next, the communication unit (250) based on an embodiment of the present invention can perform the function of transmitting and receiving data from the feature information acquisition unit (210), the decision unit (220), the sequence generation unit (230), and the sharing management unit (240) to the feature information acquisition unit (210), the decision unit (220), the sequence generation unit (230), and the sharing management unit (240).

[0065] Finally, the control unit (260) based on an embodiment of the present invention is capable of performing the function of controlling the data flow between the feature information acquisition unit (210), the decision unit (220), the sequence generation unit (230), the sharing management unit (240), and the communication unit (250). That is, the control unit (260) based on an embodiment of the present invention can control the data flow from the outside of the animation production system (200) to the outside of the animation production system (200) or the data flow between the various components of the animation production system (200) to perform the inherent functions in the feature information acquisition unit (210), the decision unit (220), the sequence generation unit (230), the sharing management unit (240), and the communication unit (250).

[0066] Figures 3 to 10 This diagram illustrates, by way of example, the process of creating an animation according to an embodiment of the present invention.

[0067] Example 1

[0068] It can be assumed that the animation production system (200) based on an embodiment of the present invention is contained in a user's smartphone (300).

[0069] First, refer to Figure 3 According to one embodiment of the present invention, an image of a person (310) wearing a suit running from location 1 to location 2 can be obtained as an input image through the camera module of a user's smartphone (300).

[0070] Next, according to an embodiment of the present invention, feature information related to the appearance, movement and emotions of a person (i.e., a real-world object) (310) can be obtained from the input image obtained above.

[0071] For example, it is possible to determine the type of the real-world object (310) as human from the above-mentioned input image by referring to at least one object analysis model (such object analysis model may be a model generated by performing deep learning on real-world objects and feature information related to appearance, movement and emotion associated with the real-world object), and to obtain feature information related to appearance, movement and emotion of the real-world object (310).

[0072] More specifically, it is possible to extract features related to the appearance of a 180cm tall male with a waxed hairstyle, wearing a shirt and suit pants, and a tie, whose eyes, nose, and mouth are in specified positions (e.g., the lengths from the hairline to the bottom of the eyebrows, from the bottom of the eyebrows to the root of the nose, and from the root of the nose to the tip of the chin are the same, and the horizontal length of the face is 5 times the horizontal length of one eye) as features related to the appearance of a real-world object (310). Furthermore, it is possible to extract features related to the movement of a real-world object (310) such as elbows bent at 80 degrees repeatedly moving back and forth at a specified speed, and knees repeatedly bending and extending at a specified speed. Additionally, it is possible to extract features related to the emotion of a low-intensity laugh (e.g., such intensity can be divided into several or dozens of levels depending on the intensity of the emotion) as features related to the emotion of a real-world object (310).

[0073] Next, according to an embodiment of the present invention, a virtual object associated with a real-world object (310) can be determined by referring to feature information related to the appearance of the real-world object (310), a virtual motion associated with the real-world object (310) can be determined by referring to feature information related to the motion of the real-world object (310), and a virtual emotion associated with the real-world object (310) can be determined by referring to feature information related to the emotion of the real-world object (310).

[0074] For example, refer to Figure 4 Multiple virtual objects stored on a user's smartphone or external system, or purchased and held by the user on an animation-related platform (see reference). Figure 4 In (a)), an object with an appearance that is at least as similar to the appearance of a real-world object (310) can be determined as a virtual object (410) based on feature information related to the appearance of the real-world object (310). Furthermore, multiple virtual motion devices (see reference) stored on the user's smartphone or external system, or purchased and held by the user on an animation production platform, are also considered. Figure 4In (b)), based on feature information related to the movement of the real-world object (310), movements that are at least as similar to the movement of the real-world object (310) at a specified level can be determined as virtual movements (420). Furthermore, among multiple virtual emotions stored on the user's smartphone or external system, or purchased and held by the user on an animation production platform, emotions that are at least as similar to the emotions of the real-world object (310) at a specified level can be determined as virtual emotions based on feature information related to the emotions of the real-world object (310).

[0075] Next, based on the virtual object (410), virtual motion (420), and virtual emotion determined as described above, an animation sequence can be generated.

[0076] For example, refer to Figure 5 It can generate virtual objects (510) with animation sequences to express virtual emotional states and move in a virtual motion (420) while moving along the same or similar motion lines (320) as the real-world objects (310).

[0077] Next, according to an embodiment of the present invention, the generated animation sequence described above can be combined with the input image obtained through a smartphone (300) to generate an augmented reality image (see reference). Figure 6 ).

[0078] Example 2

[0079] It can be assumed that an animation production system (200) based on an embodiment of the present invention and the user's augmented reality glasses (300) are interconnected through a communication network (100).

[0080] First, refer to Figure 7 According to one embodiment of the present invention, the image of a real-world object, namely a blue bus (710), traveling from location 3 to location 4 can be obtained as an input image using the user's augmented reality glasses (300).

[0081] Next, according to an embodiment of the present invention, it is possible to determine from the above-mentioned obtained input image that the type of real-world object (710) in the input image is a bus, and to extract feature information related to the appearance and movement of the real-world object (710).

[0082] Next, among multiple virtual objects, the object with a similarity level of at least that of the real-world object (710), namely the "blue TAYO-shaped bus," can be determined as a virtual object based on characteristic information related to the type and appearance of the real-world object (710). Furthermore, among multiple virtual movements, the movement with a similarity level of at least that of the real-world object (710), namely the "accelerated movement," can be determined as a virtual movement based on multiple unit characteristic information related to the movement of the real-world object (710) determined by dividing the characteristic information related to the movement of the real-world object (710) by unit time or unit area (e.g., continuous comparison).

[0083] Next, refer to Figure 8 As the bus, which is capable of generating the blue TAYO image of the aforementioned virtual object in an animation sequence, moves from location 3 to location 4, the aforementioned accelerated driving motion, which is determined to be virtual motion, is repeatedly executed within a specified time (for example, such time can be the same as the time during which the previously known real-world object, namely the blue bus (710), performs its associated motion).

[0084] 3rd Embodiment

[0085] It can be assumed that the animation sequence or augmented reality image generated by the animation production system (200) based on an embodiment of the present invention is shared with other users through social networks.

[0086] First, refer to Figure 9 According to one embodiment of the present invention, a blue bus can be repeatedly moved between location 5 and location 6 by a first user in the form of an animation sequence.

[0087] Next, at the request of the first user, the generated animation sequence can be shared with the second user via a social networking service.

[0088] Next, refer to Figure 10 The system provides a user interface for modifying the shared animation sequence described above to a second user. Using this interface, the second user can modify the animation sequence by changing the movement of the blue bus repeatedly between locations 5 and 6 to that of the red motorcycle repeatedly between locations 5, 6, 7, and 8. In other words, the system can modify the virtual object and its virtual movement within the shared animation sequence.

[0089] The embodiments of the present invention described above can be implemented and recorded in a computer-readable recording medium as program instructions executable by various computer components. The computer-readable recording medium may contain program instructions, data files, data structures, etc., individually or in combination. The program instructions recorded in the computer-readable recording medium may be program instructions specifically designed and configured for the present invention, or program instructions known to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floppy disks; and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, and flash memory. Examples of program instructions may include not only machine language code produced by a compiler, but also high-level language code executable by a computer using an interpreter. The hardware device may be modified to perform processing based on the present invention by one or more software modules, and vice versa.

[0090] The present invention has been described above using specific constituent elements and other specific matters, as well as limited embodiments and accompanying drawings. However, this description is only provided to help to understand the present invention more fully. The present invention is not limited to the described embodiments. Anyone skilled in the art to which this invention pertains can make various modifications and changes based on such description.

[0091] Therefore, the concept of the present invention should not be limited to the embodiments described above, but should encompass not only the scope of the claims described below, but also all scopes that are equivalent to or modified by the equivalent claims.

Claims

1. A method for creating animation, comprising the following steps: The steps to obtain feature information related to the appearance and movement of real-world objects from input images; The steps of determining a virtual object associated with the real-world object by referring to feature information related to the appearance of the real-world object, and determining a virtual motion associated with the real-world object by referring to feature information related to the motion of the real-world object; and The steps of generating an animation sequence based on the virtual object and the virtual motion. In the step of determining the virtual motion related to the real-world object, among multiple virtual motions, based on characteristic information related to the motion of the real-world object, the virtual motion that is at least at a certain level of similarity to the motion of the real-world object is determined to be the virtual motion related to the real-world object. Depending on the presence or absence of the skeleton of the real-world object, the method of determining the virtual motion is changed to be at least at a level of similarity to the motion of the real-world object.

2. The method according to claim 1, wherein, In the step of obtaining feature information related to the appearance and motion of real-world objects, feature information related to the appearance and motion of the real-world objects is extracted by analyzing the real-world objects in the input image using an object analysis model. The object analysis model is a model that has been learned using multiple real-world objects and feature information related to the appearance and motion of the multiple real-world objects.

3. The method according to claim 1, wherein, In the step of obtaining feature information related to the appearance and motion of a real-world object, feature information related to the appearance and motion of the real-world object is extracted with reference to the type of the real-world object determined in the input image.

4. The method according to claim 1, wherein, In the step of obtaining feature information related to the appearance and movement of a real-world object, feature information related to the emotion of the real-world object is also obtained from the input image.

5. The method according to claim 1, wherein, In the step of determining the virtual object associated with the real-world object... Among multiple virtual objects, virtual objects that have an appearance that is at least as similar to the appearance of the real-world object are determined as virtual objects related to the real-world object, based on feature information related to the appearance of the real-world object.

6. The method according to claim 5, wherein, The plurality of virtual objects and the plurality of virtual motions were determined with reference to purchase history on animation production-related platforms.

7. The method according to claim 1, wherein, In the step of generating an animation sequence, the animation sequence is combined with real-world images to generate at least one of augmented reality images and mixed reality images.

8. The method according to claim 1, wherein, This allows multiple users to share the generated animation sequence.

9. The method according to claim 8, wherein, Provide a user interface that enables at least one of the plurality of users to correct at least one of the virtual objects and the virtual motions in the shared animation sequence.

10. A non-transitory computer-readable recording medium recording a computer program for performing the method of claim 1.

11. A system for producing animation, comprising: The feature information acquisition unit acquires feature information related to the appearance and movement of real-world objects from the input image; The decision unit determines a virtual object associated with the real-world object by referring to feature information related to the appearance of the real-world object, and determines a virtual motion associated with the real-world object based on feature information related to the motion of the real-world object. and The sequence generation unit generates an animation sequence based on the virtual object and the virtual motion. The decision-making unit, among multiple virtual movements, determines, based on feature information related to the movement of the real-world object, a virtual movement that is at least at a predetermined level similar to the movement of the real-world object as the virtual movement related to the real-world object. Depending on the presence or absence of the skeleton of the real-world object, the method of determining the virtual motion is changed to be at least at a level of similarity to the motion of the real-world object.

12. The system according to claim 11, wherein, The feature information acquisition unit extracts feature information related to the appearance and movement of real-world objects by analyzing real-world objects in the input image using an object analysis model. The object analysis model is a model that has been learned using multiple real-world objects and feature information related to the appearance and motion of the multiple real-world objects.

13. The system according to claim 11, wherein, The feature information acquisition unit extracts feature information related to the appearance and movement of the real-world object by referring to the type of the real-world object determined in the input image.

14. The system according to claim 11, wherein, The feature information acquisition unit also obtains feature information related to the emotion of the real-world object from the input image.

15. The system according to claim 11, wherein, Among multiple virtual objects, the decision unit determines the virtual object that has an appearance that is at least as similar to the appearance of the real-world object as the virtual object associated with the real-world object, based on feature information related to the appearance of the real-world object.

16. The system according to claim 15, wherein, The plurality of virtual objects and the plurality of virtual motions were determined with reference to purchase history on platforms related to animation production.

17. The system according to claim 11, wherein, The sequence generation unit combines the animation sequence with real-world images to generate at least one of augmented reality images and mixed reality images.

18. The system of claim 11, further comprising a sharing management unit that enables multiple users to share the generated animation sequence.

19. The system according to claim 18, wherein, The shared management unit provides a user interface that enables at least one of the plurality of users to modify at least one of the virtual objects and the virtual motions in the shared animation sequence.

Images