Method and system for generating three-dimensional character model on basis of artificial intelligence

Abstract

The present invention relates to a method and a system for generating a three-dimensional character model on the basis of artificial intelligence. The method for generating a three-dimensional character model according to the present invention may comprise the steps of: receiving at least one person image and training an image generation model by using the person image; generating, by the trained image generation model, a character image having a full body pose from the person image by using a text prompt and a reference image; generating a three-dimensional character model by performing volume optimization on the basis of the character image; and constructing a bone skeleton of the three-dimensional model through rigging of the three-dimensional character model and connecting the bone skeleton to the character model through skinning.

Description

Method and system for creating 3D character models based on artificial intelligence

[0001] The present invention relates to a method and system for generating a three-dimensional character model based on artificial intelligence.

[0002] Recently, interest has been growing in three-dimensional virtual spaces where social, economic, and cultural activities take place, just like in the real world. These virtual spaces can be implemented not only through devices like computers, laptops, and smartphones, but also through wearable devices, utilizing virtual reality (VR), augmented reality (AR), mixed reality (MR), and extended reality (XR).

[0003] In particular, virtual spaces are being developed in a wide range of fields, from autonomous driving to tourism and gaming, and interest is growing in how to create virtual spaces that resemble real-life spaces to provide users with experiences similar to real life.

[0004] In this regard, the role of virtual CG characters in visual media such as games, movies, and animations has been gradually increasing, and the value of characters that can focus the attention of consumers is also gradually expanding.

[0005] Meanwhile, the present invention was derived as part of the national research and development project, 'Development of Universal Fashion Creation Platform Technology for Avatar Individuality Expression' (Project Unique Number: 1375027535, Project Number: 00228331, Ministry Name: Ministry of Culture, Sports and Tourism, Project Management (Specialized) Agency Name: Korea Creative Content Agency, Research Project Name: Metaverse Content IP Construction Research and Development, Research Project Name: Development of Universal Fashion Creation Platform Technology for Avatar Individuality Expression, Contribution Rate: 1 / 1, Project Performing Agency Name: Kai Co., Ltd., Research Period: 2023.04.01 ~ 2023.12.31).

[0006] The present invention provides a method and system for generating a 3D character model capable of generating a 3D character model based on artificial intelligence.

[0007] In particular, the present invention provides a method and system for creating a three-dimensional character model capable of creating a character containing the user's personality.

[0008] Furthermore, the present invention provides a method and system for creating a three-dimensional character model that can create a dynamic three-dimensional character model through rigging and skinning.

[0009] In order to solve the problem discussed above, a method for generating a three-dimensional character model according to the present invention may include the steps of receiving at least one human image and training an image generation model using the human image, the step of generating a character image having a full-body pose from the human image by the trained image generation model using a text prompt and a reference image, the step of generating a three-dimensional character model by performing volume optimization based on the character image, and the step of constructing a bone skeleton of the three-dimensional model through rigging of the three-dimensional character model and connecting the bone skeleton to the character model through skinning.

[0010] Furthermore, the person image may include multiple images, at least one of which is not a full body image of the user.

[0011] Furthermore, the step of training the image generation model can learn the checkpoints of the person image using the text corresponding to the user's token and the plurality of images.

[0012] Furthermore, the reference image includes an image of a T-pose or an A-pose, and the step of generating the character image may combine the learned checkpoint and the user's token with the pre-stored text prompt, and generate the character image using the image of the T-pose or the A-pose.

[0013] Furthermore, the step of generating the three-dimensional character model may include the step of generating a plurality of rendering images based on a camera viewpoint different from the camera viewpoint of the character image, and performing volume optimization based on Gaussian splatting using the rendering images and the character image.

[0014] Furthermore, the step of performing the above volume optimization can be calculated by defining loss values ​​corresponding to the character image and the plurality of rendered images differently.

[0015] Furthermore, the skeleton may be composed of a central skeleton excluding the face and hands.

[0016] Furthermore, the step of setting a name corresponding to a rule of the animation may be further included to support animating the character model to which the skeleton is connected.

[0017] Meanwhile, a three-dimensional character model generation system according to the present invention includes a communication unit that receives at least one human image, a storage unit that stores the human image, an image generation model learned using the human image, and a program code, and a control unit that generates a three-dimensional character model using the storage unit, the image generation model, and the program code, wherein the learned image generation model generates a character image having a full-body pose from the human image using a text prompt and a reference image, and the control unit trains the image generation model using the human image, performs volume optimization based on the character image, and generates a three-dimensional character model, constructs a bone skeleton of the three-dimensional model through rigging of the three-dimensional character model, and connects the bone skeleton to the character model through skinning.

[0018] Furthermore, the control unit can perform animation using the character model to which the skeleton is connected to generate animation content for the entire body of the subject included in the character image, and output the generated animation content using a display.

[0019] Meanwhile, a program according to the present invention is a program that is executed by one or more processes in an electronic device and can be stored in a computer-readable recording medium, wherein the program may include commands for performing the steps of receiving at least one human image, training an image generation model using the human image, generating a character image having a full-body pose from the human image using a text prompt and a reference image, performing volume optimization based on the character image to generate a three-dimensional character model, and constructing a bone skeleton of the three-dimensional model through rigging of the three-dimensional character model, and connecting the bone skeleton to the character model through skinning.

[0020] As described above, the method and system for generating a 3D character model according to the present invention can generate a character including the characteristics of each user by receiving at least one human image and training an image generation model using the human image.

[0021] Furthermore, the method and system for generating a three-dimensional character model according to the present invention can generate a character image with a full-body pose from a human image using an image generation model trained using text prompts and reference images. By utilizing the reference image in the preprocessing process, the present invention can model a character with clearly distinguishable body parts and smooth movement.

[0022] Furthermore, the method and system for generating a three-dimensional character model according to the present invention can generate a three-dimensional character model by performing volume optimization based on a character image. The present invention utilizes an image generation model that generates data from various viewpoints different from the input viewpoint, thereby generating a character with minimized empty space and a sense of volume.

[0023] Furthermore, the method and system for creating a three-dimensional character model according to the present invention can construct a three-dimensional model's bone skeleton through rigging of the three-dimensional character model, and connect the bone skeleton to the character model through skinning. By supporting the animation of the character model, the present invention can provide a service environment in which users can immediately use the character in various contents such as games, comics, and animations.

[0024] Figure 1 is a conceptual diagram for explaining a 3D character model generation system.

[0025] Figure 2 is a flowchart for explaining a method for creating a three-dimensional character model according to the present invention.

[0026] Figures 3 to 8 are conceptual diagrams for explaining a method for creating a three-dimensional character model according to the present invention.

[0027] FIG. 9 is a conceptual diagram illustrating an embodiment of the present invention that provides a user with an environment in which rigging and skinning operations can be performed.

[0028] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or similar components will be given the same reference numbers, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably only for the convenience of writing the specification, and do not in themselves have distinct meanings or roles. In addition, when describing the embodiments disclosed in this specification, if it is determined that a specific description of a related known technology may obscure the gist of the embodiments disclosed in this specification, a detailed description thereof will be omitted. In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and technical scope of the present invention.

[0029] Terms that include ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by these terms. These terms are used solely to distinguish one component from another.

[0030] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0031] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0032] In this application, terms such as “include” or “have” are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but should be understood not to exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0033] Figure 1 is a conceptual diagram for explaining a three-dimensional character model generation system according to the present invention.

[0034] Referring to FIG. 1, a three-dimensional character model generation system (100) according to the present invention can input at least one character image (1) into a two-dimensional image generation model (20) to train the two-dimensional image generation model (20).

[0035] Furthermore, the 3D character model generation system (100) can generate a 2D character image (2) having a full body pose using a learned 2D image generation model (20), and input the generated 2D character image (2) into a 3D character generation model (30) to generate a 3D character model (3).

[0036] A two-dimensional character image (2) may refer to a graphic created in a two-dimensional space. More specifically, the two-dimensional character image (2) includes two dimensions, such as width and height, and may be expressed in the form of a two-dimensional plane picture (for example, without thickness). Such a two-dimensional character image (2) may be used to visually express the appearance and characteristics of a character in animation, cartoons, games, etc. The “two-dimensional character image (2)” described in the present invention may also be referred to as a “two-dimensional avatar image,” a “two-dimensional object image,” a “2D character image,” a “2D avatar image,” a “2D object image,” etc.

[0037] A three-dimensional character model (3) may refer to graphics generated in a three-dimensional space. More specifically, the three-dimensional character model (3) includes three dimensions, such as width, height, and depth, and may be formed to have spatial depth and three-dimensionality in a three-dimensional form. Such a three-dimensional character model (2) may be used to provide realistic and three-dimensional visual effects in animations, cartoons, games, etc. The “three-dimensional character model (3)” described in the present invention may also be referred to as a “three-dimensional avatar model,” “three-dimensional object model,” “3D character model,” “3D avatar model,” “3D object model,” etc.

[0038] Furthermore, the “two-dimensional image generation model (20)” described in the present invention may also be named as “image generation model”, “two-dimensional character image generation model”, “two-dimensional avatar generation model”, “two-dimensional object generation model”, “2D image generation model”, “2D character image generation model”, “2D avatar generation model”, “2D object generation model”, etc., and the “three-dimensional character generation model (30)” may also be named as “character generation model”, “three-dimensional avatar generation model”, “three-dimensional object generation model”, “3D character generation model”, “3D avatar generation model”, “3D object generation model”, etc. In this case, the two-dimensional image generation model (20) and the three-dimensional character generation model (30) may be pre-learned and exist in the three-dimensional character model generation system (100) to generate a two-dimensional character image (2) and a three-dimensional character model (3), respectively.

[0039] Meanwhile, the two-dimensional character image (2) and three-dimensional character model (3) generated in the present invention can be configured to include the user's personality (or characteristics). For example, the three-dimensional character model generation system (100) can generate a character that includes the user's characteristics (or traits) by modeling the character not only in a style similar to the user's external features (e.g., face shape, expression, skin color, eyes, nose, mouth, etc.) included in the person image (1), but also in a style with large deformation (e.g., eyes or head drawn large, or limbs long and torso small).

[0040] Here, the 3D character model (3) may be a rigged 3D character model (3) that is configured to be capable of dynamic movement through rigging and skinning operations. Rigging is the work of constructing a bone skeleton for an avatar, and skinning is the work of connecting the bone skeleton to the character model. The rigging and skinning operations can be understood as operations performed to control the way the character model moves.

[0041] That is, the 3D character model creation system (100) can create a rigged 3D character model (3) capable of various movements and expressions by performing rigging and skinning operations on the 3D character model (3).

[0042] Meanwhile, as illustrated in FIG. 1, the three-dimensional character model generation system (100) according to the present invention may include at least one of a communication unit (110), a storage unit (120), and a control unit (130).

[0043] The communication unit (110) can be connected to a server and devices, etc. via a wireless or wired network, and can be configured to receive or transmit overall data and information required for the 3D character model creation system (100).

[0044] Additionally, the communication unit (110) may be communicatively connected to a user terminal and receive a request related to the creation of a 3D character model from the user terminal. For example, the communication unit (110) may receive at least one character image (1) containing the user's characteristics (e.g., physical characteristics, facial expressions, movements, poses, emotions, etc.) from the user terminal.

[0045] Here, the user terminal may mean at least one of a mobile phone, a smart phone, a notebook computer, a laptop computer, a slate PC, a tablet PC, an ultrabook, a desktop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, and a wearable device (e.g., a smartwatch, a smart glass, a head mounted display (HMD)).

[0046] Furthermore, the communication unit (110) can support various communication methods according to the communication standards of the communicating device.

[0047] For example, the communication unit (110) may be configured to communicate with at least one of a user terminal, a server, and a device (including a cloud server) using at least one of WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World Interoperability for Microwave Access), HSDPA (High Speed ​​Downlink Packet Access), HSUPA (High Speed ​​Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), 5G (5th Generation Mobile Telecommunication), Bluetooth™, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra-Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies.

[0048] Next, the storage unit (120) may also be referred to as a database (DB) and may be configured to store various information related to the present invention. In the present invention, the storage unit (120) may be provided in the 3D character model generation system (100) itself. In addition, at least a portion of the storage unit (120) may be configured as a cloud server (or cloud storage). That is, the storage unit (120) may be sufficient as long as it is a space where information required for the 3D character model generation system (100) according to the present invention is stored, and it can be understood that there are no restrictions on the physical space.

[0049] Furthermore, the storage unit (120) may be configured to store data and commands necessary for the operation of the 3D character model generation system (100) according to the present invention. For example, the storage unit (220) may store learning data necessary for learning an image (or character) generation model, and may store commands implemented to perform a 3D character model generation task.

[0050] Meanwhile, the control unit (130) can perform a series of processes related to the present invention and can play a role in controlling the overall operation of the 3D character model generation system (100).

[0051] Specifically, the control unit (130) can process signals, data, information, etc. input or output through the components discussed above, or perform a series of data processing to provide or process appropriate information and functions to a user.

[0052] For example, when the control unit (130) receives at least one person image (1) from a user terminal, it can input the person image (1) into a two-dimensional image generation model (20) to train the two-dimensional image generation model (20).

[0053] For another example, the control unit (130) can generate a two-dimensional character image (2) having a full-body pose using the learned two-dimensional image generation model (20), and input the generated two-dimensional character image (2) into a three-dimensional character generation model (30) to generate a three-dimensional character model (3).

[0054] Hereinafter, a method for generating a 3D character model according to the present invention will be described in more detail based on the configuration of the 3D character model generation system (100) discussed above.

[0055] Fig. 1 is a conceptual diagram illustrating a three-dimensional character model generation system. Fig. 2 is a flowchart illustrating a three-dimensional character model generation method according to the present invention, and Figs. 3 to 8 are conceptual diagrams illustrating a three-dimensional character model generation method according to the present invention. Furthermore, Fig. 9 is a conceptual diagram illustrating one embodiment of the present invention that provides a user with an environment in which rigging and skinning operations can be performed.

[0056] In the present invention, in order to generate a character image that includes the personality of a user, a process of receiving at least one character image and training an image generation model using the character image is performed (S210, see FIG. 2).

[0057] The person image described in the present invention may include multiple images, at least one of which is not a full-body image of the user. For example, the multiple images may include at least one image of the user's face captured from various angles (e.g., above, below, left, right, etc.).

[0058] As illustrated in FIG. 3, the control unit (130) may receive a person image (301) from a user terminal and receive text (302) corresponding to the user's token. This text (302) may include at least one word or sentence entered by the user to express his or her characteristics.

[0059] In addition, the control unit (130) can train the image generation model (300) using text (302) corresponding to the user's token and multiple person images (301). The image generation model (300) can learn check points of the person image using text (302) corresponding to the user's token and multiple person images (301).

[0060] Here, the checkpoint is data in which parameter values ​​constituting the image generation model (300) are stored, and may mean data containing learning characteristics (e.g., user characteristics or user-specific data).

[0061] Furthermore, the image generation model (300) can generate a checkpoint (310) learned through checkpoint learning of a person image (301) and a user token (320) including data labeled with the user's features.

[0062] That is, the control unit (130) can cause the image generation model (300) to learn check points of a person image, and generate a user token (320) labeled with the user's characteristics together with the learned check points (310).

[0063] Meanwhile, in the present invention, a process is performed in which an image generation model learned using a text prompt and a reference image generates a character image having a full body pose from a person image (S220, see FIG. 2).

[0064] Here, a prompt is an input value for generating a response (or answer or result) from an artificial intelligence model. The prompt contains information related to instructions or commands for the artificial intelligence model, and the artificial intelligence model generates results based on this.

[0065] The text prompt described in the present invention may be stored in the 3D character model generation system (100) so that a character including the characteristics of each user can be generated even if a character image for each of a plurality of users is input into the image generation model (300) and learned.

[0066] The control unit (130) can receive user input related to character creation through a pre-formatted text prompt. For example, as illustrated in FIG. 4, the control unit (130) can receive user input regarding a token name (401) and a style name (402) included in a text prompt (410). With a simple input to the text prompt (410), the user can be provided with a user environment that allows the user to create a character corresponding to their intention.

[0067] Meanwhile, in order to perform the animation work of creating detailed movements of a 3D character model, a character must be created in which each body part is clearly distinguished so that bones corresponding to each part of the character's body can be built and connected.

[0068] In the present invention, a preprocessing step can be performed using at least one reference image in different poses so as to model a three-dimensional character in which each body part is clearly distinguished and movement is smooth.

[0069] The reference image used in the present invention is an image that serves as a reference for modeling a character in which each body part is clearly distinguished, and may be stored in advance in the 3D character model generation system (100). This reference image may include at least one image in which a different action (or pose) is taken. For example, as illustrated in FIG. 4, the reference image may include at least one of an image (411) of a T-pose or an image (412) of an A-pose.

[0070] The control unit (130) can perform data processing to enable the learned image generation model (300) to generate a two-dimensional character image having a full body pose from a person image (301) using a text prompt (410) and a reference image.

[0071] Specifically, the control unit (130) can generate a two-dimensional character image (420) having a full body pose by combining the learned checkpoint (310) and the user's token (320) with a pre-stored text prompt (410), and inputting at least one reference image among the image of the T pose (411) or the image of the A pose (412) and the combined text prompt (410) as input data of the image generation model (300).

[0072] In this way, in the present invention, by utilizing a reference image in the preprocessing process for modeling a 3D character, it is possible to create a complete 3D character model in which each part of the body is clearly distinguished and movement is smooth in the 3D character modeling process described later.

[0073] Meanwhile, in the present invention, a process of generating a 3D character model by performing volume optimization based on a character image is performed (S230, see FIG. 2).

[0074] The control unit (130) can perform volume optimization work based on a two-dimensional character image generated through the image generation model (300).

[0075] As illustrated in FIG. 5, the control unit (130) can generate a plurality of rendering images (520) based on a camera viewpoint (e.g., front view) of a two-dimensional character image (510) and other camera viewpoints (e.g., left, right, up, down, back, etc.).

[0076] In addition, the control unit (130) can perform volume optimization based on Gaussian splatting using a rendering image (520) and a two-dimensional character image (510).

[0077] Here, Gaussian splatting can be understood as a volume rendering method that roughly creates a volume by arranging color chunks that follow an elliptical Gaussian distribution in space, and then gradually converges to a fine volume according to the loss value.

[0078] The control unit (130) can perform backpropagation and volume optimization by calculating the image loss value and SDS loss value based on the rendered image (520) for each iteration.

[0079] In this case, the control unit (130) can calculate loss values ​​corresponding to the two-dimensional character image (510) and the plurality of rendered images (520) by defining them differently. At this time, referring to FIG. 6A, when calculating the two loss values, the control unit (130) can overcome the limitations of rendering (e.g., honestly rendering only color values) and minimize empty space by adding a depth value image (620) to the rendered image (610).

[0080] First, if the camera viewpoint of the input 2D character image (510) is frontal, the control unit (130) calculates and optimizes the direct RGB difference value from the image rendered at that viewpoint as a loss value, thereby maintaining the appearance of the input image. This formula can be expressed as [Mathematical Formula 1] below.

[0081] [Mathematical Formula 1]

[0082]

[0083] Next, the control unit (130) can calculate the SDS loss value using the image generation model (300) for images rendered from various viewpoints other than the camera viewpoint of the input two-dimensional character image (510). This formula can be expressed as [Mathematical Formula 2] below.

[0084] [Equation 2]

[0085]

[0086] Furthermore, as illustrated in 6b, the control unit (130) performs the tasks of optimizing the initial volume (630), optimizing the volume (640) at step 250, and optimizing the volume (650) at step 500, and then can generate a three-dimensional character model (660) from the volume (650) at step 500 using the Marching Cube algorithm.

[0087] In this way, in the present invention, by using an image generation model (300) that generates data at various points in time other than the input point in time, a 3D character model (530) with minimized empty space and a sense of volume can be generated (see FIG. 5).

[0088] Meanwhile, in the present invention, a process is performed to build a bone skeleton of a 3D model through rigging of a 3D character model, and to connect the bone skeleton to the character model through skinning (S240, see FIG. 2).

[0089] The control unit (130) can construct a bone skeleton of a 3D model through rigging of the 3D character model, and connect the bone skeleton to the 3D character model through skinning. Here, the bone constructed in the 3D model can be composed of a central bone excluding the face and hands.

[0090] As illustrated in FIG. 7a, the control unit (130) can use the rigging and skinning model (700) to build a central skeleton, excluding the face and hands, of a three-dimensional character model (710). Then, the control unit (130) can connect the central skeleton to create a rigged three-dimensional character model (720).

[0091] In this case, the control unit (130) can perform a rigging operation on the 3D character model (710) based on a predefined rigging rule. For example, as illustrated in FIG. 7B, a skeleton to be constructed in the rigging operation of the 3D character model (710) may be predefined and exist in the storage unit (120). The control unit (130) can divide the 3D character model (710) into a plurality of layers based on the predefined skeleton information (or rigging information, 730) and create a skeleton of the 3D character model (710) using the plurality of layers. More specifically, the control unit (130) can divide the 3D character model (710) into a plurality of layers based on the predefined skeleton information (or rigging information, 730) and assign (or create) skeletons (e.g., spine, arms, shoulders, legs, feet, etc.) to the divided layers. And the control unit (130) can create a rigged 3D character model (720) by connecting the created skeletons with connection points (or joint points). Meanwhile, the rigging work described above is an example, and in the present invention, the skeleton of the 3D character model (710) can be created using various rigging work methods.

[0092] Furthermore, when the rigging and skinning work for the 3D character model is completed, the control unit (130) can set a name corresponding to the animation rule to support animating the 3D character model with the skeleton connected. As illustrated in FIG. 8, the animation rule may be predefined and exist in the storage unit (120). In the present invention, the animation rule is defined in advance to assign a name (or index) to a skeleton corresponding to a body part of the 3D character model. For example, the index “4 (810a)” may be assigned to the skeleton corresponding to the first spine (Spine1, 810) of the 3D character model, and the index “5 (820a)” may be assigned to the skeleton corresponding to the second spine (Spine2, 820) different from the first spine. And, the control unit (130) can match a specific skeleton and a name set for the specific skeleton and store them in the storage unit (120).

[0093] The control unit (130) can provide a user environment in which the user can set the movement of the 3D character model as desired based on the animating rule. For example, as illustrated in FIG. 9, the control unit (130) can display a graphic object (911) corresponding to a 3D character model in a first area (910) of the user terminal, and can provide a function to set an animation effect of bones whose names are set according to animating rules in a second area (920) different from the first area (910). For example, a progress bar (921) linked to an animation effect of a specific bone (e.g., a first bone) whose name is set according to animating rules is provided in the second area (920), and the control unit (130) can set an animation effect for a specific bone based on a user input for the progress bar (921). However, this is only an example, and it is obvious that various animation methods can be used to animate the 3D character model in the present invention.

[0094] In this way, the present invention constructs distinct skeletons into a 3D character model, enabling detailed movement control for each skeleton unit. Furthermore, by storing data related to the references of each skeleton or the points of the skeleton being controlled, the stored data can be effectively utilized in constructing the skeleton of a 3D character model or controlling its movements.

[0095] As described above, the method and system for generating a 3D character model according to the present invention can generate a character including the characteristics of each user by receiving at least one human image and training an image generation model using the human image.

[0096] Furthermore, the method and system for generating a three-dimensional character model according to the present invention can generate a character image with a full-body pose from a human image using an image generation model trained using text prompts and reference images. By utilizing the reference image in the preprocessing process, the present invention can model a three-dimensional character with clearly distinguishable body parts and smooth movement.

[0097] Furthermore, the method and system for generating a three-dimensional character model according to the present invention can generate a three-dimensional character model by performing volume optimization based on a character image. The present invention utilizes an image generation model that generates data from various viewpoints different from the input viewpoint, thereby generating a character with minimized empty space and a sense of volume.

[0098] Furthermore, the method and system for creating a three-dimensional character model according to the present invention can construct a three-dimensional model's bone skeleton through rigging of the three-dimensional character model, and connect the bone skeleton to the character model through skinning. By supporting the animation of the character model, the present invention can provide a service environment in which users can immediately use the character in various contents such as games, comics, and animations.

[0099] Meanwhile, the present invention discussed above can be implemented as a program that is executed by one or more processes on a computer and can be stored on a medium (or recording medium) that can be read by the computer.

[0100] Furthermore, the present invention discussed above can be implemented as computer-readable code or instructions on a program-recorded medium. In other words, the present invention can be provided in the form of a program.

[0101] Meanwhile, computer-readable media include all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include hard disk drives (HDDs), solid-state disk drives (SSDs), silicon disk drives (SDDs), ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices.

[0102] Furthermore, the computer-readable medium may include a storage device and may be a server or cloud storage device accessible via communication. In this case, the computer may download the program according to the present invention from the server or cloud storage device via wired or wireless communication.

[0103] Furthermore, in the present invention, the computer described above is an electronic device equipped with a processor, i.e., a CPU (Central Processing Unit), and there is no particular limitation on its type.

[0104] Meanwhile, the above detailed description should not be construed as limiting in any respect and should be considered illustrative. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are intended to be included within the scope of the present invention.

Claims

1. A step of receiving at least one human image and training an image generation model using the human image; A step of generating a character image having a full body pose from the person image using the learned image generation model using a text prompt and a reference image; A step of generating a three-dimensional character model by performing volume optimization based on the above character image; and A method for creating a 3D character model based on artificial intelligence, comprising the steps of constructing a bone skeleton of the 3D character model through rigging the 3D character model and connecting the bone skeleton to the character model through skinning.

2. In paragraph 1, A method for generating an artificial intelligence-based 3D character model, characterized in that the above-mentioned character image includes multiple images having at least one image other than the user's full body.

3. In paragraph 2, The step of training the above image generation model is: An artificial intelligence-based 3D character model generation method characterized by learning check points of the character image using text corresponding to the user's token and the plurality of images.

4. In paragraph 3, The above reference image includes an image of T pose or A pose, The steps for creating the above character image are: An artificial intelligence-based 3D character model generation method characterized by combining the learned checkpoint and the user's token with the previously stored text prompt and generating the character image using the image of the T pose or A pose.

5. In paragraph 1, The steps for creating the above 3D character model are: A step of generating multiple rendering images based on a camera viewpoint different from the camera viewpoint of the above character image; and An artificial intelligence-based 3D character model generation method, comprising a step of performing volume optimization based on Gaussian splatting using the above-mentioned rendering image and the above-mentioned character image.

6. In paragraph 5, The steps for performing the above volume optimization are: A method for generating a 3D character model based on artificial intelligence, characterized in that loss values ​​corresponding to the character image and the plurality of rendered images are defined differently and calculated.

7. In paragraph 1, A method for generating an artificial intelligence-based 3D character model, characterized in that the above skeleton is composed of a central skeleton excluding the face and hands.

8. In paragraph 7, An artificial intelligence-based 3D character model generation method further comprising a step of setting a name corresponding to a rule of the animation to support animating the character model to which the above skeleton is connected.

9. A communication unit receiving at least one human image; A storage unit in which the above-mentioned person image, an image generation model learned using the above-mentioned person image, and a program code are stored; and A control unit for generating a three-dimensional character model using the storage unit, the image generation model, and the program code, The above learned image generation model generates a character image with a full body pose from the person image using a text prompt and a reference image, The above control unit, Using the above human image, the image generation model is trained, A 3D character model is created by performing volume optimization based on the above character image, An artificial intelligence-based 3D character model creation system characterized by constructing a bone skeleton of the 3D model through rigging of the 3D character model and connecting the bone skeleton to the character model through skinning.

10. In paragraph 10, The above control unit, Animating is performed using the character model to which the skeleton is connected to create animation content for the entire body of the subject included in the above-mentioned person image, A system for generating a three-dimensional character model, characterized in that it outputs the generated animated content using a display.

11. A program that is executed by one or more processes on an electronic device and stored on a computer-readable recording medium, The above program is, A step of receiving at least one human image and training an image generation model using the human image; A step of generating a character image having a full body pose from the person image using the learned image generation model using a text prompt and a reference image; A step of generating a three-dimensional character model by performing volume optimization based on the above character image; and A program stored in a computer-readable recording medium, characterized in that it includes commands for constructing a bone skeleton of the three-dimensional character model through rigging of the three-dimensional character model and performing a step of connecting the bone skeleton to the character model through skinning.

Images