Learning method, program and device for generative artificial intelligence model

Abstract

To enable a generative artificial intelligence model to generate output data with consistent characteristics as intended by a user.SOLUTION: There is provided a learning method for a generative artificial intelligence model. The learning method which is executed by a processor of a server includes the steps of: collecting original data of a cartoonist; preparing valid data by removing noise from the original data; performing a first style learning step of distinguishing and learning color information and line information among the valid data; and generating a first bias model by merging training data derived from the first style learning step again.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a method and device for training a generative artificial intelligence model, and more particularly to a method, program and device for training an artificial intelligence model to be used as a more predictable tool by ensuring consistency in output data output by a generative artificial intelligence model. [Background technology]

[0002] In recent years, many people have been generating pictures through diffusion models, which are generative artificial intelligence models, without having to draw pictures themselves. These diffusion models have a huge amount of learning, and additional learning data from various people is applied through an open source policy. As a result, the quality of images generated through diffusion models is becoming increasingly advanced.

[0003] However, when such a generative AI model generates multiple images, it only randomly generates a variety of images and does not generate a consistent image as intended by the user, so the user must make many attempts to obtain the desired image.

[0004] In addition, in recent years, the comic market is expanding as people's demand for webtoons, which are digital comics serialized online, as well as comics serialized offline, increases. Generally, comic artists who serialize webtoons create webtoons at regular intervals (e.g., once a week, twice a week, etc.) and upload them online. In recent years, as the webtoon market expands, general consumers' expectations for the quality of webtoons have also increased, making it difficult to create webtoons of a quality that meets expectations at regular intervals. Therefore, attempts have been made to create comics using generative artificial intelligence models, but generative artificial intelligence models only output randomly generated images and cannot generate consistent images, making it difficult to meet the essential requirement of comics, which is to generate consistent images that match the artist's style.

[0005] Therefore, when generating images using a generative artificial intelligence model, a need has arisen for a method of training the generative artificial intelligence model to generate consistent images that reflect the user's intent. Summary of the Invention [Problem to be solved by the invention]

[0006] The present invention has been devised to solve the above-mentioned problems, and its object is to provide a learning method and device for a generative artificial intelligence model so that when the generative artificial intelligence model outputs output data, it generates output data having consistent characteristics as intended by the user.

[0007] The technical problems to be achieved by the present invention are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by a person having ordinary skill in the art to which the present invention pertains from the following description. [Means for solving the problem]

[0008] A method for training a generative artificial intelligence model executed by a processor of a server according to one embodiment of the present invention may include a step of collecting original data of a cartoonist, a step of removing noise from the plurality of original data to prepare valid data, a first style training step of distinguishing and training color information and line information from the valid data, and a step of merging the training data derived from the first style training step again to generate a first biased model.

[0009] In one embodiment, the valid data may include a tag that indicates an image that is included in the valid data.

[0010] In one embodiment, the first style learning step may include a step of distinguishing, from the valid data, color data corresponding to the color information and line drawing data corresponding to the line information, and a step of learning the color data and the line drawing data separately to derive learning data.

[0011] In one embodiment, distinguishing between the colored data and the line drawing data may further include extracting at least one of the colored data and the line drawing data from the valid data.

[0012] In one embodiment, the step of generating the first biased model may include a step of generating the first biased model by combining first trained data calculated by training the color data and second trained data calculated by training the line drawing data.

[0013] In one embodiment, the step of merging the first learning data and the second learning data may include a step of assigning a first weighting value to the first learning data, a step of assigning a second weighting value to the second learning data, and a step of merging the first learning data and the second learning data to which the weighting values ​​have been assigned.

[0014] In one embodiment, the method may further include a second style learning step of selectively learning only data including background information from the valid data, and a step of generating a second biased model using learning data derived from the second style learning step.

[0015] In an embodiment, the second style learning step may include the steps of obtaining background data corresponding to the background information from the valid data, and learning the background data.

[0016] In one embodiment, the method may further include merging the first bias model and the second bias model.

[0017] In one embodiment, the method may further include a third style learning step of selectively learning only data including personal information from the valid data, and a step of generating a third biased model using learning data derived from the third style learning step.

[0018] In an embodiment, the third style learning step may include obtaining person data corresponding to the person information from the valid data, and learning the person data.

[0019] In one embodiment, the method may further include the steps of selecting an engine based on a similarity to the original data of the cartoonist, testing the first to third biased models using the selected engine, and generating a fourth biased model using the first to third biased models that have been tested and the engine.

[0020] In one embodiment, the step of testing the first to third biased models may further include the steps of generating a plurality of result images using the selected engine and the first to third biased models, and comparing the plurality of result images with the original data of the cartoonist.

[0021] In one embodiment, the step of generating a plurality of result images using the selected engine and the first to third deflection models may include the step of generating a plurality of result images in the selected engine using a combined model in which the first deflection model and the second deflection model are combined, and the third deflection model in a pre-set ratio.

[0022] In one embodiment, comparing the plurality of result images with the original data may include calculating a similarity between the original data and the entire plurality of result images.

[0023] In one embodiment, the method may further include adjusting each weight assigned to the training data when the similarity is less than a pre-set value; selecting only result images whose similarity is equal to or greater than the pre-set value from the plurality of result images; generating additional training data using the selected result images; merging the additional training data into at least one of the merged model and the third biased model; and re-testing the modified merged model using the selected engine.

[0024] In one embodiment, if the similarity is greater than or equal to a predefined value, generating the fourth biased model may include merging the merged model and the third biased model into the engine.

[0025] According to one embodiment of the present invention, there is provided a computer program stored in a computer-readable recording medium, and when the computer program is executed by a processor of a device, a step of training a generative artificial intelligence model by the processor of the device may include a step of collecting original data of a cartoonist, a step of removing noise from the plurality of original data to prepare valid data, a first style training step of distinguishing and training color information and line information from the valid data, and a step of merging the training data derived from the first style training step again to generate a first biased model.

[0026] An apparatus for training a generative artificial intelligence model according to one embodiment of the present invention includes a storage unit storing at least one program instruction, and a processor for executing the at least one program instruction, wherein the processor collects original data of a cartoonist, removes noise from the plurality of original data to prepare valid data, performs a first style learning step in which color information and line information from the valid data are separated and learned, and merges the training data derived from the first style learning step again to generate a first biased model.

[0027] The above-described aspects of the present disclosure are merely some of the preferred embodiments of the present disclosure, and various embodiments reflecting the technical features of the present disclosure can be derived and understood by a person having ordinary skill in the art based on the detailed description of the present disclosure described below. Effect of the Invention

[0028] According to an embodiment of the present invention, in each style learning step, the server distinguishes at least two pieces of information from the available data, so that the server can learn the data corresponding to each piece of information separately. This allows the server to learn the characteristics of each piece of information more accurately than when learning all pieces of information together. Therefore, when the server generates content, an image that reflects the characteristics of the cartoonist (e.g., drawing style, characters, background information, etc.) can be generated.

[0029] When the final generated biased model is used, the server can consistently generate images that reflect the characteristics of the cartoonist's drawings. Therefore, when the cartoonist selects a biased model from the server and inputs a prompt indicating the desired characters, background, etc. in the corresponding scene, the server can generate an image of the finished book that reflects the cartoonist's drawing style. Therefore, the cartoonist only needs to create the prompt, which dramatically reduces the time and cost required for content production.

[0030] The effects obtained by the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those having ordinary skill in the art to which the present invention pertains from the following description. [Brief description of the drawings]

[0031] [Figure 1] FIG. 1 illustrates an exemplary network environment associated with a generative artificial intelligence model learning system according to an embodiment of the present invention. [Diagram 2] A block diagram showing an example of the configuration of devices that make up a system for learning a generative artificial intelligence model. [Diagram 3] A procedure diagram showing one embodiment of a method for training a generative artificial intelligence model according to an embodiment of the present invention. [Figure 4] FIG. 4 is a flow chart showing an example of a first style learning step and a first bias model generating step according to an embodiment of the present invention. [Diagram 5] FIG. 11 is a flow chart showing an example of a test step and a fourth deflection model generation step according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0032] Since the present invention can be modified in various ways and has various embodiments, specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to the specific embodiments, and it should be understood that the present invention includes all modifications, equivalents, and alternatives included in the spirit and technical scope of the present invention.

[0033] Terms such as first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only to distinguish one component from another. For example, a first component may be named a second component, and similarly, the second component may be named a first component, without departing from the scope of the invention. The term "and / or" includes a combination of multiple associated listed items or any items among multiple associated listed items.

[0034] The singular includes the plural unless the context clearly dictates differently. That is, unless otherwise specified or the context is clear as dictating the singular form, in this disclosure and claims, the singular should generally be construed to mean "one or more."

[0035] When a component is referred to as being "coupled" or "connected" to another component, it should be understood that it may be directly coupled or connected to the other component, but that there may be other components between them. Conversely, when a component is referred to as being "directly coupled" or "directly connected" to another component, it should be understood that there are no other components between them.

[0036] The terms used in this application are merely used to describe certain embodiments and are not intended to limit the present invention. A singular expression includes a plural expression unless the context clearly indicates otherwise. In this application, the terms "include" or "have" are intended to specify the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, and should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0037] As used in this disclosure, the terms "information" and "data" may be used interchangeably. Similarly, as used in this disclosure, the terms "picture" and "image" may be used interchangeably. As used in this disclosure, "content" refers to various information and matters provided via the Internet, computer communication, etc., and may include pictures, images, videos, text, etc.

[0038] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which the present invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning they have in the context of the relevant art, and should not be interpreted as idealized or overly formal unless expressly defined in this application.

[0039] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In describing the present invention, the same reference numerals are used to refer to the same components in the drawings in order to facilitate overall understanding, and duplicate descriptions of the same components will be omitted.

[0040] FIG. 1 is a diagram illustrating an exemplary network environment associated with a generative artificial intelligence model learning system according to an embodiment of the present invention.

[0041] Referring to FIG. 1, the generative artificial intelligence model learning system may include a server 110 and a user terminal 130 connected to the server 110 via a network 120 .

[0042] The network 120 may refer to a connection structure for information exchange between the server 110 and the user terminal 130. The network 120 may include the Internet, a local area network (LAN), a wireless local area network (Wireless LAN), a wide area network (WAN), a personal area network (PAN), 3G, 4G, long term evolution (LTE), voice over LTE (VoLTE), 5G new radio (NR), wireless fidelity (Wi-Fi), Bluetooth (registered trademark), NFC, radio frequency identification (RFID), a home network, and the Internet of things (IoT).

[0043] The server 110 may include an artificial intelligence module that includes a generative artificial intelligence model. A generative artificial intelligence model refers to an artificial intelligence that generates a result according to a specific request of a user. Thus, when a user of the server 110 inputs an instruction to the server 110, the server 110 can generate content according to the instruction. For example, when a user inputs an instruction to the server 110 such as "generate a picture of a woman walking on a road," the server 110 can generate a picture of a woman walking on a road.

[0044] In addition, the server 110 can train the generative artificial intelligence model. Specifically, the server 110 can train the generative artificial intelligence model or a sub-model of the generative artificial intelligence model. Here, the generative artificial intelligence model means a model that is essential when generating content, and the sub-model of the generative artificial intelligence model means a model that is not essential when generating content but assists the generative artificial intelligence model when generating content. In an embodiment of the present invention, the server 140 can reinforce the generative artificial intelligence model through the sub-model and generate content having consistent characteristics that reflect the user's intention.

[0045] In addition, a sub-model of the generative artificial intelligence model may include input data, which is basic data provided to obtain output from the server, input tools that interact with the generative artificial intelligence model, and weight adjustment tools.

[0046] Specifically, the sub-model of the generative artificial intelligence model may include a biased model that interacts with the generative artificial intelligence model to generate content according to an instruction (input data) input by a user or an instruction input by a user. In one embodiment, the biased model may be an input tool that is input together with the input data and affects the weight value of the generative artificial intelligence model. In another embodiment, the biased model may be a layer within the generative artificial intelligence model. In the following, it is assumed that the generative artificial intelligence model includes a generative artificial intelligence model and a sub-model of the generative artificial intelligence model.

[0047] The server 110 can continuously train the generative artificial intelligence model and test the trained model in order to optimize the content generated by the generative artificial intelligence model according to the user's intention. The server 110 can also store an algorithm for training the generative artificial intelligence model in advance.

[0048] The user terminal 130 may be connected to the server 110 via the network 120. The user terminal 130 may include user equipment such as a computer, a tablet, or a smartphone. The user terminal 130 may receive content desired by a user via the server 110. The user terminal 130 may display content generated via the server 110 using an output device.

[0049] The server 110 and the user terminal 130 in FIG. 1 can be referred to as devices, and the configuration of the devices is as described below.

[0050] FIG. 2 is a block diagram showing an embodiment of the configuration of devices that make up a system for learning a generative artificial intelligence model.

[0051] The device 200 included in the system may include a memory 220 having at least one program instruction stored therein and at least one processor 210 for executing the at least one program instruction.

[0052] Here, the at least one processor 210 may refer to a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor on which the method according to the embodiment of the present invention is performed. Each of the memory 220 and the auxiliary storage device 260 may be composed of at least one of a temporary recording medium and a non-temporary recording medium. For example, the memory 220 may be composed of at least one of a read only memory (ROM) and a random access memory (RAM).

[0053] The device 200 included in the system may also include a transceiver 230 that communicates via a wireless network. The device 200 included in the system may also include an input interface unit 240, an output interface unit 250, and an auxiliary storage device 260. The components included in the device 200 are connected by a bus 270 and can communicate with each other.

[0054] The algorithms and operations performed by the server of the system for training a generative artificial intelligence model are as follows.

[0055] FIG. 3 is a flow chart showing one embodiment of a method for training a generative artificial intelligence model according to an embodiment of the present invention.

[0056] In step S301, the server can collect original data of the cartoonist. In the following, the cartoonist means a person who draws pictures as a profession. Also, the original data of the cartoonist means data including pictures directly drawn by the cartoonist. The server can obtain the original data of the cartoonist by receiving the original data from the cartoonist or by collecting the original data of the cartoonist from the web.

[0057] In this case, the server may collect a preset number of original data of the cartoonist. For example, the preset number may be about 100 or more. That is, the server may collect about 100 or more pictures drawn by the cartoonist. However, the present invention is not limited to this.

[0058] In step S303, the server may prepare valid data by removing noise from the plurality of original data. Specifically, the server may remove the original data itself including noise from the plurality of original data, or may remove noise from the original data including noise. Here, noise may refer to elements that hinder learning. For example, noise may refer to special light effects and blur processing effects that cartoonists include in their drawings.

[0059] The valid data may include at least one tag indicating an image included in the valid data. That is, when the valid data is prepared, a tag corresponding to the valid data may be set. The tag may represent one or more natural languages ​​indicating the image. In this case, the natural language may include a variety of languages.

[0060] For example, if an image included in the valid data is a picture of a woman walking on a road, the valid data may include tags such as woman, on road, and walking that correspond to the image. By setting tags to the corresponding image, the server can learn more quickly what information the image contains and apply it accurately.

[0061] In step S305, the server may perform a first style learning step of classifying and learning color information and line information from the valid data. The first style learning step may refer to a step of learning a cartoonist's drawing style.

[0062] In step S307, the server may generate a first biased model using the learning data derived from the first style learning step. The first biased model generated in the first style learning step may be a tool for generating an image that primarily reflects the characteristics of the cartoonist's drawing style. In particular, the first style learning step and the first biased model generating step are described below.

[0063] FIG. 4 is a flow chart showing an example of the first style learning step and the first bias model generating step according to an embodiment of the present invention.

[0064] Referring to FIG. 4, in step S401, the server can distinguish between color data corresponding to color information and line drawing data corresponding to line information from valid data.

[0065] In one embodiment, the server can collect color data corresponding to color information and line drawing data corresponding to line information separately within the valid data. In another embodiment, the server can collect color data corresponding to colored images and line drawing data corresponding to uncolored images that only have line drawings separately within the valid data. In yet another embodiment, the server can collect color data corresponding to color images and line drawing data corresponding to black and white images separately within the valid data.

[0066] In yet another embodiment, the server can extract and collect at least one of color data and line art data from the available data. For example, if the amount of line art data is small compared to the amount of color data, the server can extract the line art data from the available data. This allows the server to further secure the amount of line art data.

[0067] In step S403, the server may derive learning data by separately learning the color data and the line drawing data, i.e., the server may derive first learning data calculated by learning the color data and second learning data calculated by learning the line drawing data, separately.

[0068] In step S405, the server may generate a first biased model by merging the training data (the first training data and the second training data) derived from the first style training step again. That is, the server may generate a first biased model by merging the first training data and the second training data.

[0069] When the server merges the first learning data and the second learning data, the server can assign a first weighting value to the first learning data and a second weighting value to the second learning data. The server can set weighting values ​​for the color data and the line drawing data according to their respective importance. The server can merge the first learning data and the second learning data to which the weighting values ​​have been assigned. Using such weighting values, the server can set which information should be weighted when the generative artificial intelligence model generates content. That is, even if the same learning data is used to generate a first biased model, different first biased models can be generated depending on the first weighting value and the second weighting value.

[0070] In addition, in the first style learning stage, the server can classify and learn information other than color information and line information from the valid data. For example, the server can classify and learn color information, line information, and post-processing information from the valid data. Here, the post-processing information may mean information included as background effects in a picture drawn by a cartoonist. That is, the server can distinguish effect data corresponding to color data, line drawing data, and post-processing information from the valid data. In addition, the server can derive learning data by learning each of the color data, line drawing data, and effect data individually. As a result, the server can generate a first biased model by merging the third learning data calculated by learning the first learning data, the second learning data, and the effect data derived from the first style learning stage.

[0071] In one embodiment, in the first style learning step, the server distinguishes at least two pieces of information including color information and line information from the available data, so that the server can learn data corresponding to each piece of information separately. This allows the server to learn the characteristics of each piece of information more accurately than when learning all pieces of information together. Therefore, when the server generates content using the learned generative artificial intelligence model, an image that reflects the characteristics (e.g., the drawing style) of the cartoonist can be generated.

[0072] 3 again, in step S309, the server may perform a second style learning step of selectively learning only data including background information from the valid data. The server may obtain background data corresponding to the background information from the valid data and learn the obtained background data.

[0073] In this case, the background information may refer to information on a part of the cartoonist's drawing that is included in the valid data and that is not directly drawn by the cartoonist. In other words, the background information may refer to information on a part of the cartoonist's drawing that is included in the valid data and that does not reflect the cartoonist's style alone. For example, the background information may refer to a part of the cartoonist's drawing that is not directly drawn by the cartoonist but is included as an object or background by processing a photograph or a 3D image. Therefore, the second style learning stage may refer to a background style learning stage that is not directly drawn by the cartoonist but is included in the cartoonist's drawing.

[0074] The server can extract only the background information portion from the valid data and learn it, or the server can selectively learn only the valid data including the background information from the valid data.

[0075] In step S311, the server may generate a second biased model using the learning data derived from the second style learning step. The second biased model generated in the second style learning step may be a tool for generating an image that focuses on the background characteristics of the cartoonist's drawings included in the valid data.

[0076] In step S313, the server may merge the first biased model and the second biased model. That is, the server may merge the first biased model and the second biased model to generate a merged model. At this time, the server may merge the first biased model and the second biased model by assigning a weight to each of them. That is, the server may set a weight to each of the first biased model for the cartoonist's style and the second biased model for the background that is not the cartoonist's style, depending on the importance of each of them. Thus, when generating content, the server may set which of the style or the background to be weighted. That is, even if the same first biased model and second biased model are merged, different first biased models may be generated depending on the weight.

[0077] In step S315, the server may perform a third style learning step of selectively learning only data including person information from the valid data. The server may obtain person data corresponding to the person information from the valid data and learn the obtained person data.

[0078] In this case, the character information may refer to information on the part corresponding to the character (e.g., the character) in the cartoonist's drawing included in the valid data. For example, the character may refer to the person, animal, object, etc. that mainly appears in the cartoonist's drawing included in the valid data. Therefore, the third style learning stage may refer to the character style learning stage included in the cartoonist's drawing included in the valid data.

[0079] The server can extract only the personal information portion from the valid data and learn it. Alternatively, the server can selectively learn only the valid data including the personal information from the valid data.

[0080] In addition, the server can separately collect gender data corresponding to the gender information and costume data corresponding to the costume information in the person information in the valid data. For example, the server can extract and collect gender data and costume data from the valid data. The server can learn the gender data and the costume data separately to derive learning data.

[0081] In step S317, the server may generate a third biased model using the learning data derived from the third style learning step. The third biased model generated in the third style learning step may be a tool for generating an image that primarily reflects the characteristics of the characters in the cartoonist's drawings included in the valid data.

[0082] In this case, the server may generate a third biased model by merging the learning data derived by learning the gender data and the costume data separately. However, the present invention is not limited to this, and the server may learn the gender data and the costume data from the person information in an integrated manner, rather than learning them separately.

[0083] In one embodiment, the server generates a biased model by separately learning the style, character, and background information of the cartoonist's drawing included in the valid data in the first to third style learning steps, so that the server can learn the data corresponding to each information separately. This allows the server to learn the characteristics of each information more accurately than when learning all information together. In addition, when a user generates content using the first to third biased models, the user can determine which information to focus on among the style, character, and background when generating content, and request the server to provide the content accordingly. Therefore, when the server generates content using a biased tool that is a generative artificial intelligence model, an image reflecting the characteristics of the cartoonist (e.g., the cartoonist's style, character, and background) can be generated as intended by the user.

[0084] In step S319, the server may select an engine based on the similarity with the cartoonist's original data. In this case, the engine may be a generative artificial intelligence model or a sub-model that is conventionally present in the server. The server may select an engine that generates an image similar to the cartoonist's original data from among various engines. That is, the server may measure the similarity between the image generated using the engine and the original data. In this case, if the similarity is equal to or greater than a pre-set reference value, the server may select the corresponding engine. For example, the pre-set reference value may be about 70% or more. Preferably, the server may select an engine that has the highest similarity between the image generated using the engine and the original data.

[0085] In addition, the server may test the first to third biased models using the selected engine. By testing the first to third biased models, the server may improve the completeness of the biased models.

[0086] In step S321, the server may generate a fourth biased model using the first to third biased models that have been tested and the selected engine. The fourth biased model thus generated may generate content as a generative artificial intelligence model. Specifically, the fourth biased model may be a generative artificial intelligence model, and each of the first to third biased models may be a sub-model of the generative artificial intelligence model. However, the present invention is not limited thereto.

[0087] Specifically, the steps of testing the first to third biased models and generating the fourth biased model are as follows.

[0088] FIG. 5 is a flow chart illustrating an example of the testing step and the fourth bias model generating step according to an embodiment of the present invention.

[0089] Referring to FIG. 5, in step S501, the server may generate a merged model by merging a first biased model and a second biased model.

[0090] In step S503, the server may generate a plurality of result images using the engine selected based on the similarity of the cartoonist and the first to third biased models. Specifically, the server may generate a plurality of result images using the selected engine by using the merged model and the third biased model in a preset ratio. Preferably, the preset ratio may be a ratio of merged model:third biased model=1:0.2. However, the present invention is not limited thereto.

[0091] In this case, using the merged model and the third biased model in a preset ratio does not mean merging the merged model and the third biased model, but means using the merged model and the third biased model in parallel in a preset ratio. By using the merged model and the third biased model in parallel without merging them, the server can make the character information (i.e., the characters) more prominent than other information (e.g., the style and background) in the content generated using the learned generative artificial intelligence model. However, the present invention is not limited thereto, and in other embodiments, the server can use each of the first to third biased models in a preset ratio. In still another embodiment, the server can merge and use the first to third biased models according to a weighted value.

[0092] In step S505, the server may compare the plurality of result images with the cartoonist's original data, i.e., the server may calculate a similarity between the cartoonist's original data and the plurality of result images as a whole.

[0093] In step S507, if the similarity is less than a previously set value, the server can adjust the weights assigned to the learning data. That is, the server can adjust the weights assigned to the first learning data and the second learning data when generating the first biased model and the weights assigned to the first biased model and the second biased model when generating the merged model. The server can also adjust the parallel use ratio of the merged model and the third biased model. The server can improve the completeness of the result image by adjusting the weights or ratios.

[0094] In step S509, the server may also select only result images having a similarity equal to or greater than a preset value from among the plurality of result images (for example, the similarity may be about 70%), although the present invention is not limited thereto.

[0095] In step S511, the server may generate additional learning data using the selected result images. The server may generate additional learning data based on result images having a similarity equal to or greater than a previously set value among a plurality of result images, and may re-learn using only highly complete data or merge the data into a merged model (step S513).

[0096] Specifically, in one embodiment, the server can reuse the selected result image as valid data. Thus, the server can increase the amount of valid data by including the selected result image as valid data. Also, the server can improve the completeness of the bias model by re-learning using only highly complete data.

[0097] In another embodiment, the server may merge additional training data into at least one of the merged model and the third biased model. That is, the server may modify the merged model or the third biased model by merging additional training data generated with a highly complete image into the merged model or the third biased model. After modifying the merged model or the third biased model, the server may retest the modified merged model or the third biased model using the selected engine. That is, the server may regenerate a plurality of result images using the selected engine and the modified merged model or the third biased model.

[0098] At this time, if the similarity is less than the previously set value, steps S507 to S513 can be repeated again. Through this process, the server can generate more complete first to third bias models.

[0099] In step S515, if the similarity is equal to or greater than the previously set value, the server may end the test for the first to third biased models. The server may generate a fourth biased model using the first to third biased models that have been tested and the engine. Specifically, the server may generate a fourth biased model by merging the merged model and the third biased model that have been tested into the engine. In this case, the server may assign weights to each of the merged model, the third biased model, and the engine to merge them.

[0100] The generated fourth bias model is a generative artificial intelligence model that can generate consistent images that reflect the characteristics of a manga artist.

[0101] In one embodiment, the finally generated fourth biased model may be generated by combining the first to third biased models that reflect the characteristics of the cartoonist's drawings with an engine similar to the cartoonist's original data. As a result, when generating an image using the fourth biased model, the server can generate content, i.e., an image, reflecting the characteristics of the cartoonist's drawings even if there is relatively little data.

[0102] In addition, when the fourth bias model is used, the server can consistently generate images that reflect the characteristics of the cartoonist's drawings. Therefore, when the cartoonist selects the fourth bias model from the server and inputs a prompt specifying the desired characters, background, etc. in the corresponding scene, the server can generate an image of the finished book that reflects the cartoonist's style. Therefore, the cartoonist only needs to create a storyboard, which dramatically reduces the time and cost required for content production.

[0103] When the server uses the fourth biased model, the server can receive a storyboard created by a cartoonist and output an image that has been sketched and colored as output data. In this case, the server can output an image that has only been sketched and an image that has only been sketched and colored separately according to the cartoonist's intention. In addition, after the server outputs an image that has only been sketched according to the cartoonist's intention, the server can use the image that has only been sketched together with the biased model to output an image that has been colored again as output data.

[0104] In conclusion, cartoonists can use images that have been colored as the final image. If cartoonists want to add or modify parts of the created image, they can create the final image by modifying the image that has only been sketched or colored. This dramatically reduces the time and cost required for cartoonists to create content.

[0105] The methods according to the present invention may be embodied in the form of program instructions executed by various computer means and stored on a computer-readable medium. The computer-readable medium may include, alone or in combination with, program instructions, data files, data structures, and the like. The program instructions stored on the computer-readable medium may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well known and available to those skilled in the computer software arts.

[0106] Examples of computer readable media include hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine code, such as produced by a compiler, as well as higher level language code that may be executed by a computer using an interpreter, etc. The above-mentioned hardware devices may be configured to operate with at least one software module to perform the operations of the present invention, and vice versa.

[0107] Although the present invention has been described with reference to the embodiments, it will be understood that those skilled in the art can make various modifications and variations of the present invention without departing from the spirit and scope of the present invention as set forth in the claims below.

Claims

1. A method for training a generative artificial intelligence model executed by a processor of a server, comprising: The stage of collecting the original data of the manga artist, removing noise from the plurality of original data to prepare valid data; a first style learning step of learning color information and line information of the valid data separately; generating a first biased model by recombining the training data derived from the first style training step; The method includes:

2. the valid data includes a tag indicating an image included in the valid data; The method of claim 1.

3. The first style learning stage includes: distinguishing from the valid data color data corresponding to the color information and line drawing data corresponding to the line information; A step of deriving learning data by separately learning the color data and the line drawing data; Including, The method of claim 1.

4. The step of distinguishing between the color data and the line drawing data comprises: extracting at least one of said color data and said line drawing data from said valid data; Further comprising: The method according to claim 3.

5. The step of generating the first deflection model comprises: generating the first biased model by combining first trained data calculated by training the color data and second trained data calculated by training the line drawing data; Including, The method according to claim 3.

6. The step of merging the first learning data and the second learning data includes: assigning a first weight to the first training data; assigning a second weight to the second training data; Merging the first training data and the second training data to which the weights have been assigned; Including, The method according to claim 5.

7. a second style learning step of selectively learning only data including background information from the valid data; generating a second biased model using training data derived from the second style training step; The method of claim 6 further comprising:

8. The second style learning step includes: obtaining background data corresponding to the background information from the valid data; learning said background data; Including, The method according to claim 7.

9. merging the first deflection model and the second deflection model; The method of claim 8 further comprising:

10. a third style learning step of selectively learning only data including person information from the valid data; generating a third biased model using training data derived from the third style training step; 10. The method of claim 9 further comprising:

11. The third style learning stage includes: obtaining person data corresponding to the person information from the valid data; learning said person data; Including, The method of claim 10.

12. selecting an engine based on a similarity to the original data of the cartoonist; testing the first through third bias models using the selected engine; generating a fourth biased model using the first to third biased models that have been tested and the engine; The method of claim 11 further comprising:

13. The step of testing the first to third deflection models includes: generating a plurality of result images using the selected engine and the first, second, third deflection models; comparing the plurality of resultant images with the original data of the cartoonist; Further comprising: The method of claim 12.

14. generating a plurality of result images using the selected engine and the first, second, third deflection models, generating a plurality of result images in the selected engine using a merged model obtained by merging the first and second biased models and the third biased model in a preset ratio; Including, The method of claim 13.

15. The step of comparing the plurality of result images with the original data includes: calculating a similarity between the original data and the entirety of the plurality of result images; Including, The method of claim 14.

16. If the similarity is less than a previously set value, adjusting the weights assigned to the training data; selecting only result images having a similarity equal to or greater than a preset value from the plurality of result images; generating additional training data using the selected result images; merging the additional training data into at least one of the merged model and the third biased model; retesting the modified merged model with the selected engine; Further comprising: The method of claim 15.

17. If the similarity is equal to or greater than a preset value, generating the fourth bias model comprises: merging the merged model and the third biased model into the engine; Including, 17. The method of claim 16.

18. A computer program stored on a computer-readable recording medium, the computer program, when executed by a processor of a device, performs a step of learning a generative artificial intelligence model by the processor of the device, the step comprising: The stage of collecting the original data of the manga artist, removing noise from the plurality of original data to prepare valid data; a first style learning step of learning color information and line information of the valid data separately; generating a first biased model by recombining the training data derived from the first style training step; A computer program stored on a computer-readable recording medium, comprising:

19. An apparatus for training a generative artificial intelligence model, the apparatus comprising: a storage unit having at least one program instruction stored therein; a processor for executing said at least one program instruction; Including, The processor, Collecting original data from manga artists, removing noise from the plurality of original data to prepare valid data; performing a first style learning step of classifying and learning color information and line information of the valid data; recombining the training data derived from the first style training stage to generate a first biased model; Device.

Images