Method, computer device, and computer program for generating more realistic and natural image by using generative model
By integrating a similarity criterion and data augmentation within a generative model, the method enhances image quality and realism, addressing the limitations of existing generative models by ensuring high similarity and optimal parameter adjustment, resulting in high-quality, natural images.
Patent Information
- Application Number
- PCT/KR2024/006806
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-31
- Filing Date
- 2024-05-20
- Publication Date
- 2024-12-05
AI Technical Summary
Current generative models often produce low-quality images that deviate significantly from the user's actual appearance, limiting their realism and naturalness, especially when used for creating avatars based on facial images.
The proposed method combines a similarity criterion with a generative model to ensure high-quality image generation by refining the model through the removal of minor images and data augmentation, using a computer device with a processor to learn and generate images that meet a similarity threshold, and dynamically adjusting model parameters based on past experience to optimize results.
This approach generates more realistic and natural images by ensuring a certain level of similarity with the input image, minimizing computation loss through optimal parameter presets and improving model learning quality, resulting in high-quality images that meet user expectations.
Smart Images

Figure KR2024006806_05122024_PF_FP_ABST
Abstract
Description
Method, computer device, and computer program for generating more realistic and natural images using generative models
[0001] The description below describes a technique for generating images using a generative model.
[0002] The recent rapid growth of artificial intelligence (AI) and deep learning has given rise to unprecedented new technologies. Generative models are a prime example.
[0003] There are various types of generative models, such as GAN (generative adversarial network) and VAE (variational autoencoder). These generative models are models that have the ability to learn and generate rules within data, such as images or audio, for which it is difficult to mathematically define a probability distribution.
[0004] These AI-based generative models can learn from images of a given person and generate various images of the same person.
[0005] For example, Korean Patent Publication No. 10-2020-0127917 (published on November 11, 2020) discloses a technology for learning a facial animation generation model.
[0006] By combining similarity criteria with generative models that generate person images, high-quality images of the same person can be generated.
[0007] Based on past experience with image generation, we can recommend learning parameter presets for the generative model.
[0008] Minor images included in the input image can be removed and augmented images for model learning can be generated based on the remaining images.
[0009] In an image generation method executed on a computer device, the computer device includes at least one processor configured to execute computer-readable instructions contained in a memory, and the image generation method includes: generating, by the at least one processor, a new image of the same person as the input image through a generative model for an input image according to an image generation request; and providing, by the at least one processor, the new image as a response image to the image generation request when a similarity between the input image and the new image is greater than or equal to a threshold value.
[0010] According to one aspect, the generating step may include a step of selecting a noisy image from among the input images; and a step of removing the noisy image and using the remaining images as learning target images to train the generating model.
[0011] According to another aspect, the selecting step may include: extracting a face feature from each of the input images; calculating a feature centroid by averaging the face features for each of the input images; and determining the noise image by comparing the face feature of each of the input images with the feature centroid.
[0012] According to another aspect, the learning step may include a step of generating an augmented image through data augmentation using the learning target image; and a step of learning the generation model using the learning target image and the augmented image.
[0013] According to another aspect, the providing step may include: extracting a facial feature from each of the input images; calculating a feature center point by averaging facial features for each of the input images; extracting a facial feature from the new image; and comparing the facial feature of the new image with the feature center point to determine the similarity.
[0014] According to another aspect, the providing step may include: extracting a facial feature from each of the learning target images; calculating a feature center point by averaging facial features for each of the learning target images; extracting a facial feature from the new image; and comparing the facial feature of the new image with the feature center point to determine the similarity.
[0015] According to another aspect, the generating step may include: generating a prompt reflecting user preferences for image generation conditions; selecting one of a plurality of model parameter presets; and training the generating model through the prompt and model parameters of the selected preset.
[0016] According to another aspect, the image generation method may further include a step of retraining the generation model by changing the model parameter preset when the similarity is less than the threshold, by the at least one processor.
[0017] According to another aspect, the relearning step may include a step of recommending one preset among the plurality of model parameter presets based on statistical information for each model parameter preset according to the similarity result for the new image.
[0018] According to another aspect, the retraining step may include: a step of storing model parameter preset information used in training the generation model that generated new images above the threshold value for a certain period of time, thereby accumulating statistical information for each model parameter preset; and a step of recommending a model parameter preset having a high probability of generating new images above the threshold value using the statistical information for each model parameter preset when the similarity is below the threshold value.
[0019] A computer program stored in a non-transitory computer-readable recording medium is provided to execute the above image generation method on the computer device.
[0020] In a computer device, a computer device is provided, comprising at least one processor configured to execute computer-readable instructions contained in a memory, wherein the at least one processor processes a process of generating a new image of the same person as the input image through a generation model for an input image according to an image generation request; and a process of providing the new image as a response image to the image generation request when the similarity between the input image and the new image is greater than a threshold value.
[0021] According to embodiments of the present invention, a high-quality image of the same person can be provided by combining a similarity criterion with a generation model to generate a person image that guarantees a similarity level above a certain level.
[0022] According to embodiments of the present invention, computational loss due to retraining can be minimized by recommending a preset that derives an optimal generation result based on past experience for the learning parameters of the generation model.
[0023] According to embodiments of the present invention, the model learning quality can be improved by removing minor images included in an input image, generating an augmented image with the remaining images, and applying the same to learning of a generative model.
[0024] FIG. 1 is a diagram illustrating an example of a network environment according to one embodiment of the present invention.
[0025] FIG. 2 is a block diagram illustrating an example of a computer device according to one embodiment of the present invention.
[0026] FIG. 3 is a flowchart illustrating an example of a method that a computer device according to an embodiment of the present invention can perform.
[0027] FIG. 4 is a diagram illustrating an example of a learning image refinement and augmentation process in one embodiment of the present invention.
[0028] FIG. 5 is a diagram illustrating an example of a similarity-based image generation process according to one embodiment of the present invention.
[0029] FIG. 6 is a diagram illustrating an example of a generative model learning process in one embodiment of the present invention.
[0030] Figure 7 illustrates an example of a learning parameter preset in one embodiment of the present invention.
[0031] FIG. 8 is a diagram illustrating an example of a learning parameter recommendation process in one embodiment of the present invention.
[0032] Figure 9 illustrates an example of the use of a generated image in one embodiment of the present invention.
[0033] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.
[0034] Embodiments of the present invention relate to a technique for generating an image using a generative model.
[0035] Embodiments including those specifically disclosed herein can generate high-quality images of the same person by incorporating a similarity criterion for the person into a generative model that generates a person image.
[0036] An image generation system according to embodiments of the present invention can be implemented by at least one computer device, and an image generation method according to embodiments of the present invention can be performed by at least one computer device included in the image generation system. At this time, a computer program according to an embodiment of the present invention can be installed and run on the computer device, and the computer device can perform the image generation method according to embodiments of the present invention under the control of the run computer program. The above-described computer program can be stored on a computer-readable recording medium so as to be combined with the computer device and cause the computer to execute the image generation method.
[0037] FIG. 1 is a diagram illustrating an example of a network environment according to an embodiment of the present invention. The network environment of FIG. 1 represents an example including a plurality of electronic devices (110, 120, 130, 140), a plurality of servers (150, 160), and a network (170). FIG. 1 is merely an example for explaining the invention, and the number of electronic devices or servers is not limited to that of FIG. 1. In addition, the network environment of FIG. 1 merely illustrates one example of environments applicable to the present embodiments, and the environments applicable to the present embodiments are not limited to the network environment of FIG. 1.
[0038] The plurality of electronic devices (110, 120, 130, 140) may be fixed terminals or mobile terminals implemented as computer devices. Examples of the plurality of electronic devices (110, 120, 130, 140) include smart phones, mobile phones, navigation systems, computers, laptops, digital broadcasting terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), tablet PCs, etc. For example, in FIG. 1, the shape of a smart phone is shown as an example of the electronic device (110), but in embodiments of the present invention, the electronic device (110) may actually mean one of various physical computer devices that can communicate with other electronic devices (120, 130, 140) and / or servers (150, 160) via a network (170) using a wireless or wired communication method.
[0039] The communication method is not limited, and may include not only a communication method that utilizes a communication network (e.g., a mobile communication network, a wired Internet, a wireless Internet, a broadcasting network) that the network (170) may include, but also short-range wireless communication between devices. For example, the network (170) may include any one or more of a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), the Internet, and the like. In addition, the network (170) may include any one or more of a network topology including, but not limited to, a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree, or a hierarchical network.
[0040] Each of the servers (150, 160) may be implemented as a computer device or multiple computer devices that communicate with multiple electronic devices (110, 120, 130, 140) via a network (170) to provide commands, codes, files, content, services, etc. For example, the server (150) may be a system that provides services (e.g., image generation services, etc.) to multiple electronic devices (110, 120, 130, 140) connected via a network (170).
[0041] FIG. 2 is a block diagram illustrating an example of a computer device according to an embodiment of the present invention. Each of the plurality of electronic devices (110, 120, 130, 140) or servers (150, 160) described above can be implemented by the computer device (200) illustrated in FIG. 2.
[0042] Such a computer device (200) may include a memory (210), a processor (220), a communication interface (230), and an input / output interface (240), as illustrated in FIG. 2. The memory (210) may be a computer-readable recording medium, and may include a random access memory (RAM), a read only memory (ROM), and a non-permanent mass storage device such as a disk drive. Here, the non-permanent mass storage device such as a ROM and a disk drive may be included in the computer device (200) as a separate permanent storage device distinct from the memory (210). In addition, an operating system and at least one program code may be stored in the memory (210). These software components may be loaded into the memory (210) from a computer-readable recording medium separate from the memory (210). Such a separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, etc. In another embodiment, the software components may be loaded into the memory (210) via a communication interface (230) other than a computer-readable recording medium. For example, the software components may be loaded into the memory (210) of the computer device (200) based on a computer program installed by files received over a network (170).
[0043] The processor (220) may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processor (220) via the memory (210) or the communication interface (230). For example, the processor (220) may be configured to execute instructions received according to program code stored in a storage device such as the memory (210).
[0044] The communication interface (230) may provide a function for the computer device (200) to communicate with other devices (e.g., the storage devices described above) via the network (170). For example, requests, commands, data, files, etc. generated by the processor (220) of the computer device (200) according to program codes stored in a recording device such as the memory (210) may be transmitted to other devices via the network (170) under the control of the communication interface (230). Conversely, signals, commands, data, files, etc. from other devices may be received by the computer device (200) via the communication interface (230) of the computer device (200) via the network (170). Signals, commands, data, etc. received via the communication interface (230) may be transmitted to the processor (220) or the memory (210), and files, etc. may be stored in a storage medium (the permanent storage device described above) that the computer device (200) may further include.
[0045] The input / output interface (240) may be a means for interfacing with an input / output device (250). For example, the input device may include a device such as a microphone, keyboard, or mouse, and the output device may include a device such as a display or speaker. As another example, the input / output interface (240) may be a means for interfacing with a device that integrates input and output functions, such as a touchscreen. The input / output device (250) may also be configured as a single device with the computer device (200).
[0046] Additionally, in other embodiments, the computer device (200) may include fewer or more components than those illustrated in FIG. 2. However, it is not necessary to explicitly illustrate most conventional components. For example, the computer device (200) may be implemented to include at least some of the input / output devices (250) described above, or may further include other components such as a transceiver, a database, etc.
[0047] Below, specific embodiments of a method and device for generating more realistic and natural images using a generative model are described.
[0048] Recently, social platforms like messengers and social networking services (SNS) have provided avatar customization features, allowing users to customize their own 2D / 3D avatars and use them as profile images. Even though these avatars are generated based on the user's facial image, they still have limitations: they differ from realistic photographs.
[0049] Meanwhile, AI-based generative models can learn from user facial images to create natural, realistic images of the user, rather than virtual characters like avatars. However, some results produced using generative models are of very low quality and sometimes resemble completely different individuals.
[0050] In this embodiment, we propose a technique to improve the limitations and quality of the generation model to generate high-quality images that are more realistic and natural.
[0051] The computer device (200) according to the present embodiment can provide an image generation service to a client through a dedicated application installed on the client or access to a web / mobile site related to the computer device (200). The computer device (200) may be configured with a computer-implemented image generation system. For example, the image generation system may be implemented in the form of an independently operating program, or may be configured in the form of an in-app of a specific application (e.g., a messenger, etc.) so as to be operable on the specific application.
[0052] The processor (220) of the computer device (200) may be implemented as a component for performing the following image generation method. Depending on the embodiment, the components of the processor (220) may be selectively included or excluded from the processor (220). Furthermore, depending on the embodiment, the components of the processor (220) may be separated or combined to express the functions of the processor (220).
[0053] These processors (220) and components of the processor (220) can control the computer device (200) to perform the steps included in the image generation method below. For example, the processor (220) and components of the processor (220) can be implemented to execute instructions according to the code of the operating system included in the memory (210) and the code of at least one program.
[0054] Here, the components of the processor (220) may be representations of different functions performed by the processor (220) according to instructions provided by the program code stored in the computer device (200).
[0055] The processor (220) can read necessary commands from the memory (210) loaded with commands related to the control of the computer device (200). In this case, the read commands may include commands for controlling the processor (220) to execute steps to be described later.
[0056] The steps included in the image generation method described below may be performed in a different order than the order shown, and some of the steps may be omitted or additional processes may be included.
[0057] The steps included in the image generation method may be performed on the server (150), and in some embodiments, at least some of the steps may also be performed on the client.
[0058] FIG. 3 is a flowchart illustrating an example of a method that a computer device according to an embodiment of the present invention can perform.
[0059] Referring to FIG. 3, in step S310, the processor (220) may receive multiple images given by the user as input images, and at this time, may remove minor images that are not helpful for learning from among the input images. A user requesting image generation may upload multiple portrait photos as a correct data set for learning the generation model. For example, the processor (220) may compare all portrait photos uploaded by the user and select minor images that are different from the common characteristics of the majority of photos, such as a portrait that is different from the person in the majority of photos, a photo whose face is covered by a mask or sunglasses, a photo with heavier makeup than the majority of photos, or a photo with lighter makeup than the majority of photos, as noisy images and remove them.
[0060] In step (S320), the processor (220) can remove minor images from the input images and use the remaining images as learning target images to generate augmented images through data augmentation for the learning target images. For example, the processor (220) can transform the learning target image into 3D using a 3D image reconstruction model such as NeRF (non-extensible reduced firmware), and then generate images of the 3D image taken from various angles as the augmented images.
[0061] In step (S330), the processor (220) can learn a generation model using the learning target image refined in step (S310) and the augmented image generated in step (S320), and can create a new image of the same person as the learning target image using the generation model learned with the learning target image and the augmented image.
[0062] In step (S340), the processor (220) may filter the new images generated in step (S330) based on the similarity with the learning target image. The processor (220) may compare the learning target image with the new images generated through the generation model, and output new images whose similarity criteria pass a threshold, i.e., new images whose similarity with the learning target image is above a certain level, as response images to the image generation request. Among the new images generated through the generation model, images whose similarity with the learning target image is below the threshold may be discarded as invalid images.
[0063] FIG. 4 is a diagram illustrating an example of a learning image refinement and augmentation process in one embodiment of the present invention.
[0064] Referring to FIG. 4, the processor (220) can detect a face region from each input image received from a user requesting image generation (face detection) (S41). At this time, the processor (220) can crop the detected face region from the input image and utilize it.
[0065] The processor (220) can obtain a face feature for each input image by extracting a feature vector as an embedding from a face region detected in each input image (S42).
[0066] The processor (220) can calculate a feature centroid by averaging facial features for each input image (S43). When N facial images are given as input images, a 512-dimensional facial feature vector can be generated for each facial image through facial feature extraction. In other words, N 512-dimensional facial feature vectors can be generated for N facial images. For example, the processor (220) can obtain a feature centroid by calculating an average or weighted average in the vector space of the N facial feature vectors.
[0067] The processor (220) can select and remove minor images from the input images based on the feature center point (S44). For example, the processor (220) can determine the similarity between the facial feature vector and the feature center point vector for each input image using cosine similarity. The processor (220) can determine an image among the input images in which the cosine similarity between the facial feature vector and the feature center point is below a threshold as a minor image.
[0068] The processor (220) can remove minor images from the input images and use the remaining images as learning target images to generate augmented images through data augmentation for the learning target images (S45). The processor (220) can transform the learning target images into 3D and use images captured from various angles as augmented images for the learning target images.
[0069] The processor (220) can learn a generative model (S46) using the refined learning target image in step (S44) and the augmented image generated in step (S45). The generative model learning process will be described again below.
[0070] According to embodiments of the present invention, the quality of model learning can be improved by using an augmented image through data augmentation as a learning dataset for a generation model together with a learning target image from which minor images have been removed.
[0071] FIG. 5 is a diagram illustrating an example of a similarity-based image generation process according to one embodiment of the present invention.
[0072] Referring to FIG. 5, the processor (220) can recalculate the feature center point (hereinafter referred to as the "reference feature center point") that serves as a reference in the reference process by using a refined learning target image rather than the entire input image (S51). In other words, the processor (220) can recalculate the feature center point by removing minor images from the input image and averaging the facial features for each learning target image based on the remaining images, i.e., the learning target image.
[0073] The processor (220) generates a new image of the same person as the learning target image using a generative model trained on the learning target image and the augmented image. At this time, a facial area can be detected from the new image (S52). The processor (220) can crop out and utilize the facial area detected from the new image.
[0074] The processor (220) can obtain a face feature vector in a new image by extracting a feature vector as an embedding from a face region cut out from a new image (S53).
[0075] The processor (220) can calculate the similarity between the reference feature center point recalculated using the learning target image and the facial feature vector of the new image (S54). For example, the processor (220) can calculate the similarity between the facial feature vector of the new image and the reference feature center point vector using the cosine distance between the vectors (S54).
[0076] The processor (220) can determine whether the cosine similarity between the face feature vector of the new image and the reference feature center point vector is greater than a threshold value (S55).
[0077] If the cosine similarity between the facial feature vector of the new image and the reference feature center point vector is greater than or equal to a threshold, the processor (220) may provide the new image as a generated image of the same person as the learning target image (S56). In other words, the processor (220) may apply a similarity criterion to the generation model to provide a new image whose cosine similarity with the reference feature center point is greater than or equal to a threshold as an output image for the input image.
[0078] If the cosine similarity between the facial feature vector of the new image and the reference feature center point vector is below a threshold, the processor (220) may invalidate the generated result for the learning target image, i.e., the new image, and proceed with the generation model retraining process (S57). The generation model retraining process will be described in detail below.
[0079] According to embodiments of the present invention, a high-quality image of the same person can be provided by combining a similarity criterion with a generation model to provide a person image that guarantees a similarity level above a certain level.
[0080] FIG. 6 is a diagram illustrating an example of a generative model learning process in one embodiment of the present invention.
[0081] Referring to FIG. 6, the processor (220) can generate a prompt for an image generation model (S61). For example, the processor (220) can create a prompt by reflecting user preferences. During the user's image generation request process, the processor (220) can receive information preferred by the user as image generation conditions along with an input image containing the target person. For example, the image generation conditions may include eye color, skin color, facial expression, hair color, hair style, image background, clothing, etc., and user preference information can be input for these conditions. The processor (220) can generate an input prompt for the generation model by combining a default prompt and a prompt reflecting user preferences. The default prompt may utilize publicly available images such as masterpiece, best quality, Nikon raw photo, 8k, realistic, photo realistic, and ultra-detailed photo.
[0082] The processor (220) can generate model parameters for a generative model such as a Stable Diffusion Model (SDM) (S62). For example, the processor (220) can generate model parameters by selecting one of presets prepared in advance for model learning. The model parameters may include a base model, a learning rate, a network dimension (Network_dim), a network attention (Network_alpha), etc. An example of a preset for a model parameter is illustrated in FIG. 7. At least one base model can be applied for a generative model, and a model parameter preset (700) can be configured by setting at least one of the learning rate, the network dimension, and the network attention to a different value for each base model.
[0083] The processor (220) trains a generation model using a learning target image refined through minor image removal and an augmented image generated through data augmentation. At this time, the generation model can be trained using the model parameters generated in step (S62) using the prompt generated in step (S61) as model input (S63).
[0084] FIG. 8 is a diagram illustrating an example of a learning parameter recommendation process in one embodiment of the present invention.
[0085] The processor (220) changes the model parameters and performs a generation model retraining process when the cosine similarity between the face feature vector of the new image and the reference feature center point vector is less than a threshold for the new image generated through the generation model.
[0086] Referring to FIG. 8, the processor (220) may accumulate the generated image filtering results for each model parameter preset while retraining the generation model with the model parameters of a preset randomly selected from among the pre-configured model parameter presets for an initial period of time (N months). In other words, the processor (220) may accumulate statistical information for each model parameter preset for a certain period of time into a DB (70) by storing the model parameter preset information used for learning the current generation model when the cosine similarity between the facial feature vector of the new image generated through the generation model through the retraining process and the reference feature center point vector is greater than a threshold value (S81). The processor (220) may accumulate statistical data according to the image filtering results for the pre-configured model parameter presets, and may give a higher priority to a preset for which a generated image has passed the similarity criterion several times in the past.
[0087] If the cosine similarity between the facial feature vector of the new image generated through the generation model and the reference feature center point vector is below a threshold after an initial predetermined period of time has elapsed, the processor (220) may recommend a model parameter preset based on a priority criterion using the preset statistics DB (70) for retraining the generation model (S82). The processor (220) may recommend a model parameter preset in a way that a preset with a high similarity criterion passing rate is weighted and selected preferentially through the preset statistics DB (70) accumulated over a predetermined period of time.
[0088] The processor (220) can retrain the generated model using the model parameters recommended in step (S82) using the prompt generated in step (S61) as model input (S83).
[0089] The processor (220) can generate a new image of the same person as the learning target image using the learning target image and the augmented image through the retrained generation model after retraining the generation model, and at this time, the similarity criterion filtering process described above (processes (S51 to S57) described through FIG. 5) can be repeatedly performed on the new image.
[0090] Accordingly, the processor (220) can repeatedly perform the model learning process until an image passing the similarity criterion is generated, and in particular, by accumulating past experience data, can recommend model parameters of a preset with a high probability of passing the similarity criterion during the model relearning process.
[0091] According to embodiments of the present invention, computational loss due to retraining can be minimized by recommending a preset that derives an optimal generation result based on past experience for the learning parameters of the generation model.
[0092] Figure 9 illustrates an example of the use of a generated image in one embodiment of the present invention.
[0093] The processor (220) can provide a new image of the same person as the input image through a generative model in which a similarity criterion is combined with the input image, and the new image can be used for various purposes on the client.
[0094] For example, referring to FIG. 9, the client can receive a new image (901) of the same person as the input image through a generation model combined with a similarity criterion, and apply the new image (901) as a messenger profile image (910). For example, the new image (901) can be set as the profile image (910) through manual selection by the user on the messenger profile home (900). According to an embodiment, when multiple new images (901) that pass the similarity criterion for the input image are provided, the profile image (910) can be set through a method of automatically selecting the image with the highest similarity score among the multiple new images (901).
[0095] The new image (901) can be stored locally on the client, and can be used in various fields such as messenger profile image (910), messenger chat room wallpaper, messenger sticker (emoticon), electronic device display wallpaper, image sharing, image editing, image-based documents, etc.
[0096] The devices described above may be implemented as hardware components, software components, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone; however, one of ordinary skill in the art will recognize that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing unit may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.
[0097] Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may independently or collectively command the processing device. The software and / or data may be embodied in any type of machine, component, physical device, computer storage medium, or device for interpretation by the processing device or for providing instructions or data to the processing device. The software may also be distributed over networked computer systems and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.
[0098] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. In this case, the medium may be one that continuously stores a computer-executable program or one that temporarily stores it for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, but may also be distributed over a network. Examples of the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and those configured to store program commands, including ROM, RAM, and flash memory. In addition, examples of other media may include recording media or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc.
[0099] Although the embodiments described above have been described by way of limited examples and drawings, those skilled in the art will appreciate that various modifications and variations can be made based on the above teachings. For example, appropriate results can still be achieved even if the described techniques are performed in a different order than described, and / or components of the described systems, structures, devices, circuits, etc. are combined or combined in a different manner than described, or are replaced or substituted with other components or equivalents.
[0100] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims described below.
Claims
1. In a method for generating an image that runs on a computer device, The computer device comprises at least one processor configured to execute computer-readable instructions contained in a memory, The above image generation method is, A step of generating a new image of the same person as the input image through a generative model according to an image generation request by at least one processor; and A step of providing the new image as a response image to the image generation request when the similarity between the input image and the new image is greater than or equal to a threshold value by at least one processor. A method for generating an image including:
2. In paragraph 1, The above generating steps are: A step of selecting a noisy image from among the above input images; and A step of removing the above noise image and training the generative model using the remaining image as a learning target image. A method for generating an image including:
3. In paragraph 2, The above selection step is, A step of extracting a face feature from each of the input images; A step of calculating a feature centroid by averaging facial features for each input image; A step of determining the noise image by comparing the facial features of each of the input images with the feature center point. A method for generating an image including:
4. In paragraph 2, The above learning steps are: A step of generating an augmented image through data augmentation using the above learning target image; and A step of learning the generation model using the above learning target image and the above augmented image. A method for generating an image including:
5. In paragraph 1, The steps provided above are: A step of extracting a facial feature from each of the input images; A step of calculating a feature center point by averaging facial features for each input image; A step of extracting facial features from the new image; and A step of comparing the facial features of the new image with the feature center point to determine the similarity. A method for generating an image including:
6. In paragraph 2, The steps provided above are: A step of extracting facial features from each of the above learning target images; A step of calculating a feature center point by averaging facial features for each learning target image; A step of extracting facial features from the new image; and A step of comparing the facial features of the new image with the feature center point to determine the similarity. A method for generating an image including:
7. In paragraph 1, The above generating steps are: A step of generating a prompt that reflects user preferences for image generation conditions; A step of selecting one preset from among multiple model parameter presets; and A step of learning the generation model through the above prompt and the model parameters of the selected preset. A method for generating an image including:
8. In paragraph 7, The above image generation method is, A step of retraining the generated model by changing the model parameter preset when the similarity is less than the threshold by at least one processor. A method for generating an image that further includes .
9. In paragraph 8, The above relearning steps are: A step of recommending one preset among the plurality of model parameter presets based on statistical information for each model parameter preset according to the similarity result for the new image. A method for generating an image including:
10. In paragraph 8, The above relearning steps are: A step of accumulating statistical information for each model parameter preset by storing model parameter preset information used in learning the generation model that generated new images above the threshold value for a certain period of time; and A step of recommending a model parameter preset having a high probability of generating a new image greater than the threshold using statistical information for each model parameter preset when the similarity is less than the threshold. A method for generating an image including:
11. A computer program stored in a non-transitory computer-readable recording medium for executing the image generation method of any one of claims 1 to 10 on the computer device.
12. In computer devices, At least one processor configured to execute computer-readable instructions contained in memory Including, At least one processor, A process of generating a new image of the same person as the input image through a generation model according to an image generation request; and A process of providing the new image as a response image to the image generation request when the similarity between the input image and the new image is greater than a threshold value. A computer device that processes .
13. In paragraph 12, At least one processor, Select a noisy image from the above input images, Training the generative model by removing the above noise image and using the remaining image as the learning target image. A computer device characterized by:
14. In paragraph 13, At least one processor, For each of the above input images, facial features are extracted from the input image, Calculate the feature center point by averaging the facial features for each input image, Determining the noise image by comparing the facial features of each of the input images with the feature center point. A computer device characterized by:
15. In paragraph 13, At least one processor, Create an augmented image by augmenting data using the above learning target image, Learning the generative model using the above learning target image and the above augmented image. A computer device characterized by:
16. In paragraph 12, At least one processor, For each of the above input images, facial features are extracted from the input image, Calculate the feature center point by averaging the facial features for each input image, Extract facial features from the new image above, Comparing the facial features of the new image with the feature center point to determine the similarity. A computer device characterized by:
17. In paragraph 13, At least one processor, For each of the above learning target images, facial features are extracted from the above learning target images, The feature center point is calculated by averaging the facial features for each image of the above learning target, Extract facial features from the new image above, Comparing the facial features of the new image with the feature center point to determine the similarity. A computer device characterized by:
18. In paragraph 12, At least one processor, Create prompts that reflect user preferences for image generation conditions, Select one of the multiple model parameter presets, Training the generative model through the above prompt and model parameters of the above selected preset. A computer device characterized by:
19. In paragraph 18, At least one processor, If the similarity is less than the threshold, retraining the generated model by changing the model parameter preset. A computer device characterized by:
20. In paragraph 19, At least one processor, By storing model parameter preset information used in learning the above-mentioned generation model that generated new images above the threshold for a certain period of time, statistical information is accumulated for each model parameter preset, If the similarity is less than the threshold, recommending a model parameter preset having a high probability of generating a new image greater than the threshold using statistical information for each model parameter preset. A computer device characterized by:
Citation Information
Patent Citations
Program, information processing device, information processing method and learning model generation method
JP2021177266A
Learning apparatus and method for creating image and apparatus and method for image creation
KR102287407B1
A method for creating a face image
KR102334666B1
A learning model recommendation device based on similarity and a cloud integrated operating system comprising the same
KR102531299B1
