Content generation method and system

Abstract

An information processing system is disclosed. The information processing system comprises a memory, and one processor connected to the memory and configured to execute at least one computer-readable program included in the memory. The at least one program may include instructions for: receiving an input content generation condition associated with a specific user; inputting the input content generation condition into a conditional content generation model to generate a first image associated with the input content generation condition; evaluating the generated first image on the basis of content generation condition data associated with other users; and outputting the generated first image on the basis of an evaluation result, wherein the input content generation condition associated with the specific user includes at least one of a positive condition and a negative condition, and the generated first image is generated to reflect the positive condition while avoiding the negative condition.

Description

Content creation methods and systems

[0001] The present disclosure relates to a method and system for generating content, and specifically, to a method for generating content using a conditional content generation model and an information processing system for the same.

[0002] Image content generation technology has primarily relied on manual editing tools or automated generation tools based on predefined templates. While this approach can guarantee a certain level of quality, it entails drawbacks such as limitations in user customization and inefficiencies in mass production. In particular, with the rapid advancement of artificial intelligence (AI) and machine learning (ML) technologies, there is a growing need for image generation technology that dynamically adjusts to user requirements.

[0003] Recently, the emergence of technologies utilizing deep learning has made it possible to generate and transform high-quality images. Such technologies increase the level of automation in image generation and offer the potential to overcome the limitations of existing technologies. However, there are still limitations in generating user-customized content.

[0004] The information described above disclosed in the background technology of this invention is intended only to enhance understanding of the background of the present invention and may therefore include information that does not constitute prior art.

[0005] The present disclosure provides a method and system for generating content customized for the user according to content generation conditions desired by the user in order to solve the above-mentioned problems.

[0006] However, the technical problems that the present invention aims to solve are not limited to those described above, and other unmentioned problems can be clearly understood by those skilled in the art from the description of the invention below.

[0007] The present disclosure may be implemented in various ways, including a method, an apparatus (system), or a computer program stored on a readable storage medium.

[0008] According to one embodiment of the present disclosure, a content generation method performed by at least one processor comprises: receiving an input content generation condition associated with a specific user; inputting the input content generation condition into a conditional content generation model to generate a first image associated with the input content generation condition; evaluating the first image generated based on content generation condition data associated with other users; and outputting the first image generated based on the evaluation result. The input content generation condition associated with a specific user may include at least one of a positive condition or a negative condition.

[0009] According to one embodiment of the present disclosure, the generated first image may be generated to reflect target conditions while avoiding exclusion conditions.

[0010] According to one embodiment of the present disclosure, the generated first image may include at least one of a generated image in which a specific person is wearing a specific product, a generated image in which a specific person is wearing a product of a specific style, a generated image in which a specific person is wearing a product of a specific brand, a generated image including a specific background, a generated image in which a specific person is not wearing a product of a specific brand, a generated image not including a specific background, a generated image not including a specific person who is in a competitive relationship with a specific user, or a generated image not including a work.

[0011] According to one embodiment of the present disclosure, a conditional content generation model may include an ID encoder that generates an identification (ID) vector by encoding content generation conditions.

[0012] According to one embodiment of the present disclosure, a content generation method may further include the steps of: obtaining a first ID encoder that is pre-trained to generate an ID vector by encoding a learning target condition; obtaining a second ID encoder that generates an ID vector by encoding a learning target condition and a plurality of learning exclusion conditions; fine-tuning the second ID encoder such that the similarity between the learning target condition and the plurality of learning exclusion conditions is low through an operation between the generated ID vector of the first ID encoder and the generated ID vector of the second ID encoder; and determining the fine-tuned second ID encoder as an ID encoder.

[0013] According to one embodiment of the present disclosure, a conditional content generation model includes a neural network model, and a content generation method may further include the steps of: generating a second image based on a learning target condition using a neural network model including a determined ID encoder; generating an ID vector for the second image using an ID encoder; generating an ID vector for a learning target condition using an ID encoder; generating ID vectors for a plurality of learning exclusion conditions using an ID encoder; and updating the neural network model such that the similarity between the ID vector for the second image and the ID vector for the learning target condition increases, while the similarity between the ID vector for the second image and the ID vectors for the plurality of learning exclusion conditions decreases.

[0014] According to one embodiment of the present disclosure, content creation condition data associated with other users includes legacy target condition ID vectors and legacy exclusion condition ID vectors encoded using an ID encoder, and the step of evaluating a generated first image may include the step of generating an ID vector for the first image using an ID encoder, and the step of calculating a similarity score between the ID vector for the first image, the legacy target condition ID vectors, and the legacy exclusion condition ID vectors using a legacy validator.

[0015] According to one embodiment of the present disclosure, the step of outputting a first image may include the step of outputting the first image if the ID vector for the first image has a similarity score of less than or equal to a predetermined threshold with all legacy ID vectors.

[0016] According to one embodiment of the present disclosure, the step of outputting a first image may include outputting an error message or outputting an image regeneration message if the ID vector for the first image has a similarity score exceeding a predetermined threshold with at least one legacy ID vector.

[0017] According to one embodiment of the present disclosure, an information processing system comprises a memory and a processor connected to the memory and configured to execute at least one computer-readable program included in the memory, wherein the at least one program includes instructions for receiving an input content generation condition associated with a specific user, inputting the input content generation condition into a conditional content generation model to generate a first image associated with the input content generation condition, evaluating the generated first image based on content generation condition data associated with other users, and outputting the generated first image based on the evaluation result, wherein the input content generation condition associated with a specific user may include at least one of a positive condition or a negative condition.

[0018] According to some embodiments of the present disclosure, content is generated in a user-customized manner according to content generation conditions desired by the user, thereby providing a user-specific specialized service.

[0019] According to some embodiments of the present disclosure, the creation of content similar to another person's work can be avoided by excluding the creation of content that the user does not want, and the issue of unfair competition caused by imitation of competitor content can be reduced.

[0020] However, the effects obtainable through the present invention are not limited to those described above, and other unmentioned technical effects will be clearly understood by those skilled in the art from the description of the invention below.

[0021] The following drawings attached to this specification illustrate preferred embodiments of the present invention and serve to further enhance understanding of the technical concept of the present invention together with the detailed description of the invention provided below; therefore, the present invention should not be interpreted as being limited only to the matters described in such drawings.

[0022] FIG. 1 is a schematic diagram illustrating a content creation system according to one embodiment of the present disclosure.

[0023] FIG. 2 is a schematic diagram showing a configuration in which an information processing system is connected to communicate with a plurality of user terminals to provide a logistics transportation service according to one embodiment of the present disclosure.

[0024] FIG. 3 is a block diagram showing the internal configuration of a user terminal and an information processing system according to one embodiment of the present disclosure.

[0025] FIG. 4 is a diagram illustrating a conditional content generation model according to one embodiment of the present disclosure.

[0026] FIG. 5 is a drawing for illustrating an example of a method for fine-tuning an ID encoder according to one embodiment of the present disclosure.

[0027] FIG. 6 is a diagram illustrating the process of a model updater according to one embodiment of the present disclosure.

[0028] FIG. 7 illustrates an example of a method for evaluating a generated image according to one embodiment of the present disclosure.

[0029] FIG. 8 is a flowchart illustrating a method for creating content according to one embodiment of the present disclosure.

[0030] FIG. 9 is a flowchart illustrating a method for determining an ID encoder according to one embodiment of the present disclosure.

[0031] FIG. 10 illustrates a process of learning a conditional content generation model according to one embodiment of the present disclosure.

[0032] FIG. 11 is a flowchart illustrating a method for evaluating an image generated through a conditional content generation model based on a legacy score according to one embodiment of the present disclosure.

[0033] Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, wherein similar reference numerals indicate similar elements, but are not limited thereto.

[0034] Hereinafter, specific details for implementing the present disclosure will be described in detail with reference to the attached drawings. However, in the following description, specific descriptions regarding widely known functions or configurations will be omitted if there is a risk that the gist of the present disclosure may be unnecessarily obscured.

[0035] In the attached drawings, identical or corresponding components are assigned the same reference numerals. Additionally, in the description of the following embodiments, the description of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such component is not included in any embodiment.

[0036] The advantages and features of the disclosed embodiments and the methods for achieving them will become clear by referring to the embodiments described below in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below but may be implemented in various different forms, and the embodiments provided are merely to make the present disclosure complete and to fully inform those skilled in the art of the scope of the invention.

[0037] The terms used in this specification will be briefly explained, and the disclosed embodiments will be described in detail. The terms used in this specification have been selected to be as generally used as possible, taking into account their functions in this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this disclosure should be defined not merely by their names, but based on their meanings and the content throughout this disclosure.

[0038] In this specification, singular expressions include plural expressions unless the context clearly specifies them as singular. Additionally, plural expressions include singular expressions unless the context clearly specifies them as plural. Throughout the specification, when a part is described as including a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0039] Additionally, the terms 'module' or 'part' as used in the specification refer to software or hardware components, and the 'module' or 'part' performs certain roles. However, the meaning of 'module' or 'part' is not limited to software or hardware. The 'module' or 'part' may be configured to reside in an addressable storage medium or configured to run on one or more processors. Thus, as an example, the 'module' or 'part' may include components such as software components, object-oriented software components, class components, and task components, and at least one of processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, or variables. The components and the functions provided within the 'module' or 'part' may be combined into a smaller number of components and 'modules' or 'parts', or further separated into additional components and 'modules' or 'parts'.

[0040] According to one embodiment of the present disclosure, a ‘module’ or ‘part’ may be implemented as a processor and memory. The term ‘processor’ should be broadly interpreted to include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, etc. In some environments, the term ‘processor’ may refer to an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), etc. The term ‘processor’ may also refer to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors combined with a DSP core, or any other combination of such configurations. Additionally, the term ‘memory’ should be broadly interpreted to include any electronic component capable of storing electronic information. 'Memory' may refer to various types of processor-readable media, such as Random Access Memory (RAM), Read-Only Memory (ROM), Non-Volatile Random Access Memory (NVRAM), Programmable Read-Only Memory (PROM), Erasable-Programmable Read-Only Memory (EPROM), Electrically Erasable PROM (EEPROM), Flash Memory, Magnetic or Optical Data Storage Devices, and Registers. If a processor can read information from memory and write information to memory, the memory is said to be in an electronic communication state with the processor. Memory integrated into the processor is in an electronic communication state with the processor.

[0041] In the present disclosure, the 'system' may include at least one of a server device and a cloud device, but is not limited thereto. For example, the system may be composed of one or more server devices. As another example, the system may be composed of one or more cloud devices. As yet another example, the system may be configured and operated with both a server device and a cloud device.

[0042] In the present disclosure, 'display' may refer to any display device associated with a computing device, for example, any display device capable of displaying any information / data controlled by or provided by the computing device.

[0043] In the present disclosure, 'each of a plurality of A' or 'each of a plurality of A' may refer to each of all components included in a plurality of A, or each of some components included in a plurality of A.

[0044] In the present disclosure, the term "artificial neural network model" may refer to a model comprising one or more artificial neural networks configured to infer an answer to a given input, including an input layer, a plurality of hidden layers, and an output layer. Here, each layer may include a plurality of nodes.

[0045] FIG. 1 is a schematic diagram for explaining a content creation system (100) according to one embodiment of the present disclosure.

[0046] Referring to FIG. 1, the content creation system (100) can receive an input content creation condition (10) and output an image (130) or output an error message (140).

[0047] The input content generation condition (10) may include an input content generation condition associated with a specific user. The input content generation condition associated with a specific user may be a condition associated with at least one of a positive condition or a negative condition desired by the specific user.

[0048] A content generation system (100) can generate an image (110) according to input content generation conditions (10). Here, the generated image may be an image that reflects the target conditions while avoiding exclusion conditions. Here, the target conditions may be positive conditions related to the desired conditions or desired outcomes of a specific user, and the exclusion conditions may include negative conditions for excluding elements or content that the specific user does not desire.

[0049] The generated image may include at least one of a generated image in which a specific person is wearing a specific product, a generated image in which a specific person is wearing a product of a specific style, a generated image in which a specific person is wearing a product of a specific brand, or a generated image including a specific background, in connection with a target condition, but the present disclosure is not limited thereto.

[0050] Additionally, the generated image may include at least one of the following in connection with exclusion conditions: a generated image in which a specific person does not wear a product of a specific brand; a generated image in which a specific background is not included; a generated image in which a specific person in a competitive relationship with a specific user is not included; and a generated image in which a work is not included, but the present disclosure is not limited thereto.

[0051] The content creation system (100) can evaluate the generated image (120). The content creation system (100) can evaluate the generated image based on content creation condition data associated with other users. The content creation system (100) can evaluate whether the generated image does not duplicate the content creation condition data associated with other users.

[0052] The content generation system (100) can determine whether to output the generated image based on the evaluation result. The content generation system (100) can output the generated image (130) if it evaluates the image and determines that it is suitable, and if it evaluates that it is unsuitable, it can output an error message (140) instead of outputting the generated image.

[0053] FIG. 2 is a schematic diagram showing a configuration in which an information processing system (230) is connected to communicate with a plurality of user terminals (210_1, 210_2, 210_3) to generate content according to one embodiment of the present disclosure.

[0054] Referring to FIG. 2, the information processing system (230) may include system(s) capable of generating content (e.g., images, videos, etc.) based on input content generation conditions and providing the generated content. In one embodiment, the information processing system (230) may include one or more server devices and / or databases capable of storing, providing, and executing computer-executable programs (e.g., downloadable applications) and data related to content generation and provision, or one or more distributed computing devices and / or distributed databases based on cloud computing services. For example, the information processing system (230) may include separate systems (e.g., servers) for content generation and provision services.

[0055] Content creation and provision, etc. provided by the information processing system (230) can be provided to the user through applications installed on each of the multiple user terminals (210_1, 210_2, 210_3).

[0056] Multiple user terminals (210_1, 210_2, 210_3) can communicate with an information processing system (230) through a network (220). The network (220) can be configured to enable communication between the multiple user terminals (210_1, 210_2, 210_3) and the information processing system (230). Depending on the installation environment, the network (220) may be configured as a wired network such as Ethernet, Power Line Communication, telephone line communication devices and RS-serial communication, a mobile communication network, a Wireless LAN (WLAN), Wi-Fi, Bluetooth and ZigBee, or a combination thereof. The communication method is not limited and may include not only communication methods utilizing communication networks that the network (220) may include (e.g., mobile communication network, wired internet, wireless internet, broadcasting network, satellite network, etc.) but also short-range wireless communication between user terminals (210_1, 210_2, 210_3).

[0057] In FIG. 2, a mobile phone terminal (210_1), a tablet terminal (210_2), and a PC terminal (210_3) are illustrated as examples of user terminals, but are not limited thereto, and the user terminals (210_1, 210_2, 210_3) may be any computing device capable of wired and / or wireless communication and capable of installing and running applications, etc. For example, user terminals may include smartphones, mobile phones, navigation systems, computers, laptops, digital broadcasting terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), tablet PCs, game consoles, wearable devices, IoT (Internet of Things) devices, VR (Virtual Reality) devices, AR (Augmented Reality) devices, etc. Additionally, FIG. 2 illustrates three user terminals (210_1, 210_2, 210_3) communicating with an information processing system (230) through a network (220), but is not limited thereto, and may be configured so that a different number of user terminals communicate with an information processing system (230) through a network (220).

[0058] In one embodiment, each of the user terminals (210_1, 210_2, 210_3) can receive information or data from another user terminal or transmit it to another user terminal through the network (220).

[0059] In FIG. 2, the information processing system (230) is shown as an independent device separated from the user terminals (210_1, 210_2, 210_3), but is not limited thereto, and the information processing system (230) may be implemented in an integrated manner with the user terminals (210_1, 210_2, 210_3).

[0060] FIG. 3 is a block diagram showing the internal configuration of a user terminal (210) and an information processing system (230) according to one embodiment of the present disclosure.

[0061] The user terminal (210) may refer to any computing device capable of running applications, web browsers, etc., and capable of wired / wireless communication, and may include, for example, the mobile phone terminal (210_1), tablet terminal (210_2), PC terminal (210_3) of FIG. 2. Referring to FIG. 3, the user terminal (210) may include a memory (312), a processor (314), a communication module (316), and an input / output interface (318). Similarly, the information processing system (230) may include a memory (332), a processor (334), a communication module (336), and an input / output interface (338). As illustrated in FIG. 3, the user terminal (210) and the information processing system (230) may be configured to communicate information and / or data through a network (220) using their respective communication modules (316, 336). Additionally, the input / output device (320) may be configured to input information and / or data to the user terminal (210) or output information and / or data generated from the user terminal (210) through the input / output interface (318).

[0062] The memory (312, 332) may include any non-transient computer-readable recording medium. According to one embodiment, the memory (312, 332) may include a permanent mass storage device such as ROM (read-only memory), a disk drive, a solid-state drive (SSD), or flash memory. As another example, a permanent mass storage device such as ROM, an SSD, flash memory, or a disk drive may be included in the user terminal (210) or information processing system (230) as a separate permanent storage device distinct from the memory. Additionally, an operating system and at least one program code may be stored in the memory (312, 332).

[0063] These software components may be loaded from a computer-readable recording medium separate from memory (312, 332). This separate computer-readable recording medium may include a recording medium that can be directly connected to the user terminal (210) and the information processing system (230), for example, a computer-readable recording medium such as a floppy drive, disk, tape, DVD / CD-ROM drive, or memory card. As another example, the software components may be loaded into memory (312, 332) via a communication module (316, 336) rather than a computer-readable recording medium. For example, at least one program may be loaded into memory (312, 332) based on a computer program installed by files provided through a network (220) by developers or a file distribution system that distributes installation files for the application.

[0064] The processor (314, 334) may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processor (314, 334) by memory (312, 332) or a communication module (316, 336). For example, the processor (314, 334) may be configured to execute instructions received according to program code stored in a recording device such as memory (312, 332).

[0065] The communication module (316, 336) may provide a configuration or function for the user terminal (210) and the information processing system (230) to communicate with each other via the network (220), and may provide a configuration or function for the user terminal (210) and / or the information processing system (230) to communicate with another user terminal or another system (e.g., a separate cloud system). For example, a request or data generated by the processor (314) of the user terminal (210) according to program code stored in a recording device such as memory (312) may be transmitted to the information processing system (230) via the network (220) under the control of the communication module (316). Conversely, a control signal or command provided under the control of the processor (334) of the information processing system (230) may be received by the user terminal (210) via the communication module (316) of the user terminal (210) through the communication module (336) and the network (220).

[0066] The input / output interface (318) may be a means for interfacing with an input / output device (320). As an example, the input device may include a device such as a camera including an audio sensor and / or an image sensor, a keyboard, a microphone, or a mouse, and the output device may include a device such as a display, a speaker, or a haptic feedback device. As another example, the input / output interface (318) may be a means for interfacing with a device in which the configuration or function for performing input and output is integrated into one, such as a touchscreen. For example, when the processor (314) of the user terminal (210) processes instructions of a computer program loaded in memory (312), a service screen configured using information and / or data provided by an information processing system (230) or another user terminal may be displayed on a display through the input / output interface (318). In FIG. 3, the input / output device (320) is depicted as not being included in the user terminal (210), but is not limited thereto and may be configured as a single device with the user terminal (210). Additionally, the input / output interface (338) of the information processing system (230) may be a means for interfacing with a device (not shown) for input or output that is connected to the information processing system (230) or that the information processing system (230) may include. In FIG. 3, the input / output interface (318, 338) is shown as an element configured separately from the processor (314, 334), but is not limited thereto, and the input / output interface (318, 338) may be configured to be included in the processor (314, 334).

[0067] The user terminal (210) and the information processing system (230) may include more components than those of FIG. 3. However, it is not necessary to clearly illustrate most of the prior art components. In one embodiment, the user terminal (210) may be implemented to include at least some of the input / output devices (320) described above. Additionally, the user terminal (210) may further include other components such as a transceiver, a GPS (Global Positioning System) module, a camera, various sensors, a database, etc. For example, if the user terminal (210) is a smartphone, it may include components that are generally included in a smartphone, and may be implemented to include various components such as an accelerometer, a gyroscope, a microphone module, a camera module, various physical buttons, buttons using a touch panel, input / output ports, and a vibrator for vibration.

[0068] While a program for an application including content provision and display services is running, the processor (314) can receive text, images, video, voice and / or actions, etc. that are input or selected through an input device such as a touch screen, keyboard, audio sensor and / or image sensor, camera, microphone, etc. connected to an input / output interface (318), and can store the received text, images, video, voice and / or actions, etc. in memory (312) or provide them to an information processing system (230) through a communication module (316) and a network (220).

[0069] The processor (314) of the user terminal (210) may be configured to manage, process, and / or store information and / or data received from an input / output device (320), another user terminal, an information processing system (230), and / or a plurality of external systems. The information and / or data processed by the processor (314) may be provided to the information processing system (230) through a communication module (316) and a network (220). The processor (314) of the user terminal (210) may transmit information and / or data to the input / output device (320) through an input / output interface (318) and output it. For example, the processor (314) may display the received information and / or data on the screen of the user terminal (210).

[0070] The processor (334) of the information processing system (230) may be configured to manage, process, and / or store information and / or data received from a plurality of user terminals (210) and / or a plurality of external systems. The information and / or data processed by the processor (334) may be provided to the user terminals (210) through a communication module (336) and a network (220).

[0071] FIG. 4 is a diagram illustrating a conditional content generation model (400) according to one embodiment of the present disclosure. The conditional content generation model (400) may be included in an information processing system (230), but the present disclosure is not limited thereto.

[0072] Referring to FIG. 4, the conditional content generation model (400) may include a neural network model (410) and a legacy validator (440). The neural network model (410) may include a neural network algorithm that generates content based on input content generation conditions. The neural network model (410) may include an identification encoder (420) that generates an identification vector by encoding the content generation conditions, and a model updater (430) that has a loss function (e.g., contrastive ID loss) to maintain consistency of the ID by utilizing contrast learning.

[0073] Additionally, the conditional content generation model (400) may include a legacy checker. The legacy checker (440) may include a legacy condition DB (450) that stores content generation condition data associated with various users. The legacy checker (440) can evaluate the similarity between the ID vector associated with the image generated by the neural network model (410) and the legacy ID vectors stored in the legacy condition DB (450).

[0074] The processor may evaluate the suitability of the image to be newly generated using a legacy checker (440). The processor may issue a license for the image that has obtained a suitability evaluation, but the present disclosure is not limited thereto.

[0075] In one embodiment, the processor may manage conditional content generation models by version. When the processor generates an image using a specific version, it may automatically generate and provide a report associated with a specific user's exclusion conditions for the generated image. For example, the processor may notify that a specific version of the conditional content generation model has been trained to have a similarity to a competitor's content generation condition data reduced by a predetermined amount, and may calculate and provide a score regarding the learning accuracy. Additionally, after generating an image based on input content generation conditions associated with a specific user, the processor may provide the specific user with a report containing a similarity score with images generated by a competitor. The processor may periodically search for similar images at the user's request and publish warning reports, which can be used as supporting evidence in claims.

[0076] Hereinafter, the configurations of FIG. 4 will be described in more detail with reference to FIG. 5 to 7.

[0077] FIG. 5 is a drawing for explaining an example (500) of a method for fine-tuning an ID encoder according to one embodiment of the present disclosure.

[0078] Referring to FIG. 5, the processor may obtain a first ID encoder (510) that is pre-trained to generate an ID vector (514) by encoding a learning target condition (512). Here, the learning target condition (512) is shown as singular, but may be composed of plurals depending on the implementation example.

[0079] The ID vector (514) is located in the latent space and can correspond to the target condition for learning, and can include the compressed features of the target condition for learning (512).

[0080] The processor may obtain a second ID encoder (520) that generates an ID vector (530, 531 to 53N) by encoding a learning target condition (512) and a plurality of learning exclusion conditions (521 to 52N).

[0081] The processor can fine-tune the second ID encoder (520) through operations between the ID vector (514) generated by the first ID encoder (510) and the ID vectors (530, 531 to 53N) generated by the second ID encoder (520) so that the similarity between the learning target condition (512) and the learning target condition (512) is high, and the similarity between the learning target condition (512) and the multiple learning exclusion conditions (521 to 52N) is low.

[0082] Specifically, the processor can fine-tune the second ID encoder (520) so that the similarity between the ID vector (514) generated by the first ID encoder (510) and the ID vector (530) associated with the learning target condition (512) by the second ID encoder (520) increases (e.g., so that the directions of the vectors match). For example, the processor can fine-tune the second ID encoder (520) so that the result of the inner product between the ID vector (514) generated by the first ID encoder (510) and the ID vector (530) associated with the learning target condition (512) by the second ID encoder (520) is close to 1 (540) or 1. In one example, the norm (magnitude) of the ID vector (514) generated by the first ID encoder (510) and the norm of the ID vector (530) associated with the learning target condition (512) generated by the second ID encoder (520) can be set to 1, but the present disclosure is not limited thereto.

[0083] Additionally, the processor may fine-tune the second ID encoder (520) so that the similarity between the ID vector (514) generated by the first ID encoder (510) and the ID vector (530) associated with the learning exclusion conditions (521–52N) generated by the second ID encoder (520) is reduced (e.g., the directions of the vectors are orthogonal). For example, the processor may fine-tune the second ID encoder (520) so that the inner product result between the ID vector (514) generated by the first ID encoder (510) and the ID vector (531–53N) associated with the learning exclusion conditions (521–52N) generated by the second ID encoder (520) is 0 (541–54N) or close to 0. In one example, the norm (magnitude) of the ID vector (514) generated by the first ID encoder (510) and the norm of the ID vector (531–53N) associated with the learning exclusion condition (521–52N) of the second ID encoder (520) may be set to 1, but the present disclosure is not limited thereto.

[0084] The processor may repeat the fine-tuning process of the second ID encoder (520) until it meets preset criteria, and the processor may determine / use the second ID encoder (520) that has completed the fine-tuning process as the ID encoder of the conditional content generation model.

[0085] In one embodiment, the processor may use a determined ID encoder for data augmentation to obtain data for a conditional content generation model. Additionally, the processor may organize the ID encoder prior to fine-tuning into an archive, which may be to ensure that the previous result was reasonably generated.

[0086] FIG. 6 is a drawing for explaining the process (600) of a model updater (640) according to one embodiment of the present disclosure.

[0087] Referring to FIG. 6, the first ID encoder (620) and the second ID encoder (650) may be the fine-tuned ID encoder of FIG. 5, and the first ID encoder (620) and the second ID encoder (650) may be the same encoder. However, according to the embodiment, the first ID encoder (620) and the second ID encoder (650) may be encoders having the same parameters at the beginning of the learning phase, but may be set to different parameters during the learning phase, but the present disclosure is not limited thereto. Also, FIG. 6 is illustrated as using two encoders (the first ID encoder (620) and the second ID encoder (650)), but only one encoder may be used.

[0088] The processor can generate an image (610, IA) based on a target condition for training using a neural network model including a first ID encoder (EN1) determined in FIG. 5. The neural network model is configured on a multi-modal basis and can generate an ID vector associated with the input content generation condition when at least one of text and an image associated with the input content generation condition is input. Accordingly, the neural network model can receive a text-based prompt and / or a prompt containing both an image and text, and generate an image corresponding to the prompt.

[0089] The processor can generate an ID vector (630, IDA) for an image (610) using a first ID encoder (620).

[0090] The processor can generate an ID vector for a target condition (660) for training using a second ID encoder (650). Additionally, the processor can generate ID vectors for multiple exclusion conditions (671 to 67N) for training using the second ID encoder (650).

[0091] The processor can update the neural network model through the model updater (640) so that the similarity between the ID vector (630) for the image (610) and the ID vector for the learning target condition (660) increases, and the similarity between the ID vector (630) for the image (610) and the ID vectors for the multiple learning exclusion conditions (671~67N) decreases.

[0092] Specifically, the model updater (640) may include a loss function based on contrastive ID loss. For example, the model updater (640) may calculate the Euclidean distance (d) between embeddings for input ID vector pairs (X1, X2), and then calculate the contrastive loss value (L) according to [Equation 1] below.

[0093] [Formula 1]

[0094] L = (1-y) * max (0, md) 2 + y * d 2 ,

[0095] Here, L is the contrast loss value, and y can be set to 1 if it corresponds to the target condition and 0 if it corresponds to the exclusion condition. m is a hyperparameter representing the distance (margin) from other ID vectors excluding the input ID vector pair (X1, X2).

[0096] In this way, the model updater (640) can compare target samples (positive samples) and exclusion samples (negative samples) based on contrast learning to place similar samples in close proximity in the embedding space and different samples in far proximity. Through the model updater (640), the neural network model can generate images that are more similar to the target condition than to the exclusion condition. During the training of the model updater (640), the parameters of the first ID encoder (620) and the second ID encoder (650) can be updated.

[0097] In this way, after receiving input content generation conditions associated with a specific user, the processor can generate an image associated with the input content generation conditions by inputting the input content generation conditions into the neural network model of the conditional content generation model.

[0098] FIG. 7 illustrates an example of a method for evaluating a generated image according to one embodiment of the present disclosure.

[0099] The processor can construct a legacy condition DB (720). The processor can generate legacy target condition ID vectors (716) and legacy exclusion condition ID vectors (718) for content creation condition data (712, 714) associated with various users using a first ID encoder (710). The processor can construct the legacy condition DB (720) by storing the generated legacy target condition ID vectors (716) and legacy exclusion condition ID vectors (718) in the legacy condition DB (720) through an evaluation process. Here, the legacy condition DB (720) may be managed together with the creation version of the conditional content creation model, but the present disclosure is not limited thereto.

[0100] The processor can use a neural network model to input an image (e.g., 732, 734) generated in association with input content generation conditions associated with a specific user into a second ID encoder (730) to generate an ID vector (736, 738) for the generated image (e.g., 732, 734). Although FIG. 7 is illustrated as using two ID encoders (a first ID encoder (710) and a second ID encoder (730)), a single ID encoder may be used.

[0101] The processor can evaluate the generated images (732, 734) using a legacy condition DB (720) generated based on content generation condition data (716, 718) associated with other users. The content generation condition data (716, 718) associated with other users may include legacy target condition ID vectors (716) and legacy exclusion condition ID vectors (718) encoded using a first ID encoder (710).

[0102] The processor can calculate a similarity score between the ID vectors (736, 738) for the generated images (732, 734) using the legacy checker (740), the legacy target condition ID vectors (716) and legacy exclusion condition ID vectors (718) stored in the legacy condition DB (720).

[0103] In one embodiment, the legacy checker (740) can calculate a similarity score (750) between vectors through cosine similarity. For example, the legacy checker (740) can output a similarity score close to 1 for vectors in similar directions and a similarity score close to 0 for vectors in orthogonal directions. That is, the legacy checker (740) can output a value of 1 or close to 1 for vectors in the same direction and a value of 0 or close to 0 for vectors in orthogonal directions.

[0104] FIG. 8 is a flowchart illustrating a content creation method (800) according to one embodiment of the present disclosure.

[0105] In step S810, the processor can receive conditions for generating input content associated with a specific user.

[0106] In one embodiment, the input content generation condition may include at least one of text and / or an image.

[0107] In step S820, the processor can generate a first image associated with the input content generation condition by inputting the input content generation condition into a conditional content generation model.

[0108] In step S830, the processor can evaluate the first image generated based on content creation condition data associated with other users.

[0109] In step S840, the processor can output the generated first image based on the evaluation result.

[0110] The conditions for generating input content associated with a specific user may include at least one of a positive condition or a negative condition.

[0111] Here, the generated first image can be generated to reflect the target condition while avoiding the exclusion condition.

[0112] For example, the generated first image may include at least one of a generated image in which a specific person is wearing a specific product, a generated image in which a specific person is wearing a product of a specific style, a generated image in which a specific person is wearing a product of a specific brand, a generated image including a specific background, a generated image in which a specific person is not wearing a product of a specific brand, a generated image not including a specific background, a generated image not including a specific person in a competitive relationship with a specific user, or a generated image not including a work.

[0113] A conditional content generation model may include an ID encoder that generates an identification (ID) vector by encoding content generation conditions. The content generation method may determine the ID encoder.

[0114] FIG. 9 is a flowchart illustrating a method (900) for determining an ID encoder according to one embodiment of the present disclosure.

[0115] In step S910, the processor may obtain a first ID encoder that has been pre-trained to generate an ID vector by encoding a target condition for learning.

[0116] In step S920, the processor may obtain a second ID encoder that generates an ID vector by encoding a learning target condition and a plurality of learning exclusion conditions.

[0117] In step S930, the processor can fine-tune the second ID encoder through an operation between the generated ID vector of the first ID encoder and the generated ID vector of the second ID encoder so that the similarity between the learning target condition and the plurality of learning exclusion conditions is low.

[0118] In step S940, the processor can determine the fine-tuned second ID encoder as the ID encoder.

[0119] FIG. 10 is a flowchart illustrating a process (1000) for training a conditional content generation model according to one embodiment of the present disclosure. The conditional content generation model may include a neural network model.

[0120] In step S1010, the processor can generate a second image based on a target condition for training using a neural network model including a determined ID encoder.

[0121] In step S1020, the processor can generate an ID vector for the second image using an ID encoder.

[0122] In step S1030, the processor can generate an ID vector for the target condition for training using an ID encoder.

[0123] In step S1040, the processor can generate ID vectors for multiple learning exclusion conditions using an ID encoder.

[0124] In step S1050, the processor can update the neural network model such that the similarity between the ID vector for the second image and the ID vector for the learning target condition increases, while the similarity between the ID vector for the second image and the ID vectors for a plurality of learning exclusion conditions decreases.

[0125] Here, content creation condition data associated with other users may include legacy goal condition ID vectors and legacy exclusion condition ID vectors encoded using an ID encoder.

[0126] FIG. 11 is a flowchart illustrating a method (1100) for evaluating an image generated by a conditional content generation model based on a legacy score according to one embodiment of the present disclosure.

[0127] In step S1110, when the processor evaluates the generated first image, it can generate an ID vector for the first image using an ID encoder.

[0128] In step S1120, the processor can use a legacy checker to calculate a similarity score between the ID vector for the first image, the legacy target condition ID vectors, and the legacy exclusion condition ID vectors.

[0129] In step S1130, the processor can output the first image if the ID vector for the first image has a similarity score below a predetermined threshold with all legacy ID vectors.

[0130] If the processor has a similarity score exceeding a predetermined threshold with at least one legacy ID vector for the first image, it may not output the first image and may request a follow-up action from the user based on the similarity score exceeding the threshold.

[0131] Specifically, the processor may output a message guiding the regeneration of an image. For example, the processor may output a message guiding the change of conditions for generating input content associated with a specific user, but the present disclosure is not limited thereto.

[0132] The sequence diagrams of FIGS. 8 to 11 and the description above are merely examples, and the scope of the present disclosure is not limited thereto. For example, at least one step may be added, changed, or deleted, or the order of each step may be changed.

[0133] The method described above may be provided as a computer program stored on a computer-readable recording medium for execution on a computer. The medium may continuously store a program executable by a computer, or temporarily store it for execution or download. Additionally, the medium may be various recording or storage means in the form of a single or multiple hardware components combined, and may not be limited to a medium directly connected to a computer system but may exist distributed over a network. Examples of media may include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and media configured to store program instructions, including ROM, RAM, and flash memory. Furthermore, other examples of media may include recording or storage media managed by app stores that distribute applications or sites and servers that supply or distribute various other software.

[0134] The methods, operations, or techniques of the present disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those skilled in the art will understand that the various exemplary logical blocks, modules, circuits, and algorithmic steps described in connection with the disclosure herein may be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate such interchangeability between hardware and software, various exemplary components, blocks, modules, circuits, and steps have been generally described above in terms of their functional aspects. Whether such functions are implemented in hardware or in software depends on the design requirements imposed on the specific application and the overall system. Those skilled in the art may implement the functions described in various ways for each specific application, but such implementations should not be construed as departing from the scope of the present disclosure.

[0135] In a hardware implementation, the processing units used to perform the techniques may be implemented in one or more ASICs, DSPs, GPUs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described in this disclosure, computers, or a combination thereof.

[0136] Accordingly, the various exemplary logic blocks, modules, and circuits described in connection with the present disclosure may be implemented or performed by any combination of general-purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or those designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, for example, a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors coupled with a DSP core, or any other combination of configurations.

[0137] In firmware and / or software implementations, techniques may be implemented as instructions stored on a computer-readable medium such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), magnetic or optical data storage devices, etc. The instructions may be executable by one or more processors, and the processor(s) may be enabled to perform specific aspects of the functions described in this disclosure.

[0138] When implemented in software, techniques may be stored on a computer-readable medium as one or more instructions or code, or transmitted through a computer-readable medium. Computer-readable media include both computer storage media and communication media, including any medium that facilitates the transmission of a computer program from one place to another. Storage media may be any available medium that can be accessed by a computer. As a non-limiting example, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to transfer or store desired program code in the form of instructions or data structures and can be accessed by a computer. Additionally, any connection is appropriately made to the computer-readable medium.

[0139] For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line, or wireless technologies such as infrared, radio, and microwave are included within the definition of a medium. As used herein, disk and disc include CD, laser disc, optical disc, DVD (digital versatile disc), floppy disk, and Blu-ray disc, wherein disks usually play data magnetically, whereas discs play data optically using a laser. The above combinations should also be included within the scope of computer-readable media.

[0140] The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other known form of storage medium. An exemplary storage medium may be connected to a processor so that the processor can read information from the storage medium or write information to the storage medium. Alternatively, the storage medium may be integrated into the processor. The processor and the storage medium may exist within an ASIC. The ASIC may exist within a user terminal. Alternatively, the processor and the storage medium may exist as separate components within the user terminal.

[0141] Although the embodiments described above have been described as utilizing aspects of the subject matter disclosed herein in one or more standalone computer systems, the present disclosure is not limited thereto and may be implemented in conjunction with any computing environment, such as a network or a distributed computing environment. Furthermore, aspects of the subject matter in the present disclosure may be implemented in a plurality of processing chips or devices, and storage may be similarly affected across a plurality of devices. Such devices may include PCs, network servers, and portable devices.

[0142] Although the present disclosure has been described in relation to some embodiments, various modifications and changes may be made without departing from the scope of the present disclosure as understood by a person skilled in the art to which the invention of the present disclosure pertains. Furthermore, such modifications and changes should be considered to fall within the scope of the claims appended to this specification.

Claims

1. A method for generating content performed by at least one processor, A step of receiving input content generation conditions associated with a specific user; A step of generating a first image associated with the input content generation condition by inputting the above input content generation condition into a conditional content generation model; A step of evaluating the first image generated based on content creation condition data associated with other users; and Based on the above evaluation result, the step of outputting the generated first image Includes, The input content generation condition associated with the specific user mentioned above includes at least one of a positive condition or a negative condition, and A content generation method in which the first image generated above reflects the target condition, but is generated to avoid the exclusion condition.

2. In Paragraph 1, A method for generating content, wherein the first generated image comprises at least one of the following: a generated image in which a specific person is wearing a specific product; a generated image in which a specific person is wearing a product of a specific style; a generated image in which a specific person is wearing a product of a specific brand; a generated image in which a specific person is wearing a product of a specific brand; a generated image in which a specific person is not who is in a competitive relationship with the specific user is not included; or a generated image in which a work is not included.

3. In Paragraph 1, A content generation method comprising an ID encoder that generates an ID (identification) vector by encoding content generation conditions, wherein the above conditional content generation model is a content generation method.

4. In Paragraph 3, A step of obtaining a first ID encoder that is pre-trained to generate an ID vector by encoding a learning target condition; A step of obtaining a second ID encoder that generates an ID vector by encoding a learning target condition and a plurality of learning exclusion conditions; A step of fine-tuning the second ID encoder such that the similarity between the learning target condition and the plurality of learning exclusion conditions is low through an operation between the generated ID vector of the first ID encoder and the generated ID vector of the second ID encoder; and Step of determining the above-mentioned fine-tuned second ID encoder as the above-mentioned ID encoder A content creation method that further includes 5. In Paragraph 4, The above conditional content generation model includes a neural network model, and The above content creation method is, A step of generating a second image based on a target condition for training using the neural network model including the determined ID encoder; A step of generating an ID vector for the second image using the ID encoder; A step of generating an ID vector for the learning target condition using the ID encoder; A step of generating ID vectors for the plurality of learning exclusion conditions using the ID encoder; and A step of updating the neural network model such that as the similarity between the ID vector for the second image and the ID vector for the learning target condition increases, the similarity between the ID vector for the second image and the ID vectors for the plurality of learning exclusion conditions decreases. A content creation method that further includes 6. In Paragraph 5, The content creation condition data associated with the other users mentioned above includes legacy target condition ID vectors and legacy exclusion condition ID vectors encoded using the ID encoder, and The step of evaluating the first image generated above is, A step of generating an ID vector for the first image using the ID encoder; and A step of calculating a similarity score between the ID vector for the first image, the legacy target condition ID vectors, and the legacy exclusion condition ID vectors using a legacy validator. A content creation method including 7. In Paragraph 6, The step of outputting the first image above is, A step of outputting the first image if the ID vector for the first image has a similarity score below a predetermined threshold with all legacy ID vectors. A content creation method including 8. In Paragraph 7, The step of outputting the first image above is, If the ID vector for the first image has a similarity score exceeding a predetermined threshold with at least one legacy ID vector, the first image is not output, and a follow-up action is requested based on the similarity score exceeding the threshold. A content creation method including 9. A computer-readable, non-transient recording medium recording instructions for executing the method according to paragraph 1 on a computer.

10. In information processing systems, Memory; and A processor connected to the memory and configured to execute at least one computer-readable program contained in the memory. Includes, The above at least one program is, Receive input content generation conditions associated with a specific user, and Input content generation conditions are input into a conditional content generation model to generate a first image associated with the input content generation conditions, and Evaluate the first image generated above based on content creation condition data associated with other users, and Based on the above evaluation result, it includes commands for outputting the first image generated above, and An information processing system in which the input content generation condition associated with the specific user mentioned above includes at least one of a positive condition or a negative condition.

Images