Electronic device for generating personalized content and operation method thereof

Abstract

Disclosed are an electronic device for generating target content and an operation method thereof. The electronic device comprises memory for storing instructions, and at least one processor for executing the instructions, wherein the instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to: obtain keywords, related to the generation of target content, from a user; determine whether at least one learning DB related to the keywords exists; display the at least one learning DB if the at least one learning DB exists; select a target learning DB by receiving a selection command of the user for the at least one learning DB; and generate the target content including a target object generated on the basis of the target learning DB.

Description

Electronic device for generating personalized content and method of operation thereof

[0001] An electronic device for generating personalized content and a method of operating the same are disclosed.

[0002] Generative AI models are technologies that generate new content (e.g., images, text, audio, etc.) through training on large-scale data. Deep learning models such as Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs) were utilized as early technologies, while Transformer-based models have recently been gaining attention. Generative AI models are being used in various industries, including advertising, design, entertainment, and healthcare. Furthermore, by combining with ultra-high-resolution transformation and style conversion technologies, Generative AI models can provide more sophisticated results.

[0003] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art related to the present disclosure.

[0004] According to one embodiment, the electronic device may include a memory for storing instructions. The electronic device may include at least one processor for executing the instructions. When the instructions are executed individually or collectively by the at least one processor, the electronic device may acquire content related to the creation of target content. When the instructions are executed individually or collectively by the at least one processor, the electronic device may extract keywords from the content. When the instructions are executed individually or collectively by the at least one processor, the electronic device may determine whether at least one learning DB related to the keywords exists. When the instructions are executed individually or collectively by the at least one processor, the electronic device may display an object related to the at least one learning DB if the at least one learning DB exists. When the instructions are executed individually or collectively by the at least one processor, the electronic device may receive a user selection command regarding an object related to the at least one learning DB and select a target object. When the above commands are executed individually or collectively by the at least one processor, the electronic device may be able to generate the target content including a target element generated based on a target learning DB corresponding to the target object.

[0005] According to one embodiment, an electronic device may include a memory for storing instructions. The electronic device may include at least one processor for executing the instructions. When the instructions are executed individually or collectively by the at least one processor, the electronic device may select reference content that serves as the basis for generating target content. When the instructions are executed individually or collectively by the at least one processor, the electronic device may determine whether the reference content is personalized based on the metadata of the reference content. When the instructions are executed individually or collectively by the at least one processor, the electronic device may display at least one personalized element included in the reference content in the reference content if the reference content is personalized. When the instructions are executed individually or collectively by the at least one processor, the electronic device may display a target keyword that serves as the basis for generating the selected element based on the user input when receiving user input for any one of the at least one personalized element. When the above commands are executed individually or collectively by the at least one processor, the electronic device may be enabled to display an object associated with at least one learning DB corresponding to the target keyword. When the above commands are executed individually or collectively by the at least one processor, the electronic device may be enabled to receive a selection command for an object associated with the at least one learning DB and select a target object.When the above commands are executed individually or collectively by the at least one processor, the electronic device may generate the target content in which the selected element is modified based on the target learning DB corresponding to the target object.

[0006] According to one embodiment, the method of operation of an electronic device may include an operation of acquiring content related to the creation of target content. The method of operation of the electronic device may include an operation of extracting keywords from the content. The method of operation of the electronic device may include an operation of determining whether at least one learning DB related to the keywords exists. If the at least one learning DB exists, the method of operation of the electronic device may include an operation of displaying an object related to the at least one learning DB. The method of operation of the electronic device may include an operation of receiving a user's selection command for an object related to the at least one learning DB and selecting a target object. The method of operation of the electronic device may include an operation of creating the target content including a target element created based on the target learning DB corresponding to the target object.

[0007] According to one embodiment, the method of operation of an electronic device may include an operation of selecting reference content that serves as the basis for generating target content. The method of operation of the electronic device may include an operation of determining whether the reference content is personalized based on the metadata of the reference content. If the reference content is personalized, the method of operation of the electronic device may include an operation of displaying at least one personalized element included in the reference content on the reference content. When user input regarding any one of the at least one personalized element is received, the method of operation of the electronic device may include an operation of displaying a target keyword that serves as the basis for generating the selected element based on the user input. The method of operation of the electronic device may include an operation of displaying an object related to at least one learning DB corresponding to the target keyword. The method of operation of the electronic device may include an operation of receiving a selection command for the object related to the at least one learning DB and selecting a target object. The method of operation of the electronic device may include an operation of generating the target content in which the selected element is modified based on a target learning DB corresponding to the target object.

[0008] According to one embodiment, a non-transient computer-readable recording medium may store one or more computer programs. One or more computer programs may include instructions for executing an operation to obtain content related to the creation of target content from a user. One or more computer programs may include instructions for executing an operation to extract keywords from said content. One or more computer programs may include instructions for executing an operation to determine whether at least one learning DB related to said keywords exists. One or more computer programs may include instructions for executing an operation to display an object related to said at least one learning DB if said at least one learning DB exists. One or more computer programs may include instructions for receiving a user's selection command regarding an object related to said at least one learning DB and executing an operation to select a target object. One or more computer programs may include instructions for executing an operation to create said target content including a target element created based on said target learning DB corresponding to said target object.

[0009] According to one embodiment, a non-transient computer-readable recording medium may store one or more computer programs. One or more computer programs may include instructions for executing an operation to select reference content that forms the basis for generating target content. One or more computer programs may include instructions for executing an operation to determine whether the reference content is personalized based on the metadata of the reference content. If the reference content is personalized, one or more computer programs may include instructions for executing an operation to display at least one personalized element included in the reference content on the reference content. One or more computer programs may include instructions for executing an operation to display a target keyword that forms the basis for generating the selected element based on the user input when receiving user input for any one of the at least one personalized element. One or more computer programs may include instructions for executing an operation to display an object related to at least one learning DB corresponding to the target keyword. One or more computer programs may include instructions for executing an operation to select a target object upon receiving a selection command for the object related to the at least one learning DB. One or more computer programs may include instructions that execute an operation to generate the target content in which the selected element is modified, based on a target learning DB corresponding to the target object.

[0010] In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0011] FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments.

[0012] FIG. 2 is a block diagram for illustrating a generative artificial intelligence system according to one embodiment.

[0013] FIG. 3 is a flowchart for explaining the operation of an electronic device that generates target content according to one embodiment.

[0014] FIG. 4 is a diagram illustrating the operation of an electronic device that generates target content according to one embodiment.

[0015] FIG. 5 is a block diagram of an electronic device for generating target content according to one embodiment.

[0016] FIG. 6 is a diagram illustrating the creation of a learning DB according to one embodiment.

[0017] FIG. 7 is a flowchart of an electronic device for explaining the creation of target content using reference content according to one embodiment.

[0018] FIG. 8 is a diagram illustrating the creation of target content using reference content according to one embodiment.

[0019] FIG. 9 is a diagram illustrating the display of a learning DB based on classification through analysis of reference content when the reference content is not personalized according to one embodiment.

[0020] FIGS. 10 to 12 are drawings for explaining masking information according to one embodiment.

[0021] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are given the same reference numeral regardless of the drawing number, and redundant descriptions thereof will be omitted.

[0022] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or may communicate with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0023] The processor (120) can control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., a program (140)), and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store the resulting data in a non-volatile memory (134).

[0024] The processor (120) may be implemented as a circuitry (e.g., a processing circuit) such as a system on chip (SoC) or an integrated circuit (IC). The processor (120) may include one or more processors. For example, the processor (120) may include a combination of one or more processors such as a CPU, GPU, MPU, AP, and CP. Additionally, the processor (120) may include various processing circuits and / or multiple processors. For example, as used in this specification and claims, the term "processor" may include various processing circuits including at least one processor, and one or more of the at least one processor may be configured to perform the various functions described in this specification, either alone or together in a distributed manner. Where "processor," "at least one processor," or "one or more processors" are described in this specification as being configured to perform various functions, these terms may include, for example without limitation, cases where one processor performs some of the described functions and one or more other processors perform the remaining functions, as well as cases where a single processor performs all of the described functions. Additionally, at least one processor may be a combination of multiple processors that perform the various described or disclosed functions in a distributed manner, etc. At least one processor may execute program instructions to achieve or perform the various functions.

[0025] According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0026] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0027] The number of processors (120) may be one or more. For example, the processor (120) may have the structure of a multi-core processor such as a dual core, a quad core, or a hexa core.

[0028] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0029] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0030] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0031] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0032] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0033] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0034] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0035] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0036] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0037] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0038] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0039] The power management module (188) can manage the power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0040] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0041] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0042] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0043] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0044] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0045] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0046] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0047] FIG. 2 is a block diagram for illustrating a generative artificial intelligence system according to one embodiment.

[0048] A User Query / Response Interface (210) can receive user input. User input may be of a type such as natural language, image, audio, and / or video. Additionally, when user input is transmitted, context information may also be transmitted. Context information may include various side information related to the time when user input is input into the artificial intelligence system (200). For example, application information currently being used by the user or location information of the user. Additionally, user input may be of a type that is a mixture of the aforementioned natural language, image, audio, video, and / or context information. Additionally, user input may include non-natural language input, such as selecting a menu.

[0049] The user query / response interface (210) can provide the user with the output of the generative artificial intelligence system. The output may include a natural language-based response and / or specific content. The output may also include an action requested by the user.

[0050] The AI ​​framework (220) can receive user input. Based on the user input (e.g., user query), the AI ​​framework (220) can coordinate and control one or more components necessary to perform an action corresponding to the user's intent.

[0051] User input received from the user query / response interface (210) can be transmitted to a prompt design component (221). The prompt design component (221) can be used to generate a prompt suitable as input to a generative model (e.g., a large language model (LLM) and / or a large multimodal model (LMM)) based on the user input.

[0052] The prompt design component (221) may be an AI component that uses a machine learning algorithm or a neural network. The prompt design component (221) may generate improved prompts through learning over time. The prompt design component (221) may access a knowledge component (230) to generate prompts based on user input. The knowledge component (230) may contain user preference data, a prompt library, and / or prompt examples. The prompt design component (221) may provide the generated prompts to a generative model (e.g., LLM and / or LMM).

[0053] The APIs / Plugins management component (223) can communicate with an external information source based on a request for additional information when user input is sent to a generative model.

[0054] The APIs / Plugins management component (223) can establish a communication channel for communication with the outside of the artificial intelligence system (200) via the API. The APIs / Plugins management component (223) can enable access to various data sources through the communication channel. The acquired information can be used to generate a prompt by the prompt design component (221) together with user input, or can be used as input for the input of the generative model (250).

[0055] The APIs / Plugins management component (223) can request the final action via API when the final action corresponding to user input, rather than the intermediate action, needs to be performed by the application or service.

[0056] The refiner component (225) can fine-tune the output of the generative model (250). For example, the refiner component (225) can determine the relevance (e.g., score) between the output of the generative model (e.g., content) and the user input. For example, the refiner component (225) can determine whether the output contains biased information (e.g., selective information). For example, the refiner component (225) can determine whether the output contains harmful information (e.g., violent content or profanity).

[0057] The refinement component (225) can determine the degree of matching (e.g., score) between the output of the generative model (250) and the user input (e.g., intent of the user input). If the refinement component (225) determines that the output of the generative model (250) does not correspond to the user input, the refinement component (225) can modify the output to correspond to the user input.

[0058] The refinement component (225) can provide the user with a hint (e.g., a hint for generating a prompt) so that the user can obtain information corresponding to the user's intention from the generative model (250).

[0059] A generative model (250) may refer to an artificial intelligence neural network that generates new data (e.g., text, images, audio, or video) based on user input (e.g., user utterance). The generative model (250) may include an image generation model and / or a language generation model.

[0060] Image generation models may include generative adversarial networks (GANs) and / or variational autoencoders (VAEs). An example of an image generation model is a diffusion-based generative model that has the structure of a VAE and a transformer.

[0061] A language generation model (e.g., ChatGPT) may be a model trained to generate the statistically most appropriate output based on input. Language generation models may include LMMs. LMMs can identify various types of input, such as text, images, audio (e.g., speech), and / or video, and generate new data corresponding to the input.

[0062] For convenience of explanation, the following description describes a large language model (LM) and / or a large vision model (LVM) that may be used in this disclosure; however, it is obvious that the artificial intelligence neural network of this disclosure may include not only a language model but also various foundation models such as code models and image models, and / or other artificial intelligence neural network models.

[0063] Artificial intelligence models that can be used in the present disclosure may include an LLM, which is an artificial intelligence neural network-based language model that has learned a large amount of text data through prior training. An LLM may include relatively more parameters (e.g., about 10 billion or more) than existing general language models. An LLM may use a Transformer artificial intelligence neural network structure based on an attention mechanism.

[0064] According to one embodiment, the training of the LLM may include pre-training and / or fine-tuning. Pre-training may include a process of training the LLM to acquire general language knowledge using a large amount of text data. For example, pre-training may include self-supervised learning that predicts the next word using the previous sequence of words in a sequence of text. Fine-tuning may include a process of training the LLM to be suitable for a specific domain (e.g., chatbot, AI assistant, translation, summary generation, question answering) and / or task. Fine-tuning may include a process of further training the LLM (e.g., supervised learning, adaptive learning) using a dataset corresponding to the specific domain and / or task based on the pre-trained model. The LLM may perform tasks based on text input containing natural language referred to as a prompt.

[0065] According to one embodiment, fine-tuning may be omitted during the training of the LLM. To improve performance for a desired task, the user may control the prompts input to the LLM. For example, the user may control the prompts to additionally provide examples of the task and / or guidance for performing the task, such as in-context learning, zero-shot learning, and / or few-shot learning. Examples of publicly available LLMs include BERT (Bidirectional Encoder Representations from Transformer) and GPT (generative pre-trained transformer).

[0066] The term 'LLM' may refer to the language neural network model itself, but it may also refer to models of LLM-based applications (e.g., chatbots, AI assistants, translation, summary generation, text classification, sentence generation). For example, an LLM-based chatbot or LLM-based translator such as ChatGPT may also be referred to as 'LLM'.

[0067] 'LLM' may include an inference engine utilizing an LLM neural network model. For example, 'inputting an input prompt into the LLM' may mean 'inputting an input prompt into an LLM-based inference engine.' For instance, 'the output of the LLM for the input prompt' may refer to the output information of the last neural network layer of the LLM obtained when the input prompt is input into the LLM-based inference engine, and / or output information modified through additional processing.

[0068] An attention mechanism is a technique that enables an artificial intelligence model to focus on important parts within input data. The attention mechanism can be utilized for predicting output data by predicting the extent to which parts of time-series input data (e.g., time-series input data such as speech or video, or input data for specific layers of a neural network) contribute to the output of intermediate layers and / or the final output. While recurrent neural network (RNN) structures, which process each element of a sequence sequentially, may experience degraded predictive performance when there is information dependency over long time-series distances, the attention mechanism can account for information dependency over long time-series distances by controlling the degree of weighted concentration (attention level) within the context of the entire input data and / or a portion thereof. A transformer can be composed of an encoder-decoder structure. The encoder processes the input data to output compressed information (e.g., contextual representation). The decoder can process compressed information and output data in token units. Each of the encoder and decoder may include an independent attention network, and may further include a cross-attention network connecting the encoder and the decoder.

[0069] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0070] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0071] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0072] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0073] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). TM It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0074] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0075] FIG. 3 is a flowchart for explaining the operation of an electronic device that generates target content according to one embodiment.

[0076] The operations described below may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. Additionally, some operations may be omitted depending on some embodiments. Operations (310) through (360) may be performed by at least one component (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1).

[0077] According to one embodiment, instructions stored in memory (e.g., memory (130) of FIG. 1) by at least one processor may be executed individually and / or collectively, and the instructions may cause an electronic device to perform the following operations (310) to operations (360).

[0078] In operation (310), the electronic device can obtain keywords related to the creation of target content from the user.

[0079] According to one embodiment, keywords may be included in a command that requests an artificial intelligence model (e.g., the generative model (250) of FIG. 2) to generate target content. Target content may represent content personalized for a user of an electronic device. Keywords may be for elements that must be included in the target content. For example, if the command is "Smile daughter and dog," the keywords may be "daughter" and "dog" because the target content must include daughter and dog as elements.

[0080] According to one embodiment, an electronic device can acquire content related to the creation of target content. The electronic device can extract keywords from the content.

[0081] According to one embodiment, the content may include a command related to the creation of the target content as reference content or text for the creation of the target content. When the electronic device receives a command related to the creation of the target content as content, the electronic device may perform an operation (310). The case in which the content acquired by the electronic device includes a reference content for the creation of the target content is described later in FIG. 7.

[0082] In operation (320), the electronic device can determine from the user whether there is at least one training database related to the keyword.

[0083] According to one embodiment, when an electronic device receives a keyword, it can determine whether at least one learning DB related to the keyword exists. The electronic device may include a plurality of learning DBs. The electronic device may store a plurality of learning DBs that have learned the features of elements corresponding to various keywords. The plurality of learning DBs may be a weight vector DB or a low-rank adaptation DB (LoRA DB) generated using the features of objects. The elements may be extracted from images stored in the electronic device. For example, the electronic device may store a learning DB that has learned the features of an element corresponding to "dog," and the element corresponding to "dog" may be a dog raised by a user included in an image stored in the electronic device. For example, the electronic device may store a learning DB that has learned the features of an element corresponding to "daughter," and the element corresponding to "daughter" may be the user's daughter included in an image stored in the electronic device. The learning DB and the method for generating the learning DB are described later in FIG. 6.

[0084] According to one embodiment, if there is no at least one training DB having a tag matching a keyword, the electronic device can determine whether there is at least one training DB having a tag with high similarity to the keyword. Similarity may include semantic similarity or morphological similarity. For example, when the keyword is "daughter," if there is no at least one training DB having "daughter" as a tag, the electronic device can determine whether there is at least one training DB having "family" as a tag, which is semantically similar.

[0085] According to one embodiment, in operation (320), if at least one learning DB related to the keyword exists, the electronic device can perform operation (330). In operation (320), if at least one learning DB related to the keyword does not exist, the electronic device can perform operation (310) again. For example, the electronic device can receive the following keywords.

[0086] In operation (330), the electronic device can display an object associated with at least one learning DB if at least one learning DB exists.

[0087] According to one embodiment, if there is at least one learning DB related to a keyword, the electronic device may display an object related to at least one learning DB on a display module (e.g., the display module (160) of FIG. 1). The electronic device may display the object related to at least one learning DB in association with a corresponding keyword. For example, the electronic device may display a separate window containing a keyword and display an object related to at least one learning DB within the separate window.

[0088] According to one embodiment, there may be at least one learning database corresponding to a keyword. For example, if the keyword is "family," the learning database corresponding to the keyword may include a learning database for a wife, a learning database for a son, and a learning database for a daughter. When an electronic device receives "family" as a keyword, it may display objects related to the learning database for a wife, objects related to the learning database for a son, and objects related to the learning database for a daughter.

[0089] According to one embodiment, the learning DB can correspond to multiple keywords. For example, the learning DB for a son can correspond to "son," "family," and "man," etc. An object associated with the learning DB for a son may be displayed when the keyword "man" is entered as well as the keyword "family."

[0090] In operation (340), the electronic device can select a target object by receiving a user's selection command for an object associated with at least one learning DB.

[0091] According to one embodiment, an electronic device may receive a user's selection command for any one of the objects associated with at least one indicated learning database. Based on the user's selection command, the electronic device may determine the object associated with the selected learning database as a target object. A target learning database corresponding to the target object may be used to generate target content.

[0092] In operation (350), the electronic device can determine whether the input of a command related to the creation of target content has been completed.

[0093] According to one embodiment, the completion of the command input may indicate a state in which the input of a command for generating target content is completed. If the input of the command is not completed, the electronic device may perform an operation (310). For example, the electronic device may receive other keywords. If the input of the command is completed, the electronic device may perform an operation (360).

[0094] According to one embodiment, a command can be input into a generative artificial intelligence model (e.g., the generative model (250) of FIG. 2). Depending on the embodiment, the command may be referred to as a prompt.

[0095] In operation (360), the electronic device can generate target content including a target element generated based on a target training database corresponding to a target object.

[0096] According to one embodiment, target content may include target elements. Target elements may be generated based on a target learning DB corresponding to a target object. Since the target learning DB is based on objects included in the user's image stored in the electronic device, the generated content may be personalized for the user of the electronic device. For example, if the target element is a daughter, the target learning DB may be a learning DB for the daughter included in the image stored in the electronic device. The electronic device may generate a personalized image that includes the user's daughter as the target element.

[0097] According to one embodiment, an electronic device may provide a target training DB along with a completed input command to a generative artificial intelligence model. The generative artificial intelligence model may generate target content for a user based on the command and the target training DB. The generative artificial intelligence model may be included inside the electronic device or on an external server.

[0098] According to one embodiment, the target training DB can enable the generative model to reflect data included in the target training DB without retraining all parameters of the generative AI model. For example, by adding the target training DB as a LoRa DB to the generative AI model, the generative AI model can output target content that reflects the LoRa DB.

[0099] According to one embodiment, all parameters of a generative AI model can be fine-tuned based on a target training DB. The fine-tuned generative AI model can generate an output that reflects the target training DB. For example, the target training DB is the dreambooth DB, and the generative AI model fine-tuned based thereon can generate an output that reflects the target training DB.

[0100] According to one embodiment, the electronic device can display the generated target content on a display module.

[0101] Below, the method for generating target content is explained using the screen of an electronic device.

[0102]

[0103] FIG. 4 is a diagram illustrating the operation of an electronic device that generates target content according to one embodiment.

[0104] Referring to FIG. 4, screens (410) to (440) of an electronic device (e.g., the electronic device (101) of FIG. 1) in which an application providing for the creation of target content is running are shown. The electronic device described below may operate according to the flowchart shown in FIG. 3. Below, the operation and screen of the electronic device as the command "Smile daughter and dog" is entered sequentially will be described.

[0105] According to one embodiment, the screen (410) may include a first area (411), a second area (413), and a third area (415). The first area (411) may be an area where reference content or target content is displayed. The second area (413) may be an area where an entered keyword is displayed. The third area (415) may be an area where a keyboard for keyword input is displayed.

[0106] According to one embodiment, the electronic device may receive a keyword input. For example, the electronic device may receive "Smile" input. Since there is no at least one learning DB related to "Smile", the electronic device may receive the keyword input again without displaying a separate learning DB. For example, the electronic device may receive "daughter" input. The electronic device may determine whether there is at least one learning DB related to "daughter". The electronic device may display at least one learning DB (423, 425) related to "daughter" on the screen (420).

[0107] According to one embodiment, a learning DB stored in an electronic device may include tags. For example, an object (423) associated with the learning DB may include "daughter" as a tag. For example, an object (425) associated with the learning DB may include "daughter" as a tag. The object (423) associated with the learning DB and the object (425) associated with the learning DB may be objects associated with a DB that has learned features for different people, even though they include the same tags. For example, the object (423) associated with the learning DB may be a DB that has learned features for the first daughter, and the object (425) associated with the learning DB may be a DB that has learned features for the second daughter.

[0108] According to one embodiment, the electronic device can determine whether there is at least one learning DB related to the keyword by searching for the existence of at least one learning DB having a tag matching the keyword. If at least one learning DB exists, the electronic device may display a user interface (e.g., a pop-up window (421)) on the screen (410) and display objects (423, 425) related to at least one learning DB within the user interface. In FIG. 4, for convenience of explanation, the user interface is displayed separately rather than inside the screen (420). The method of displaying objects (423, 425) related to at least one learning DB is merely an example and the present disclosure is not limited thereto. For example, objects related to at least one learning DB may be briefly displayed in a location adjacent to the keyword. For example, in FIG. 4, objects related to at least one learning DB may be briefly displayed in the upper area of ​​the keyword "daughter".

[0109] According to one embodiment, the electronic device may select any one of at least one object (423, 425) associated with a learning DB. The electronic device may receive a selection command for any one of at least one object (423, 425) associated with a learning DB. Based on the selection command, the electronic device may determine the selected object associated with the learning DB as the target object. For example, it may receive a selection command for an object (423) associated with a learning DB. The electronic device may determine the object (423) associated with the learning DB as the target object.

[0110] According to one embodiment, the target object may be briefly displayed at a location adjacent to the corresponding keyword. For example, on the screen (430), the object (423) related to the training DB selected as the target object may be briefly displayed above "daughter". However, this is merely an example and the present disclosure is not limited thereto. For example, the target object may not be displayed.

[0111] According to one embodiment, an electronic device can determine whether the input of a command related to content creation has been completed. For example, the electronic device may determine that the input of the command is completed when a create button (not shown) is selected. However, this is merely an example and the present disclosure is not limited thereto. For example, the electronic device may determine that the input of the command has not been completed and may receive additional input of "and dog".

[0112] According to one embodiment, when the electronic device receives "dog" as a keyword input, it can determine whether there exists an object (433, 435) associated with at least one learning DB related to "dog". If there exists an object (433, 435) associated with at least one learning DB, the electronic device can display the object (433, 435) associated with at least one learning DB within a user interface (e.g., a pop-up window (431)). The electronic device can select a target object by receiving a user's selection command for the object (433, 435) associated with at least one learning DB. For example, the electronic device can determine the object (433) associated with the learning DB as the target object.

[0113] According to one embodiment, the electronic device can determine whether the input of a command related to content creation has been completed. If it is determined that the input of the command has been completed, the electronic device can create target content (450).

[0114] According to one embodiment, the electronic device can generate target content (450) based on a target learning DB corresponding to a command and a target object. For example, the electronic device can generate target content (450) based on a target learning DB corresponding to the command "Smile daughter and dog" and an object (423) related to the learning DB that is the target object, and an object (433) related to the learning DB that is the target object.

[0115] According to one embodiment, an electronic device can input commands and a target learning DB into a generative artificial intelligence model (e.g., the generative model (250) of FIG. 2). The generative artificial intelligence model can generate target content (450) based on the commands and the target learning DB.

[0116] According to one embodiment, the electronic device may generate target content (450) and display it on a screen (440). The target content (450) may include a target object generated based on a target learning DB. For example, the target content (450) may include a target element (e.g., daughter) generated based on a target learning DB corresponding to an object (423) related to the learning DB determined as the target object. For example, the target content (450) may include a target element (e.g., dog) generated based on a target learning DB corresponding to an object (433) related to the learning DB determined as the target object.

[0117] The "A" and "C" displayed on the target element in the target content (450) of FIG. 4 are merely exemplary markings to explain that the target object was created based on the target learning DB, and may not be displayed in the actual created target content (450).

[0118] According to one embodiment, an electronic device may receive reference content for generating target content. Upon receiving the reference content, the electronic device may determine whether the reference content is personalized based on the metadata of the reference content. The electronic device may generate target content based on whether the reference content is personalized.

[0119] According to one embodiment, if the electronic device determines that the reference content is personalized, it can modify the reference content based on the target learning DB to generate target content.

[0120] According to one embodiment, if masking information indicating the area and shape to which a target learning DB is applied in the target content is predetermined, the electronic device can generate target content including target elements in the area and shape.

[0121] The generation of target content based on reference content is further explained in FIGS. 7 and 8. Masking information is further explained in FIGS. 10 to 12.

[0122] Below, an electronic device that generates target content is described.

[0123] FIG. 5 is a block diagram of an electronic device for generating target content according to one embodiment.

[0124] Referring to FIG. 5, an electronic device (e.g., the electronic device (101) of FIG. 1) is illustrated with an input module (510) (e.g., the input module (150) of FIG. 1), an application (520), a database (530), and an artificial intelligence model (540) (e.g., the generative model (250) of FIG. 2). FIG. 5 illustrates only the components related to the embodiments thereof. Therefore, it is obvious to those skilled in the art that the electronic device may include other general-purpose components in addition to those illustrated in FIG. 5.

[0125] According to one embodiment, the input module (510) may be a module capable of receiving input of commands used to generate target content (e.g., target content (450) of FIG. 4). For example, the input module (510) may include a keyboard, a microphone, and a touch-enabled display module (e.g., display module (160) of FIG. 1). The electronic device may receive input of commands containing keywords through the input module (510). The electronic device may receive selection of a target object from among at least one object related to a learning DB (e.g., objects related to the learning DB (423, 425, 433, 435) of FIG. 5) through the input module (510).

[0126] According to one embodiment, the application (520) may provide for the creation of target content. The target content may be content personalized for the user of the electronic device. The application (520) may transmit commands input through the input module (510) to the artificial intelligence model (540).

[0127] According to one embodiment, the database (530) can store and manage learning DBs. The database (530) can store and manage learning DBs using tags of the learning DBs. For example, the database (530) can store and manage learning DBs by grouping them to have the same tags. For example, the database (530) can determine learning DBs hierarchically by determining a super-tag that has a semantically higher meaning and a sub-tag that has a semantically lower meaning. For example, the database (530) can determine learning DBs hierarchically by determining "family" and "friend" as sub-tags of the super-tag "person". However, this is merely an example and should not be interpreted as limiting the embodiments of the present disclosure. For example, learning DBs may not have a hierarchy.

[0128] The database (530) can provide the target training DB to the artificial intelligence model (540).

[0129] According to one embodiment, an artificial intelligence model (540) (e.g., the generative model (250) of FIG. 2) can generate target content based on commands and a target learning DB. The artificial intelligence model (540) may be a large language model (LLM) and / or a large vision model (LVM). The artificial intelligence model can generate target content and provide it to an application (520). Upon receiving the target content, the application (520) can display the target content on the screen of an electronic device (e.g., the screen (440) of FIG. 4).

[0130] The creation of the learning DB is explained below.

[0131] FIG. 6 is a diagram illustrating the creation of a learning DB according to one embodiment.

[0132] Referring to FIG. 6, a plurality of images (610) are illustrated. An electronic device (e.g., the electronic device (101) of FIG. 1) may store the plurality of images (610). The plurality of images (610) may include images captured using the camera module of the electronic device (e.g., the camera module (180) of FIG. 1). The plurality of images (610) may include images received from an external electronic device (e.g., the electronic device (102, 104) of FIG. 1). The plurality of images (610) may be part of a total of images stored in the electronic device.

[0133] According to one embodiment, a learning DB (651, 653) can be obtained by learning features of elements extracted from images stored in an electronic device. A method for generating a learning DB (651, 653) will be described below.

[0134] According to one embodiment, an electronic device can perform classification and grouping on a plurality of images (610). The electronic device can perform classification and grouping according to elements included in the images. Through classification and grouping, the electronic device can generate a plurality of groups (620, 630). For example, the electronic device can classify images containing element A among the plurality of images (610) and group the classified images to generate a group (620). For example, the electronic device can classify images containing element B among the plurality of images (610) and group the classified images to generate a group (630).

[0135] According to one embodiment, the electronic device can extract the elements that formed the basis of the grouping in each group. The electronic device can extract the elements that formed the basis of the grouping in each group by identifying elements in an image through segmentation and then performing a crop on the image to include the elements.

[0136] For example, it is assumed that the image (640) is an image included in the group (620). The electronic device can crop the element A that formed the basis of the grouping from the image (640) to extract a cropped image (641) containing element A.

[0137] According to one embodiment, the electronic device can obtain a learning DB (651, 653) that learns the features of each element based on cropped images of each element. For example, the electronic device can obtain a learning DB (651) that learns the features of element A from cropped images of element A. The electronic device can obtain a learning DB (653) that learns the features of element B from cropped images of element B.

[0138] According to one embodiment, the electronic device can obtain a learning DB (651, 653) from an artificial intelligence model (650). The electronic device can input cropped images of each element into the artificial intelligence model (650). For example, the electronic device can input cropped images of element A into the artificial intelligence model (650).

[0139] According to one embodiment, an artificial intelligence model (650) can learn the characteristics of elements included in the cropped images based on the cropped images. By learning the characteristics of the elements, the artificial intelligence model (650) can generate a learning DB (651, 653) that includes these characteristics. The learning DB may include weights obtained through learning. The learning DB (651, 653) may include weights in matrix form.

[0140] According to one embodiment, the artificial intelligence model (650) may be a learning model that is smaller in scale than a generative artificial intelligence model (e.g., the generative model (250) of FIG. 2 or the artificial intelligence model (540) of FIG. 5). For example, the artificial intelligence model (650) may be a learning model such as a LORA model (low-rank adaptation model), and the generative artificial intelligence model may be a generative model such as CHAT-GPT. The artificial intelligence model (650) may generate a training DB (651, 653) for use in the generative artificial intelligence model based on cropped images, element by element.

[0141] However, this is merely an illustrative description to aid understanding and should not be interpreted as limiting or restricting the scope of other embodiments.

[0142] According to one embodiment, the learning DB (651, 653) obtained from the artificial intelligence model (650) can be reflected as a matrix multiplication when generating the target content of the generative artificial intelligence model (e.g., the target content (450) of FIG. 4).

[0143] According to one embodiment, the creation of a training DB (651, 653) may be omitted for the creation of target content (e.g., target content (450) of FIG. 4). The electronic device may create target content using only a reference image without using a training DB (651, 653) provided with a reference image. The electronic device may input a reference image and a command into a generative artificial intelligence model. For example, a command for creating target content and a single reference image may be input into the generative artificial intelligence model, and the generative artificial intelligence model may create target content based on the single reference image.

[0144] According to one embodiment, an electronic device can store acquired learning DBs (651, 653) in a database (660) (e.g., the database (530) of FIG. 5). The electronic device can store and manage the learning DBs in the database (660) by attaching tags to them. For example, the electronic device can store the learning DB (651) in the database (660) by attaching a tag of "daughter". For example, the electronic device can store the learning DB (653) in the database (660) by attaching a tag of "dog". Tags can be modified according to a user's modification command. Learning DBs with the same tag can be searched and displayed together when a corresponding keyword is entered.

[0145] Below, the creation of target content when reference content is provided is explained.

[0146] FIG. 7 is a flowchart of an electronic device for explaining the creation of target content using reference content according to one embodiment.

[0147] The operations described below may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. Additionally, some operations may be omitted depending on some embodiments. Operations (710) through (709) may be performed by at least one component (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1).

[0148] According to one embodiment, instructions stored in memory (e.g., memory (130) of FIG. 1) by at least one processor may be executed individually and / or collectively, and the instructions may cause an electronic device to perform the following operations (710) to operations (790).

[0149] In operation (710), the electronic device can select reference content.

[0150] According to one embodiment, an electronic device can acquire content related to the creation of target content. The electronic device can extract keywords from the content.

[0151] According to one embodiment, the content may include reference content or text for creating target content, and commands related to the creation of target content. When an electronic device receives reference content for creating target content as content, the electronic device may perform operations below operation (710).

[0152] According to one embodiment, the electronic device may display a screen that provides a selection for reference content. The electronic device may receive a selection command from a user. The electronic device may perform a selection for reference content based on the selection command. The reference content may be content that serves as the basis for the creation of target content (e.g., target content (450) of FIG. 4).

[0153] In operation (720), the electronic device can determine whether the reference content is personalized.

[0154] According to one embodiment, reference content may include metadata. If the reference content is personalized according to the method described in this disclosure, the metadata may include personalized information. For example, the metadata includes prompt information, and the prompt information<lora:USER_DB_A:lbw=daughter> It may include personalized information indicating that it is personalized, such as<lora:USER_DB_A:lbw=daughter> "USER_DB_A", which is a training DB obtained using the LORA model (e.g., training DB (651, 653) of FIG. 6), may be used, and the keyword (or element) may be indicated as "daughter". Depending on the embodiment, the metadata may further include masking information. The masking information will be described later in FIG. 10 to 12. According to one embodiment, the prompt information may include a command used to generate the reference content.

[0155] According to one embodiment, an electronic device can determine whether reference content is personalized based on metadata. Based on metadata, the electronic device can identify at least one personalized element (e.g., target element) and at least one keyword that forms the basis for the creation of at least one personalized element in the reference content. Based on personalization information in the metadata, the electronic device can identify at least one keyword that forms the basis for the creation of the personalized element. Since the keyword corresponds one-to-one with the personalized element, the electronic device can identify at least one personalized area (e.g., personalized element).

[0156] According to one embodiment, the electronic device may perform an operation (780) when it is determined that the reference content is not personalized. For example, if the metadata of the reference content does not contain personalization information, the electronic device may determine that the reference content is not personalized and perform an operation (780). For example, the electronic device may generate target content using only commands without using a learning DB.

[0157] According to one embodiment, the electronic device may perform an operation (730) when it is determined that the reference content is personalized. For example, if the metadata of the reference content includes personalized information, the electronic device may determine that the reference content is personalized.

[0158] According to one embodiment, in operation (730), the electronic device may display at least one personalized element included in the reference content on the reference content. The electronic device may display at least one personalized element among the elements included in the reference content. The electronic device may apply visual effects to the at least one personalized element to distinguish it from other non-personalized elements. For example, the electronic device may display the outermost border of the at least one personalized element in bold or in a different color. For example, the electronic device may display a shape such as a circle on the at least one personalized element.

[0159] According to one embodiment, in operation (740), the electronic device may receive user input for any one of at least one personalized element. The electronic device may receive user input for any one of the personalized elements. The electronic device may select any one of the personalized elements based on the user input.

[0160] According to one embodiment, in operation (750), the electronic device may display a target keyword that forms the basis for the creation of a selected element. For example, the electronic device may display the target keyword "daughter" when an element corresponding to a daughter is selected among at least one personalized element.

[0161] According to one embodiment, the electronic device may provide a modification for a displayed target keyword. For example, if "daughter" is displayed as a target keyword according to a selection command for at least one of the personalized elements, but the user wants to create target content that includes their son, the user may modify "daughter" to "son".

[0162] In operation (760), the electronic device can determine whether there is at least one learning DB corresponding to the target keyword. The determination of whether there is at least one learning DB corresponding to the target keyword is described in detail in the detailed description of operation (320) of FIG. 3, so a detailed description is omitted.

[0163] According to one embodiment, the electronic device can perform operation (780) if there is no at least one learning DB corresponding to the target keyword. For example, the electronic device can perform operation (780) if there is no at least one learning DB tagged with a keyword identical to or adjacent to the target keyword. For example, the electronic device can generate target content using only commands without using a learning DB. The electronic device can perform operation (770) if there is at least one learning DB corresponding to the target keyword.

[0164] In operation (770), the electronic device can select a target object.

[0165] According to one embodiment, the electronic device may display objects associated with at least one learning DB corresponding to a target keyword (e.g., objects associated with the learning DB of FIG. 4 (423, 425, 433, 435)). The electronic device may select a target object from among the objects associated with at least one learning DB based on a user's selection command. As the target object is selected, a target learning DB corresponding to the target object may be selected.

[0166] According to one embodiment, when a target keyword is modified, the electronic device may display an object associated with at least one training DB corresponding to the modified keyword. The electronic device may select a target object from among the objects associated with at least one training DB corresponding to the modified keyword based on a user's selection command.

[0167] In operation (780), the electronic device can receive a command related to the creation of target content.

[0168] According to one embodiment, an electronic device may receive a command including a target keyword identified from reference content. According to one embodiment, the command may be identical to the command used to generate the reference content. However, this is merely an example and the present disclosure is not limited thereto. For example, the electronic device may provide a modification to the command used to generate the reference content, and a user may modify the command used to generate the reference content.

[0169] In operation (790), the electronic device can generate target content. The electronic device can generate target content based on commands and a target learning DB. The electronic device can generate target content by inputting commands and a target learning DB into a generative artificial intelligence model (e.g., the generative model (250) of FIG. 2 or the artificial intelligence model (540) of FIG. 5).

[0170] According to one embodiment, an electronic device can update the metadata of reference content based on a target learning DB used to generate target content. The electronic device can store the updated metadata in association with the target content. The updated metadata can be used to determine whether the target content is personalized when used as reference content.

[0171] Below, a method for generating target content based on reference content is explained using the screen of an electronic device.

[0172] FIG. 8 is a diagram illustrating the creation of target content using reference content according to one embodiment.

[0173] Referring to FIG. 8, screens (810) to (840) of an electronic device (e.g., the electronic device (101) of FIG. 1) in which an application providing for the creation of target content is running are shown. The electronic device described below may operate according to the flowchart shown in FIG. 7.

[0174] According to one embodiment, the screen (810) may display reference content (811). An electronic device (e.g., the electronic device (101) of FIG. 1) may receive the reference content (811) through the screen (810). The reference content (811) may be target content that has been generated in the electronic device (e.g., target content (450) of FIG. 4). The reference content (811) may be target content that has been generated in an external electronic device received from an external electronic device (e.g., the electronic device (102, 104) of FIG. 1). The reference content (811) may not be target content. For convenience of explanation, it is assumed below that the reference content (811) is target content.

[0175] According to one embodiment, the electronic device may display the reference content (811) on the screen (820) when the reference content (811) is selected. For example, the electronic device may display the reference content (811) in a first area of ​​the screen (820) (e.g., the first area (411) of FIG. 4).

[0176] According to one embodiment, when reference content (811) is selected, the electronic device can determine whether the reference content is personalized. The electronic device can determine whether the reference content (811) is personalized based on the metadata (813) of the reference content (811). The electronic device can determine whether the reference content (811) is personalized based on the personalization information included in the metadata (813). The determination of whether the reference content (811) is personalized is described in detail in the operation (720) of FIG. 7, so a detailed explanation is omitted.

[0177] According to one embodiment, the electronic device may display at least one personalized element included in the reference content (811) on the reference content (811). For example, the electronic device may indicate that the element included in the reference content (811) is personalized by using a shape such as a circle on the personalized element. For example, by referring to the screen (820), the electronic device may indicate that the first element and the second element are personalized by displaying a first circle (821) on the first element and a second circle (823) on the second element.

[0178] According to one embodiment, the electronic device may receive user input for any one of at least one personalized element. For example, the electronic device may receive user input (e.g., a selection command) for an element (e.g., a first element) indicated by a first circle (821).

[0179] According to one embodiment, the electronic device may receive user input regarding an area in reference content (811) where no personalized element exists. Upon receiving user input regarding an area where no personalized element exists, the electronic device may provide the user with input for keywords for that area. The electronic device may display at least one object related to a learning DB associated with the keyword obtained from the user (e.g., objects related to the learning DB of FIG. 4 (423, 425, 433, 435)). The electronic device may determine a target object by receiving a selection command for any one of the objects related to at least one learning DB associated with the keyword obtained from the user (831, 833). Through the above-described operation, the electronic device may add elements that do not exist in the reference content (811) to the target content to be created later.

[0180] According to one embodiment, the electronic device may display a target keyword that served as the basis for the creation of a selected element. The target keyword may be identified based on personalized information contained in metadata (813). For example, a first element that received user input may have been created based on a target keyword called "daughter". The electronic device may display the target keyword in a second area (e.g., the second area (413) of FIG. 4).

[0181] According to one embodiment, the electronic device can determine whether there exists at least one learning DB corresponding to a target keyword (e.g., the learning DB (651, 653) of FIG. 6). The electronic device can determine whether there exists at least one learning DB corresponding to a target keyword in a database (e.g., the database (530) of FIG. 5 or the database (660) of FIG. 6). For example, the electronic device can determine whether there exists at least one learning DB corresponding to the target keyword "daughter," and if there is at least one learning DB, the electronic device can display an object (831, 833) related to the at least one learning DB within a user interface (e.g., a pop-up window (830) (e.g., the pop-up window (421, 431) of FIG. 4). A detailed description of the display of the object (831, 833) related to the learning DB is omitted as it has been described in detail in FIG. 4.

[0182] According to one embodiment, the electronic device can select a target object from among objects (831, 833) associated with at least one learning DB. The electronic device can select a target object from among objects associated with at least one learning DB based on user input. A target learning DB corresponding to the target object can be used to generate target content. As the target object is selected, a target learning DB corresponding to the target object can be selected.

[0183] According to one embodiment, once the selection of the target learning DB is completed, the electronic device may receive a command. The electronic device may receive a command that is the same as or different from the command that formed the basis for the creation of the reference content (811). The command that formed the basis for the creation of the reference content (811) may be determined based on the personalization information of the metadata (813).

[0184] According to one embodiment, the electronic device can automatically display the command that formed the basis for the creation of the reference content (811). For example, the command "Smile daughter and dog" that formed the basis for the creation of the reference content (811) can be automatically displayed in the second area. The electronic device can provide modifications to the command. For example, the electronic device can modify "Smile daughter and dog" to "Smile daughter and cat" based on a user's modification command.

[0185] According to one embodiment, an electronic device can generate target content (841). The electronic device can generate target content (841) based on reference content (811). Target content (841) may include personalized elements (e.g., target elements) generated based on a target learning DB.

[0186] For example, assume that reference content (811) is received from a friend's electronic device. The reference content (811) may include personalized elements created based on the friend's daughter and dog. Through the above-described operation, the target content (841) created based on the reference content (811) may include personalized elements created based on the user's daughter and dog of the electronic device.

[0187] According to one embodiment, the electronic device can determine the composition of the personalized elements included in the target content (841) based on the composition of the personalized elements present in the reference content (811). For example, if a dog is placed on the left and a person on the right in the reference content (811), a dog may also be placed on the left and a person on the right in the target content (841).

[0188] According to one embodiment, the electronic device may generate an area in the target content (841) excluding an area of ​​a personalized element based on an area (e.g., background) excluding a personalized element present in the reference content (811). For example, the electronic device may generate a background in the target content (841) that is identical to the reference content (811).

[0189] According to one embodiment, the electronic device may retain only the personalized elements corresponding to the personalized elements present in the reference content (811) in the target content (841) and create the remaining areas anew. For example, only the personalized elements corresponding to the dog and person present in the reference content (811) (e.g., person and dog) may be retained in the target content (841), and the pose, composition, and background may be newly created.

[0190] However, the creation of target content (841) based on the aforementioned reference content (811) is merely an illustrative description to aid understanding and should not be interpreted as limiting or restricting the scope of other embodiments.

[0191]

[0192] FIG. 9 is a diagram illustrating the display of a learning DB based on classification through analysis of reference content when the reference content is not personalized according to one embodiment.

[0193] According to one embodiment, even if the reference content (900) (e.g., the reference content (811) of FIG. 8) is not personalized, target content based on the reference content described above in FIG. 7 and FIG. 8 (e.g., the target content (450) of FIG. 4 or the target content (841) of FIG. 8) can be generated.

[0194] According to one embodiment, when reference content (900) is selected, the electronic device can determine whether the reference content (900) is personalized. If the electronic device determines that the reference content (900) is not personalized, it can perform semantic segmentation on the reference content (900). For example, the electronic device can identify the class of elements included in the reference content (900) through semantic segmentation.

[0195] According to one embodiment, the electronic device may receive a selection command for any one of the elements included in the reference content (900). For example, if the electronic device receives a selection command for an element identified as "Person", it may indicate the class of the element as a keyword (910).

[0196] According to one embodiment, if the electronic device includes at least one learning DB related to a keyword (910) (e.g., the learning DB of FIG. 6 (651, 653)), it may display an object related to at least one learning DB (e.g., an object related to the learning DB of FIG. 4 (423, 425, 433, 435) or an object related to the learning DB of FIG. 8 (831, 833)). For example, the electronic device may display an object related to at least one learning DB related to "Person".

[0197] According to one embodiment, the electronic device may provide a modification for a keyword (910). The electronic device may modify the keyword (910) to a modified keyword (920) based on a modification command for the keyword (910). When the keyword (910) is modified, the electronic device may display an object associated with at least one learning DB associated with the modified keyword (920).

[0198] Even if the reference content (900) is not personalized, the electronic device can generate personalized content based on the learning DB through semantic division of the reference content (900) according to the method described above.

[0199] The following describes the creation of target content using masking information.

[0200] FIGS. 10 to 12 are drawings for explaining masking information according to one embodiment.

[0201] Referring to FIG. 10, an execution screen (1000) of an application that provides for the creation of target content is shown. Referring to the screen (1000), a command (1010) for creating target content (e.g., target content (450) of FIG. 4 or target content (841) of FIG. 8) is shown.

[0202] According to one embodiment, the electronic device can determine whether the command (1010) includes information related to the creation location and orientation of a personalized element (e.g., a target element). For example, the electronic device can determine whether the command (1010) includes information indicating the location of a keyword corresponding to the personalized element (e.g., left, right, middle, up and down, etc.). If such information exists, the electronic device can generate masking information (1020) corresponding to such information.

[0203] For example, referring to the command (1010) "My daughter is sitting left down corner of a picture," it commands the creation of a personalized element (e.g., daughter) in the bottom left corner of the target content.

[0204] According to one embodiment, masking information may indicate the area and shape to which the target learning DB is to be applied. The masking information may have a value of "1" for the area and shape to which the target learning DB is to be applied and a value of "0" for the area to which the target learning DB is not to be applied. For example, the masking information may be a matrix composed of "1" and "0". However, this is merely an illustrative description to aid understanding and should not be interpreted as limiting or restricting the scope of other embodiments.

[0205] According to one embodiment, masking information may be transmitted together when a command (1010) and a target learning DB are transmitted to a generative artificial intelligence model (e.g., the generative model (250) of FIG. 2 or the artificial intelligence model (540) of FIG. 5) for generating target content.

[0206] Referring to Fig. 11, the creation of a personalized element based on pre-generated masking information is illustrated.

[0207] According to one embodiment, masking information (e.g., masking information (1020) of FIG. 10) indicating the area and shape to which the target learning DB is applied in the target content (1130) (e.g., target content (450) of FIG. 4 or target content (841) of FIG. 8) may be preset. Masking information indicating the area and shape to which a personalized element is to be generated in the target content (1130) may be preset.

[0208] For example, in the case of "person", masking information (1110) may be predetermined so that it is created in the right area of ​​the target content (1130). For example, in the case of "dog", masking information (1120) may be predetermined so that it is created in the left area of ​​the target content (1130).

[0209] According to one embodiment, if the command (e.g., the command (1010) of FIG. 10) includes information related to the creation location and orientation of an element, the masking information generated based on said information can be used to create the target content (1130). If the command does not include information related to the creation location and orientation of an element, the target content (1130) can be created based on predetermined masking information (1110, 1120).

[0210] According to one embodiment, a calculated value based on the target learning DB may be reflected in "1" of the masking information (e.g., predetermined masking information (1110, 1120)). A calculated value based on the target learning DB may not be reflected in "0" of the masking information (1110, 1120). An element reflecting the target learning DB based on the masking information may be reflected in an area containing "1".

[0211] Below, the generation of masking information through the division of elements is explained.

[0212] Referring to FIG. 12, a diagram illustrating the generation of masking information through the division of elements is shown.

[0213] According to one embodiment, an electronic device can generate reference content (1210). The electronic device can generate reference content (1210) based on the flowchart of FIG. 3. Reference content (1210) may be reference content generated before personalized elements based on a target learning DB are generated. After generating reference content, the electronic device can generate target content (1240)) by replacing the reference elements included in the reference content with personalized elements (e.g., target content (450) of FIG. 4, target content (841) of FIG. 8, or target content (1130) of FIG. 11).

[0214] According to one embodiment, an electronic device can generate reference content based on a command (e.g., a command (1010) of FIG. 10). The reference content may include a reference element (1220). The electronic device can identify the reference element (1220) through semantic partitioning of the reference content. The electronic device can generate masking information (1230) indicating the location and shape where the reference element (1220) is generated. The electronic device can generate target content (1240) by generating a target element at the location and shape corresponding to the masking information (1230) (e.g., the masking information (1020) of FIG. 10 or the masking information (1110, 1120) of FIG. 11) using a target learning DB.

[0215] According to one embodiment, the reference content (1210) is reference content (e.g., reference content (811) of FIG. 8 or reference content (900) of FIG. 9) and may not be personalized. The electronic device can identify the reference element (1220) through semantic segmentation with respect to the reference content. The electronic device can generate masking information (1230) indicating the location and shape where the reference element (1220) was generated. Based on the masking information (1230), commands, and reference content, the electronic device can generate target content that includes elements generated based on the target learning DB at the location and shape corresponding to the masking information.

[0216]

[0217] According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1) may include a memory (e.g., the memory (130) of FIG. 1) that stores instructions. The electronic device may include at least one processor (e.g., the processor (120) of FIG. 1) that executes instructions. When the instructions are executed individually or collectively by at least one processor, the electronic device may be able to obtain content related to the creation of target content (e.g., the target content (450) of FIG. 4 or the target content (841) of FIG. 8). When the instructions are executed individually or collectively by at least one processor, the electronic device may be able to extract keywords from the content. When the instructions are executed individually or collectively by at least one processor, the electronic device may be able to determine whether at least one learning DB (e.g., the learning DB (651, 653) of FIG. 6) related to the keywords exists. When the instructions are executed individually or collectively by at least one processor, the electronic device may be enabled to display an object associated with at least one learning DB (e.g., an object associated with the learning DB of FIG. 4 (423, 425, 433, 435) or an object associated with the learning DB of FIG. 8 (831, 833)) if at least one learning DB exists. When the instructions are executed individually or collectively by at least one processor, the electronic device may be enabled to receive a user selection command for an object associated with at least one learning DB and select a target object. When the instructions are executed individually or collectively by at least one processor, the electronic device may be enabled to generate target content including a target element generated based on a target learning DB corresponding to the target object.

[0218] According to one embodiment, when instructions are executed individually or collectively by at least one processor, the electronic device may generate masking information representing location information (e.g., masking information (1020) of FIG. 10, masking information (1110, 1120) of FIG. 11, or masking information (1230) of FIG. 12) when the content includes instructions related to the creation of target content (e.g., instructions (1010) of FIG. 10) and the instructions include location information to be applied to the target content, and generate target content based on the masking information.

[0219] According to one embodiment, at least one learning DB is obtained by learning features of objects corresponding to keywords extracted from images stored in an electronic device (e.g., a plurality of images (610) of FIG. 6), and can be classified into tags corresponding to keywords.

[0220] According to one embodiment, when the instructions are executed individually or collectively by at least one processor, the electronic device may determine whether the reference content is personalized based on the metadata of the reference content (e.g., metadata of FIG. 8 (813)) when the content includes reference content for generating target content (e.g., reference content of FIG. 8 (811) or reference content of FIG. 9 (900)). When the instructions are executed individually or collectively by at least one processor, the electronic device may extract keywords based on whether the reference content is personalized.

[0221] According to one embodiment, when instructions are executed individually or collectively by at least one processor, the electronic device may be configured to modify the reference content based on the target learning DB to generate target content when it is determined that the reference content is personalized.

[0222] According to one embodiment, when instructions are executed individually or collectively by at least one processor, the electronic device may be enabled to generate target content including target elements in the region and shape, provided that masking information indicating the region and shape to which the target learning DB is applied in the target content is predetermined.

[0223] According to one embodiment, when the instructions are executed individually or collectively by at least one processor, the electronic device may generate content including a reference element corresponding to a target element (e.g., a reference element (1220) of FIG. 12). When the instructions are executed individually or collectively by at least one processor, the electronic device may identify a reference element in reference content through semantic partitioning of reference content. When the instructions are executed individually or collectively by at least one processor, the electronic device may generate masking information indicating the location and shape where the reference element is generated. When the instructions are executed individually or collectively by at least one processor, the electronic device may generate target content by generating a target element having a shape at the location of the reference content based on the masking information.

[0224] According to one embodiment, the electronic device may include a memory for storing instructions. The electronic device may include at least one processor for executing instructions. When the instructions are executed individually or collectively by at least one processor, the electronic device may select reference content that serves as the basis for generating target content. When the instructions are executed individually or collectively by at least one processor, the electronic device may determine whether the reference content is personalized based on the metadata of the reference content. When the instructions are executed individually or collectively by at least one processor, the electronic device may display at least one personalized element included in the reference content in the reference content if the reference content is personalized. When the instructions are executed individually or collectively by at least one processor, the electronic device may display a target keyword that serves as the basis for generating the selected element based on user input when user input regarding any one of the at least one personalized element is received. When the instructions are executed individually or collectively by at least one processor, the electronic device may be enabled to display an object associated with at least one training DB corresponding to a target keyword. When the instructions are executed individually or collectively by at least one processor, the electronic device may be enabled to receive a selection command for an object associated with at least one training DB and select a target object. When the instructions are executed individually or collectively by at least one processor, the electronic device may be enabled to generate target content in which the selected element is modified based on the target training DB corresponding to the target object.

[0225] According to one embodiment, when instructions are executed individually or collectively by at least one processor, the electronic device may identify masking information corresponding to the creation location and shape of at least one personalized element in metadata. When instructions are executed individually or collectively by at least one processor, the electronic device may generate target content based on the masking information.

[0226] According to one embodiment, when instructions are executed individually or collectively by at least one processor, the electronic device may identify at least one personalized object and at least one keyword that forms the basis for the creation of at least one personalized object in reference content based on metadata of reference content.

[0227] According to one embodiment, metadata may include prompt information used to generate reference content. The prompt information may include at least one keyword corresponding to at least one personalized element included in the reference content when the reference content is personalized.

[0228] According to one embodiment, when instructions are executed individually or collectively by at least one processor, the electronic device may be able to update the metadata of reference content based on a target learning DB and store it in association with the target content.

[0229] According to one embodiment, the method of operation of an electronic device may include an operation of acquiring content related to the creation of target content. The method of operation of the electronic device may include an operation of extracting keywords from the content. The method of operation of the electronic device may include an operation of determining whether at least one learning DB related to the keywords exists. If at least one learning DB exists, the method of operation of the electronic device may include an operation of displaying an object related to at least one learning DB. The method of operation of the electronic device may include an operation of receiving a user's selection command for an object related to at least one learning DB and selecting a target object. The method of operation of the electronic device may include an operation of creating target content including a target element created based on a target learning DB corresponding to the target object.

[0230] According to one embodiment, the operation of generating target content can generate masking information representing location information when the content includes a command related to the generation of target content and the command includes location information to which a target learning DB is applied to the target content, and generate target content based on the masking information.

[0231] According to one embodiment, at least one learning DB is obtained by learning features of objects corresponding to keywords extracted from images stored in an electronic device, and can be classified into tags corresponding to keywords.

[0232] According to one embodiment, the method of operating an electronic device may further include an operation of determining whether the reference content is personalized based on the metadata of the reference content when the content includes reference content for generating target content. The operation of extracting keywords from the content may extract keywords based on whether the reference content is personalized.

[0233] According to one embodiment, the operation of generating target content can generate target content by modifying the reference content based on the target learning DB when it is determined that the reference content is personalized.

[0234] According to one embodiment, the operation of generating target content can generate target content including target elements in the region and shape when masking information indicating the region and shape to which a target learning DB is applied to the target content is predetermined.

[0235] According to one embodiment, the operation of generating target content may further include the operation of generating reference content including a reference element corresponding to a target element. The operation of generating target content may further include the operation of identifying a reference element in the reference content through semantic partitioning of the reference content. The operation of generating target content may further include the operation of generating masking information indicating the location and shape where the reference element was generated. The operation of generating target content may further include the operation of generating target content by generating a target element having a shape at the location of the reference content based on the masking information.

[0236] According to one embodiment, a non-transient computer-readable recording medium may store one or more computer programs. One or more computer programs may include instructions for executing an operation to obtain content related to the creation of target content from a user. One or more computer programs may include instructions for executing an operation to extract keywords from content. One or more computer programs may include instructions for executing an operation to determine whether at least one learning DB related to the keywords exists. One or more computer programs may include instructions for executing an operation to display an object related to at least one learning DB if at least one learning DB exists. One or more computer programs may include instructions for receiving a user's selection command regarding an object related to at least one learning DB and executing an operation to select a target object. One or more computer programs may include instructions for executing an operation to create target content including a target element created based on a target learning DB corresponding to the target object.

[0237] Furthermore, the embodiments of the present invention disclosed in this specification and drawings are merely specific examples provided to facilitate the explanation of the technical content according to the embodiments of the present invention and to aid in understanding the embodiments of the present invention, and are not intended to limit the scope of the embodiments of the present invention. Accordingly, the scope of the various embodiments of the present invention should be interpreted to include all modifications or variations derived based on the technical concept of the various embodiments of the present invention, in addition to the embodiments disclosed herein.

Claims

1. In an electronic device (101), Memory (130) for storing instructions; and At least one processor (120) that executes the above instructions Includes, When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) is enabled, Acquire content related to the creation of target content (450, 841), and Extract keywords from the above content, and Determine whether there exists at least one training DB (651, 653) related to the above keywords, and If at least one learning DB (651, 653) exists, objects (423, 425, 433, 435, 831, 833) related to the at least one learning DB (651, 653) are displayed, and A user's selection command for an object (423, 425, 433, 435, 831, 833) associated with at least one learning DB (651, 653) is received to select a target object, and Generating the target content (450, 841) including a target element generated based on a target learning DB corresponding to the target object, Electronic device (101).

2. In Paragraph 1, When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) is enabled, If the above content includes a command (1010) related to the creation of the above target content (450, 841) and the command (1010) includes location information in which the target learning DB is to be applied to the above target content (450, 841), then masking information (1020, 1110, 1120, 1230) representing the location information is created, and the above target content (450, 841) is created based on the masking information (1020, 1110, 1120, 1230). Electronic device (101).

3. In either Paragraph 1 or Paragraph 2, The above at least one learning DB (651, 653) is, The object corresponding to the keyword extracted from the images (610) stored in the electronic device (101) is obtained by learning the features of the object corresponding to the keyword, and is classified as a tag corresponding to the keyword. Electronic device (101).

4. In any one of paragraphs 1 through 3, When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) is enabled, If the above content includes reference content (811, 900) for generating the target content (450, 841), determining whether the reference content (811, 900) is personalized based on the metadata (813) of the reference content (811, 900), and extracting the keyword based on whether the reference content (811, 900) is personalized. Electronic device (101).

5. In any one of paragraphs 1 through 4, When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) is enabled, If the above reference content (811, 900) is determined to be personalized, the above reference content (811, 900) is modified based on the above target learning DB to generate the above target content (450, 841). Electronic device (101).

6. In any one of paragraphs 1 through 5, When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) is enabled, When masking information (1020, 1110, 1120, 1230) indicating the area and shape to which the target learning DB is applied to the target content (450, 841) is predetermined, the target content (450, 841) including the target element in the area and the shape is generated. Electronic device (101).

7. In any one of paragraphs 1 through 6, When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) is enabled, A reference content including a reference element (1220) corresponding to the above target element is generated, the reference element (1220) is identified in the reference content through semantic partitioning of the reference content, masking information (1020, 1110, 1120, 1230) indicating the location and shape where the reference element (1220) is generated is generated, and the target content (450, 841) is generated by generating the target element having the shape at the said location of the reference content based on the masking information (1020, 1110, 1120, 1230). Electronic device (101).

8. In the method of operating the electronic device (101), An action of obtaining content related to the creation of target content (450, 841); The operation of extracting keywords from the above content; An operation to determine whether at least one learning DB (651, 653) related to the above keyword exists; An operation to display objects (423, 425, 433, 435, 831, 833) associated with at least one learning DB (651, 653) when at least one learning DB (651, 653) exists; An operation to receive a user's selection command for an object (423, 425, 433, 435, 831, 833) associated with at least one learning DB (651, 653) and select a target object; and The operation of generating the target content (450, 841) including a target element generated based on a target learning DB corresponding to the target object. including, Method of operation.

9. In Paragraph 8, The operation of generating the above target content (450, 841) is, If the above content includes a command (1010) related to the creation of the above target content (450, 841) and the command (1010) includes location information where the target learning DB is to be applied to the above target content (450, 841), then masking information (1020, 1110, 1120, 1230) representing the location information is created, and the above target content (450, 841) is created based on the masking information (1020, 1110, 1120, 1230). Electronic device (101).

10. In either paragraph 8 or paragraph 9, The above at least one learning DB (651, 653) is, The object corresponding to the keyword extracted from the images (610) stored in the electronic device (101) is obtained by learning the features of the object corresponding to the keyword, and is classified as a tag corresponding to the keyword. Method of operation.

11. In any one of paragraphs 8 through 10, If the above content includes reference content (811, 900) for generating the target content (450, 841), the operation of determining whether the reference content (811, 900) is personalized based on the metadata (813) of the reference content (811, 900). Includes more, The operation of extracting keywords from the above content is, Extracting the keyword based on whether the above reference content (811, 900) is personalized, Method of operation.

12. In any one of paragraphs 8 through 11, The operation of generating the above target content (450, 841) is, If the above reference content (811, 900) is determined to be personalized, the above reference content (811, 900) is modified based on the above target learning DB to generate the above target content (450, 841). Method of operation.

13. In any one of paragraphs 8 through 12, The operation of generating the above target content (450, 841) is, When masking information (1020, 1110, 1120, 1230) indicating the area and shape to which the target learning DB is applied to the target content (450, 841) is predetermined, the target content (450, 841) including the target element in the area and the shape is generated. Method of operation.

14. In any one of paragraphs 8 through 13, The operation of generating the above target content (450, 841) is, The operation of generating reference content including a reference element (1220) corresponding to the above target element; An operation of identifying the reference element (1220) in the reference content through semantic division of the reference content; An operation to generate masking information (1020, 1110, 1120, 1230) indicating the location and shape where the reference element (1220) is generated; and An operation to generate the target content (450, 841) by generating the target element having the shape at the position of the reference content based on the masking information (1020, 1110, 1120, 1230). including more, Method of operation.

15. A non-transient computer-readable recording medium storing one or more computer programs comprising instructions for executing the method of any one of paragraphs 8 through 14.

Images