Electronic device for transforming subject included in image, and operation method thereof

Abstract

An electronic device is provided. The electronic device may comprise: a display; a memory that stores instructions; and a processor. The instructions, when executed by the processor, may cause the electronic device to: identify an input for transforming a first subject included in a plurality of first images stored in the memory to be related to a specific scene or a specific operation of first content stored in the memory; in response to the input, acquire at least one image related to the specific scene or the specific operation from among a plurality of second images related to the first content; acquire a text command for transforming the first subject to be related to the specific scene or the specific operation; and acquire an image including a second subject obtained by transforming the first subject so as to be related to the specific scene or the specific operation by inputting the text command into a first artificial intelligence model stored in the memory. Other various embodiments are possible.

Description

Electronic device for transforming a subject included in an image, and method of operation thereof

[0001] Embodiments of the present disclosure relate to an electronic device for transforming a subject included in an image, and a method of operating the same.

[0002] Thanks to remarkable advancements in information and communication technology and semiconductor technology, the proliferation and use of various electronic devices is rapidly increasing. Electronic devices are being developed to enable users to carry and communicate with one another. Electronic devices can refer to devices that perform various functions depending on the software installed on them, such as mobile communication terminals, tablet PCs, audio / video devices, desktop / laptop computers, or in-car navigation systems.

[0003] Recently, users have become more interested in editing images, beyond simply capturing them using electronic devices. Accordingly, electronic devices are increasingly offering image editing capabilities.

[0004] However, editing images containing other people's work can lead to issues of portrait rights infringement. Editing works protected by another person's copyright can also lead to copyright infringement.

[0005] According to one embodiment, an electronic device may include a display, memory, and a processor.

[0006] According to one embodiment, the electronic device can identify an input for transforming a first subject included in a plurality of first images stored in the memory to be related to a specific scene or a specific action of the first content stored in the memory.

[0007] According to one embodiment, the electronic device, in response to the input, may obtain at least one image related to the specific scene or the specific action from among a plurality of second images related to the first content.

[0008] According to one embodiment, the electronic device may obtain a text command to transform the first subject to relate to the specific scene or the specific action.

[0009] According to one embodiment, the electronic device may input the text command to the first artificial intelligence model stored in the memory to obtain an image including a second subject modified to relate the first subject to the specific scene or the specific action.

[0010] According to one embodiment, a method of operating an electronic device may include an operation of identifying an input for transforming a first subject included in a plurality of first images stored in a memory of the electronic device to be related to a specific scene or a specific action of first content stored in the memory.

[0011] According to one embodiment, a method of operating an electronic device may include, in response to the input, obtaining at least one image related to the specific scene or the specific action from among a plurality of second images related to the first content.

[0012] According to one embodiment, a method of operating an electronic device may include obtaining text for transforming the first subject to be related to the specific scene or the specific action.

[0013] According to one embodiment, a method of operating an electronic device may include inputting the text command to a first artificial intelligence model stored in the memory to obtain an image in which the first subject is transformed to be related to the specific scene or the specific action.

[0014] According to one embodiment, a non-transitory recording medium may store instructions that can execute an operation of identifying an input for transforming a first subject included in a plurality of first images stored in a memory of an electronic device to be related to a specific scene or a specific action of first content stored in the memory.

[0015] According to one embodiment, the non-transitory recording medium may store instructions that, in response to the input, may execute an operation of acquiring at least one image related to the specific scene or the specific action from among a plurality of second images related to the first content.

[0016] According to one embodiment, the non-transitory recording medium can store instructions that can execute an operation to obtain text for transforming the first subject to be related to the specific scene or the specific action.

[0017] According to one embodiment, the non-transitory recording medium may store instructions that can cause the first artificial intelligence model stored in the memory to perform an operation of obtaining an image including a second subject that is transformed so as to be related to the specific scene or the specific action by inputting the text command.

[0018] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0019] FIG. 2 is a schematic block diagram of an electronic device according to one embodiment.

[0020] FIG. 3 is a schematic block diagram of an image generation module according to one embodiment.

[0021] FIG. 4 is a flowchart illustrating an operation of an electronic device acquiring a first artificial intelligence model according to one embodiment.

[0022] FIG. 5 is a flowchart illustrating an operation of an electronic device acquiring an image using a first artificial intelligence model, according to one embodiment.

[0023] FIG. 6 is a flowchart illustrating an operation of an electronic device acquiring data for learning a first artificial intelligence model based on a purchase history for content, according to one embodiment.

[0024] FIG. 7 is a diagram illustrating an operation of an electronic device to obtain a first artificial intelligence model for transforming a subject to be related to content, according to one embodiment.

[0025] FIG. 8 is a diagram illustrating an operation of an electronic device to obtain artificial intelligence models for transforming a subject to be related to content, according to one embodiment.

[0026] FIG. 9 is a diagram illustrating an operation of an electronic device to confirm an input for transforming a subject to be related to content, according to one embodiment.

[0027] FIG. 10 is a diagram illustrating an operation of an electronic device obtaining a text command according to one embodiment.

[0028] FIG. 11 is a diagram illustrating an image acquired by an electronic device using a first artificial intelligence model according to one embodiment.

[0029] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with the electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (104) or the server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0030] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0031] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, in the electronic device (101) itself where artificial intelligence is performed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0032] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0033] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0034] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0035] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0036] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0037] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0038] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0039] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0040] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0041] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0042] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0043] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0044] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0045] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0046] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0047] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas by, for example, the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0048] In one embodiment, the antenna module (197) may generate a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

[0049] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0050] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0051] FIG. 2 is a schematic block diagram of an electronic device according to one embodiment.

[0052] Referring to FIG. 2, according to one embodiment, an electronic device (201) (e.g., the electronic device (101) of FIG. 1) may include a memory (210), a processor (220), and a display (260). According to one embodiment, the processor (220) may control the overall operation of the electronic device (201). According to one embodiment, the processor (220) may be implemented in the same or similar manner as the processor (120) of FIG. 1.

[0053] According to one embodiment, the electronic device (201) (or processor (220)) may generate an image by transforming an image (e.g., an image of a person) stored in the memory (210) (e.g., the memory (130) of FIG. 1)) to fit a specific scene or specific action of the content (e.g., a movie, an animation, a music video, a photograph, or a drawing) stored in the memory (210). In addition, the electronic device (201) (or processor (220)) may transform the image to fit the concept or style of the content stored in the memory (210). To this end, the electronic device (201) (or processor (220)) may use an artificial intelligence model (e.g., a first artificial intelligence model hereinafter) for transforming or generating an image.

[0054] According to one embodiment, the processor (220) may obtain a first artificial intelligence model (330) (e.g., the first artificial intelligence model (330) of FIG. 3) that is a fine-tuned version of a pre-learned reference artificial intelligence model (320) (e.g., a baseline model or a foundation model) (e.g., the artificial intelligence model (320) of FIG. 3) stored in the memory (210) (e.g., the memory (130) of FIG. 1). According to one embodiment, training data for fine-tuning the first artificial intelligence model may include images including subjects (e.g., people, animals, and / or objects) stored in the memory (210), and images related to content stored in the memory (210) (e.g., at least one image included in the content).

[0055] According to one embodiment, the processor (220) may obtain a command (e.g., a text command or a text prompt) to transform a subject included in an image so as to be related to a specific scene of the content or a specific action of the content, and to obtain a transformed subject from the subject included in the image. For example, the text command may represent a command in the form of text that the first artificial intelligence model (330) can recognize. For example, the text command may include a command to cause the first artificial intelligence model (330) to output an image including the transformed subject. According to one embodiment, the processor (220) may input the text command to the first artificial intelligence model (330) so as to transform a subject included in an image so as to be related to a specific scene of the content or a specific action of the content. According to one embodiment, the processor (220) may transform the subject into a subject performing the specific action, and may transform the subject by animating it.

[0056] According to one embodiment, the processor (220) may obtain a command (e.g., a text command or a text prompt) to transform a subject included in an image (e.g., an image of a person) stored in the memory (210) to match the concept of the content or the style of the content. For example, the concept of the content may include at least one of the composition, background, or color of a specific scene of the content. According to one embodiment, the processor (220) may generate an image including a new first scene or first action that is not included in the first content, based on the background, color, composition, or style included in at least one scene related to the first content.

[0057] Through this, the electronic device (201) according to one embodiment can output an image that does not infringe on the portrait rights of others and does not infringe on the copyright of others' works (e.g., content) by using the first artificial intelligence model (330).

[0058] According to one embodiment, the processor (220) may obtain a plurality of images including a plurality of subjects stored in the memory (210). For example, the processor (220) may obtain at least some of the plurality of images using a camera included in the electronic device (201). Alternatively, the processor (220) may receive or obtain at least some of the plurality of images from an external electronic device via a communication circuit included in the electronic device (201). According to one embodiment, the plurality of subjects may include people, objects, or animals. According to one embodiment, the processor (220) may use an image clustering technique to classify the plurality of images by images including the same subject (e.g., a person). According to one embodiment, the processor (220) may associate names (e.g., son, daughter, me, wife, our dog, dad, mom, etc.) with different subjects based on a user input.

[0059] According to one embodiment, the processor (220) may obtain at least one image related to content stored in the memory (210). According to one embodiment, the content may include a picture, a movie, a game, a cartoon, or a music video. According to one embodiment, the at least one image related to the content may include an image for a specific scene of the content, an image for a specific action provided in the content (e.g., a specific action of a main character), or a representative image representing the content (e.g., a poster, a thumbnail).

[0060] According to one embodiment, the processor (220) may obtain training data for training (e.g., fine-tuning) an artificial intelligence model (320) stored in the memory (210). According to one embodiment, the processor (220) may verify a user input for selecting training data from among a plurality of images including a plurality of subjects, and a user input for selecting training data from among a plurality of images related to content. Depending on the implementation, according to one embodiment, the training data for training the artificial intelligence model (320) may be automatically selected by the processor (220).

[0061] According to one embodiment, the processor (220) may acquire a first artificial intelligence model (320) by learning (or fine-tuning) the artificial intelligence model (320). The processor (220) may generate an image that is transformed from an image stored in the memory (210) (e.g., an image of a person) to a specific scene or specific action of content stored in the memory (e.g., a movie, animation, music video, photo, or picture) based on a user's request (e.g., a user input).

[0062] In one embodiment, the processor (220) may fine-tune the artificial intelligence model (320) with the first artificial intelligence model (330) to generate an image of a specific subject (e.g., a specific person) that is transformed to relate to a specific scene or specific action of the content.

[0063] Below, we will explain how the processor (220) fine-tunes the artificial intelligence model (320) into the first artificial intelligence model.

[0064] According to one embodiment, the processor (220) may confirm a user input for selecting a specific subject (e.g., a person) from among classified subjects (e.g., people) from among a plurality of images stored in the memory (210). For example, the processor (220) may classify the plurality of images based on names (e.g., son, daughter, me, wife, our dog, dad, mom, etc.) specified by the user or the processor (220). The processor (220) may display information (e.g., a list) about the classified names on the display (260). According to one embodiment, when a first subject (e.g., daughter) is selected by the user input, the processor (220) may acquire a plurality of first images including the first subject from among the plurality of images including the plurality of subjects.

[0065] According to one embodiment, the processor (220) may confirm a user input for selecting one of a plurality of contents stored in the memory (210) through the display (260). For example, the processor (220) may display identifiers (e.g., titles) of the plurality of contents through the display (260) and confirm a user input for selecting one of the identifiers (e.g., titles). The identifiers of the plurality of contents may include a game title, a movie title, a music video title, and / or a cartoon title. According to one embodiment, when a first content is selected from the plurality of contents, the processor (220) may obtain a plurality of second images related to the first content from among a plurality of images related to the plurality of contents. According to one embodiment, the plurality of second images may include a poster image of the first content, images representing specific scenes of the first content, and images representing specific actions of subjects of the first content.

[0066] According to one embodiment, the processor (220) may input a plurality of first images and a plurality of second images as learning data to the artificial intelligence model (320), thereby training (or fine-tuning) the artificial intelligence model (320). According to one embodiment, the processor (220) may obtain a first artificial intelligence model (330) (e.g., the first artificial intelligence model (330) of FIG. 3) as a result of training the artificial intelligence model (320). According to one embodiment, the first artificial intelligence model (330) may include an artificial intelligence model for transforming a first subject to be related to a first content.

[0067] According to one embodiment, the processor (220) may verify whether the first content is legitimately purchased by the electronic device (201). For example, if the processor (220) determines that the first content was acquired through an unverifiable path, the processor (220) may not use images of the first content as training data. For example, the processor (220) may not use images related to the first content as training data based on determining that there is no purchase history for the first content. According to one embodiment, if the processor (220) determines that the user of the electronic device (201) does not have the right to use the first content, the processor (220) may not use images of the first content as training data for the artificial intelligence model (320).

[0068] Below, an operation of the processor (220) to transform the first subject to be related to the first content using the first artificial intelligence model (330) after the first artificial intelligence model (330) is acquired will be described.

[0069] According to one embodiment, the processor (220) may identify an input for transforming a first subject included in a plurality of first images stored in the memory (210) to be related to a specific scene of the first content or a specific action.

[0070] According to one embodiment, the processor (220) may obtain at least one image (or multiple images) including the first subject based on verifying the input, and may obtain at least one image (or multiple images) related to a specific scene or a specific action of the first content.

[0071] According to one embodiment, the processor (220) may check whether there is a purchase history for the first content. According to one embodiment, if it is determined that there is a purchase history for the first content, the processor (220) may acquire at least one image related to a specific scene or a specific action of the first content. According to one embodiment, if it is determined that there is no purchase history for the first content, the processor (220) may not acquire at least one image related to a specific scene or a specific action of the first content. For example, if it is determined that the first content was acquired through an unverifiable path, the processor (220) may not acquire at least one image related to a specific scene or a specific action of the first content.

[0072] In one embodiment, the processor (220) may obtain a text command to be input into the first artificial intelligence model (330). In one embodiment, the text command may include a name that classifies the first subject (e.g., "daughter") and an identifier of the first content (e.g., "title"). For example, the identifier of the first content may represent A. For example, the text command may include "Using movie A, transform the daughter to be related to movie A."

[0073] According to one embodiment, the processor (220) may obtain a text command that includes a first subject and a specific scene or action of the first content. For example, the text command may include “Using a scene of a main character in movie A climbing a building’s exterior wall, transform the daughter to be related to the scene.” According to one embodiment, the processor (220) may obtain a text command that causes an image to be generated that includes a first scene or a first action that includes a first subject corresponding to the concept of the first content. For example, the concept of the first content may include at least one of the composition, background, or color of a specific scene of the first content. For example, the first scene or the first action may represent a new scene or new action not included in the first content. For example, the text command may include “Based on the concept of movie A, generate a scene of the daughter climbing a building’s exterior wall.” For example, a scene in which a subject (e.g., a subject included in the first content (Movie A)) climbs the exterior wall of a building may be a scene not included in Movie A.

[0074] According to one embodiment, the processor (220) may input a text command to the first artificial intelligence model (330) to obtain an image including a second subject that has been transformed so that the first subject is associated with a specific scene or a specific action.

[0075] According to one embodiment, the processor (220) may obtain an image including a transformed second subject by using at least one of at least one subject, composition, background, or color included in at least one image related to a specific scene or a specific action of the first content.

[0076] For example, an image including a modified second subject may have at least one of the composition, background, or color of at least one image related to a specific scene or a specific action of the first content.

[0077] According to one embodiment, the processor (220) may replace a subject included in at least one image related to a specific scene or a specific action of the first content with the first subject.

[0078] In one embodiment, when the first content is a drawing, an animated film, or a cartoon, the processor (220) may transform the first subject into a second subject having the same drawing style as the first content. For example, the processor (220) may transform the first subject into a second subject animated to match the drawing style of the first content.

[0079] According to one embodiment, the processor (220) may transform the first subject into a second subject wearing the same clothing as the subject included in the specific scene, using the body information of the subject included in the specific scene. For example, the body information may include at least one of muscles, height, face shape, eye shape, nose shape, ear shape, mouth shape, arm length, leg length, or torso length. According to one embodiment, the processor (220) may transform the first subject into a second subject so as to have the same facial expression as the subject included in the specific scene.

[0080] According to one embodiment, when a specific scene includes multiple subjects, the processor (220) may transform a first subject into a second subject wearing the same clothing as the subject corresponding to the main character, using the body information of the subject corresponding to the main character.

[0081] According to one embodiment, the processor (220) may input a text command to the first artificial intelligence model (330) to generate an image including a first scene or a first action including a first subject based on the concept of the first content. For example, the image including the first scene or the first action may be an image including a scene or action not included in the first content. According to one embodiment, the processor (220) may generate a first scene or a first action including a first subject based on a background, color, composition, or painting style included in at least one scene related to the first content. For example, the processor (220) may generate an image including a first action of the first subject in a background included in at least one scene related to the first content. For example, the processor (220) may generate an image including a new first scene or first action not included in the first content based on a background, color, composition, or painting style included in at least one scene related to the first content.

[0082] FIG. 3 is a schematic block diagram of an image generation module according to one embodiment.

[0083] Referring to FIG. 3, according to one embodiment, the image generation module (310) may be stored in memory (210) (e.g., memory (210) of FIG. 2). According to one embodiment, the image generation module (310) may be implemented in software. Depending on the implementation, at least a portion of the image generation module (300) may be implemented in hardware.

[0084] According to one embodiment, the image generation module (310) may include an artificial intelligence model (320) and a first artificial intelligence model (330). According to one embodiment, the artificial intelligence model (320) may include a pre-learned reference artificial intelligence model (e.g., a baseline model or a foundation model). According to one embodiment, the processor (220) (e.g., the processor (220) of FIG. 2) may obtain a first artificial intelligence model (330) that is a fine-tuned version of the artificial intelligence model (320).

[0085] According to one embodiment, the processor (220) may acquire a plurality of first images (340) including a first subject. For example, the first subject may include a person, an animal, or an object.

[0086] According to one embodiment, the processor (220) may obtain a plurality of second images (350) related to the first content. According to one embodiment, the first content may include a game, a movie, a music video, a cartoon, or a picture. According to one embodiment, the plurality of second images (350) may include a poster image of the first content, images including specific scenes of the first content, and images including specific actions of subjects of the first content.

[0087] According to one embodiment, the processor (220) may input a plurality of first images (340) and a plurality of second images (350) as learning data to the artificial intelligence model (320), thereby obtaining a first artificial intelligence model (330) for transforming a first subject to be related to a specific scene or specific action of the first content.

[0088] According to one embodiment, the processor (220) may input a plurality of first images (340) and a plurality of second images (350) as learning data to the artificial intelligence model (320), thereby obtaining a first artificial intelligence model (330) for generating an image including a first scene or a first action based on the first content, wherein the image includes a first subject based on the first content or includes a first action. For example, the image including a first scene or a first action based on the first content may include an image based on at least one of a composition, a background, a color, or a style of at least one scene included in the first content. For example, the first scene or the first action may include a scene or an action not included in the first content. For example, the processor (220) may generate an image including a new first scene or a first action not included in the first content, based on a background, a color, a composition, or a style included in at least one scene related to the first content.

[0089] In one embodiment, the processor (220) may obtain a text command for transforming a first subject to be related to a specific scene or a specific action of the first content. For example, the text command may include text or a command that causes the first artificial intelligence model (330) to output an image including a second subject transformed from the first subject. In one embodiment, the processor (220) may input the text command to the first artificial intelligence model (330) to obtain an image in which the first subject is transformed to be related to a specific scene or a specific action of the first content.

[0090] According to one embodiment, the processor (220) may input a text command to the first artificial intelligence model (330) to generate an image including a new first scene or first action not included in the first content. In this case, the image including the new first scene or first action not included in the first content may include an image based on at least one of the composition, background, color, or painting style of at least one scene included in the first content.

[0091] According to one embodiment, the processor (220) may acquire a plurality of third images (not shown) related to second content that is different from the first content. According to one embodiment, the second content may include a game, a movie, a music video, a cartoon, or a picture. According to one embodiment, the plurality of third images may include a poster image of the second content, images including specific scenes of the second content, and images including specific actions of subjects of the second content.

[0092] According to one embodiment, the processor (220) inputs a plurality of first images (340) and a plurality of third images as learning data to the artificial intelligence model (320), thereby obtaining a second artificial intelligence model (not shown) for transforming the first subject to be related to a specific scene or specific action of the second content.

[0093] According to one embodiment, the processor (220) may obtain a text command for transforming a first subject to be related to a specific scene or a specific action of the second content. For example, the text command may include text or a command that causes the second artificial intelligence model to output an image including a third subject transformed from the first subject. According to one embodiment, the processor (220) may input the text command to the second artificial intelligence model to generate an image including a new second scene or a second action not included in the second content. In this case, the image including the new second scene or the second action not included in the second content may include an image based on at least one of the composition, background, color, or painting style of at least one scene included in the second content.

[0094] In FIG. 3, a plurality of first images (340) including a first subject are described as training data for an artificial intelligence model (320), but an image including a subject different from the first subject can be used as training data for an artificial intelligence model (320), and the same description as for the plurality of first images (340) can be applied to an image including a subject different from the first subject.

[0095] In FIG. 3, a plurality of second images (350) related to the first content are described as learning data for the artificial intelligence model (320), but images related to content different from the first content can be used as learning data for the artificial intelligence model (320), and the same description as for the plurality of second images (350) can be applied to images related to content different from the first content.

[0096] The operations of the electronic device (201) described in the drawings below may be performed by the processor (220). However, for convenience of explanation, the operations performed by the processor (220) will be described as being performed by the electronic device (201).

[0097] FIG. 4 is a flowchart illustrating an operation of an electronic device acquiring a first artificial intelligence model according to one embodiment.

[0098] Referring to FIG. 4, according to one embodiment, in operation 411, an electronic device (201) (e.g., the electronic device (201) of FIG. 2) may acquire a plurality of first images including a first subject from among a plurality of images stored in a memory (210) (e.g., the memory (210) of FIG. 2). According to one embodiment, the first subject may include a person, an object, or an animal.

[0099] According to one embodiment, the electronic device (201) may use image clustering technology to classify multiple images into images containing the same subject. According to one embodiment, the electronic device (201) may associate names (e.g., son, daughter, me, wife, our dog, dad, mom, etc.) designated by the user or the processor (220) to different subjects based on user input.

[0100] According to one embodiment, the electronic device (201) can confirm a user input for selecting a specific subject from among classified subjects (e.g., people or animals) from among a plurality of images stored in the memory (210) through the display (260) (e.g., the display (260) of FIG. 2). For example, the electronic device (201) can display information about names classified in response to the plurality of subjects through the display (260). According to one embodiment, when a first subject (e.g., a daughter) is selected by the user input, the electronic device (201) can acquire a plurality of first images including the first subject from among the plurality of images.

[0101] According to one embodiment, in operation 413, the electronic device (201) may obtain a plurality of second images related to a first content from among a plurality of contents stored in the memory (210). According to one embodiment, the electronic device (201) may obtain the plurality of second images based on confirming a user input for selecting a first content from among the plurality of contents stored in the memory (210) through the display (260). For example, the first content may include a picture, a movie, a game, a cartoon, or a music video. For example, the electronic device (201) may display identifiers (e.g., titles) of the plurality of contents through the display (260) and confirm a user input for selecting one of the identifiers (e.g., titles). For example, the plurality of second images related to the first content may include images representing specific scenes of the first content and specific actions of subjects of the first content.

[0102] According to one embodiment, in operation 415, the electronic device (201) may input a plurality of first images and a plurality of second images as learning data into an artificial intelligence model (320) stored in a memory (210) (e.g., the artificial intelligence model (320) of FIG. 3), thereby obtaining a first artificial intelligence model (330) (e.g., the first artificial intelligence model (330) of FIG. 3).

[0103] In one embodiment, the artificial intelligence model (320) may include a pre-learned baseline model or foundation model. In one embodiment, the electronic device (201) may obtain a first artificial intelligence model (330) that is a fine-tuned version of the artificial intelligence model (320). In one embodiment, the first artificial intelligence model (330) may include an artificial intelligence model for transforming a first subject to be related to a first content.

[0104] FIG. 5 is a flowchart illustrating an operation of an electronic device acquiring an image using a first artificial intelligence model, according to one embodiment.

[0105] Referring to FIG. 5, according to one embodiment, in operation 511, the electronic device (201) (e.g., the electronic device (201) of FIG. 2) may confirm an input for transforming a first subject included in a plurality of first images stored in a memory (210) (e.g., the memory (210) of FIG. 2) so as to be related to first content. For example, transforming so as to be related to the first content may include transforming the first subject based on at least one of a composition, a background, a color, or a style of an image including a specific scene or a specific action included in the first content. According to one embodiment, the electronic device (201) may display a plurality of first indicators corresponding to a plurality of subjects and a plurality of second indicators corresponding to a plurality of contents through a display (260) (e.g., the display (260) of FIG. 2). According to one embodiment, the plurality of contents may include pictures, games, movies, music videos, and cartoons.

[0106] According to one embodiment, the plurality of first indicators may include names classified in correspondence with the plurality of subjects (e.g., son, daughter, me, wife, our dog, dad, mom, etc.). According to one embodiment, the electronic device (201) may confirm a user input for a first indicator corresponding to the first subject among the plurality of first indicators (e.g., daughter).

[0107] According to one embodiment, the plurality of second indicators may include identifiers (e.g., titles) of a plurality of contents. For example, the identifiers of the plurality of contents may include an identifier of a picture, an identifier of a game, an identifier of a movie, an identifier of a music video, and an identifier of a comic book. According to one embodiment, the electronic device (201) may verify a second input for a second indicator corresponding to the first content among the plurality of second indicators.

[0108] According to one embodiment, the electronic device (201) can determine an input for transforming the first subject to be related to the first content based on a user input selecting a first indicator corresponding to the first subject and a second indicator corresponding to the first content.

[0109] According to one embodiment, in operation 513, the electronic device (201) may verify an input for a specific scene or a specific action.

[0110] For example, the electronic device (201) may display a plurality of indicators corresponding to a plurality of scenes or a plurality of actions through the display (260). According to one embodiment, when a user input for any one of the plurality of indicators corresponding to the plurality of scenes or the plurality of actions is confirmed, the electronic device (201) may determine that an input for a specific scene or a specific action has been confirmed.

[0111] For example, the electronic device (201) may identify a text input for a specific scene or a specific action. According to one embodiment, in operation 515, the electronic device (201) may obtain a text command based on a first subject and a specific scene. According to one embodiment, the text command may include a text command that can be identified by the first artificial intelligence model (330) (e.g., the first artificial intelligence model (330) of FIG. 3 ).

[0112] In one embodiment, the text command may include a label classified corresponding to the first subject (e.g., daughter), an identifier of the first content (e.g., title), and an action or scene of the first subject to be transformed. For example, the text command may include “generate a scene (e.g., an action or scene of the first subject to be transformed) in which the daughter (e.g., a label classified corresponding to the first subject) climbs the outer wall of a building in a movie (e.g., an identifier of the first content).” For example, based on the painting style, background, composition, color, or subject of an image including a specific scene or a specific action of the first content, an image including a second subject transformed from the first subject may be obtained. For example, the scene of the subject climbing the outer wall of a building may be a scene not included in the first content.

[0113] In one embodiment, the text command may include a classified name corresponding to the first subject and an identifier (e.g., title) of the first content. For example, the text command may include “Transform the first subject using movie A.” For example, the electronic device (201) may use the first artificial intelligence model (330) to randomly determine an action or scene of the first subject based on the first content, thereby obtaining an image including the transformed second subject. For example, an image including the second subject that is the transformed first subject may be obtained based on the painting style, background, composition, color, or subject of an image including a specific scene or specific action of the first content.

[0114] According to one embodiment, in operation 517, the electronic device (201) may input a text command to the first artificial intelligence model (330) to obtain an image in which the first subject is transformed to be related to the first content.

[0115] According to one embodiment, the electronic device (201) may transform the first subject to be related to the first content by using at least one of the subject, composition, background, or color included in a plurality of second images included in the first content. According to one embodiment, the image including the transformed first subject may include an image having the same painting style as that of at least one image included in the plurality of second images, or an animated image.

[0116] FIG. 6 is a flowchart illustrating an operation of an electronic device acquiring data for learning a first artificial intelligence model based on a purchase history for content, according to one embodiment.

[0117] Referring to FIG. 6, according to one embodiment, in operation 611, an electronic device (201) (e.g., the electronic device (201) of FIG. 2) may acquire a plurality of first images including a first subject stored in a memory (210) (e.g., the memory (210) of FIG. 2).

[0118] According to one embodiment, in operation 613, the electronic device (201) may confirm an input to acquire a plurality of second images related to the first content. According to one embodiment, the electronic device (201) may display a plurality of indicators corresponding to the plurality of contents. According to one embodiment, the plurality of contents may include pictures, games, movies, music videos, and cartoons. According to one embodiment, the electronic device (201) may confirm a user input to select an indicator corresponding to the first content among the plurality of indicators corresponding to the plurality of contents. According to one embodiment, the electronic device (201) may confirm an input to acquire a plurality of second images related to the first content based on the user input to select the indicator corresponding to the first content.

[0119] In one embodiment, at operation 615, the electronic device (201) may, in response to an input to acquire a plurality of second images related to the first content, determine whether there is a purchase history for the first content. In one embodiment, the electronic device (201) may determine whether the first content is content that has been legitimately purchased by the electronic device (201).

[0120] According to one embodiment, in operation 617, the electronic device (201) may acquire a plurality of second images related to the first content as training data for the artificial intelligence model (320) (e.g., the artificial intelligence model (320) of FIG. 3) based on verifying a purchase history for the first content. According to one embodiment, if the electronic device (201) determines that there is no purchase history for the first content, the electronic device (201) may not acquire the plurality of second images related to the first content as training data for the artificial intelligence model (320). For example, if the electronic device (201) determines that the first content was acquired through an unverifiable path, the electronic device (201) may not use the plurality of second images related to the first content as training data for the artificial intelligence model (320). According to one embodiment, the electronic device (201) may not use a plurality of second images related to the first content as training data for the artificial intelligence model (320) based on determining that there is no purchase history for the first content.

[0121] According to one embodiment, the electronic device (201) can acquire a plurality of first images including a first subject as training data for an artificial intelligence model (320).

[0122] Through this, according to one embodiment, the electronic device (201) can obtain an image that does not infringe on the copyright of another person's work.

[0123] FIG. 7 is a diagram illustrating an operation of an electronic device to obtain a first artificial intelligence model for transforming a subject to be related to content, according to one embodiment.

[0124] Referring to (a) of FIG. 7, according to one embodiment, the electronic device (201) (e.g., the electronic device (201) of FIG. 2) may use an image clustering technique to classify a plurality of images including a plurality of subjects stored in the memory (210) (e.g., the memory (210) of FIG. 2) into images including the same subject. According to one embodiment, the electronic device (201) may, based on a user input, associate names designated by the user or the processor (220) with different subjects. For example, the designated names may include daughter, me, wife, and our dog.

[0125] According to one embodiment, the electronic device (201) can display a plurality of first indicators (711, 712, 713, 714) corresponding to a plurality of subjects. According to one embodiment, the plurality of first indicators (711, 712, 713, 714) can include indicators corresponding to different subjects. For example, the plurality of first indicators (711, 712, 713, 714) can display an indicator (711) representing me, an indicator (712) representing my wife, an indicator (713) representing my daughter, and an indicator (714) representing my dog. According to one embodiment, the plurality of first indicators (711, 712, 713, 714) may include names (e.g., me, wife, daughter, my dog) specified by user input or processor (220).

[0126] According to one embodiment, the electronic device (201) can verify a user input for an indicator (713) representing a daughter among a plurality of first indicators (711, 712, 713, 714).

[0127] According to one embodiment, the electronic device (201) may acquire a plurality of first images including a first subject corresponding to the daughter based on verifying a user input for an indicator (713) representing the daughter.

[0128] Referring to (b) of FIG. 7, according to one embodiment, the electronic device (201) may display a plurality of second indicators (721, 722, 723, 724) corresponding to a plurality of contents stored in the memory (210). For example, the plurality of second indicators (721, 722, 723, 724) may display an indicator (721) corresponding to a first content (e.g., movie A), an indicator (722) corresponding to a second content (e.g., movie B), an indicator (723) corresponding to a third content (e.g., music video C), and an indicator (724) corresponding to a fourth content (e.g., game D). For example, A and B may represent identifiers (e.g., titles) of movies. For example, C may represent identifiers (e.g., song titles) of songs. For example, D may represent an identifier for a game (e.g., the game title).

[0129] According to one embodiment, the electronic device (201) can confirm a user input for an indicator (721) corresponding to a first content (e.g., movie A) among a plurality of second indicators (721, 722, 723, 724). According to one embodiment, the plurality of second indicators (721, 722, 723, 724) can include identifiers (e.g., movie title, music video title, game title).

[0130] According to one embodiment, the electronic device (201) may acquire at least one image related to a specific scene or a specific action among a plurality of second images related to the first content based on verifying a user input for an indicator (721) corresponding to the first content (e.g., movie A).

[0131] According to one embodiment, the electronic device (201) may apply a visual effect to the plurality of second indicators (721, 722, 723, 724) so ​​that the plurality of second indicators (721, 722, 723, 724) are visually distinguishable from each other. According to one embodiment, the electronic device (201) may check whether there is a purchase history for each of the plurality of contents. According to one embodiment, the electronic device (201) may display, through the display (260), at least one first indicator corresponding to at least one first content without a purchase history among the plurality of second indicators (721, 722, 723, 724) so ​​as to be visually distinguishable from at least one second indicator corresponding to at least one second content with a purchase history among the plurality of second indicators (721, 722, 723, 724).

[0132] FIG. 8 is a diagram illustrating an operation of an electronic device to obtain artificial intelligence models for transforming a subject to be related to content, according to one embodiment.

[0133] Referring to (a) of FIG. 8, according to one embodiment, the electronic device (201) (e.g., the electronic device (201) of FIG. 2) may input a plurality of first images including a subject corresponding to the daughter and a plurality of second images related to the first content to the artificial intelligence model (320) (e.g., the artificial intelligence model (320) of FIG. 3) based on verifying a user input for an indicator (713) representing the daughter (e.g., 713 of FIG. 7) and a user input for an indicator (721) corresponding to the first content (e.g., A movie).

[0134] In one embodiment, the electronic device (201) may fine-tune the artificial intelligence model (320) to obtain a first artificial intelligence model (330) (e.g., the first artificial intelligence model (330) of FIG. 2 ). The first artificial intelligence model (330) may represent an artificial intelligence model for transforming a subject corresponding to a daughter to be related to the first content. In one embodiment, the artificial intelligence model (320) may be stored in the memory (210).

[0135] According to one embodiment, the electronic device (201) may display information (e.g., generating an avatar model) indicating that it is training an artificial intelligence model (320) through a display (260) (e.g., display (260) of FIG. 2).

[0136] According to one embodiment, the electronic device (201) may input a plurality of third images including a subject corresponding to me and a plurality of second images related to the first content to the artificial intelligence model (320) based on verifying a user input for an indicator (711) representing me (e.g., 711 of FIG. 7) and a user input for an indicator (721) corresponding to the first content (e.g., movie A).

[0137] In one embodiment, the electronic device (201) may fine-tune the artificial intelligence model (320) to obtain a second artificial intelligence model (810). The second artificial intelligence model (810) may represent an artificial intelligence model for transforming a subject corresponding to me to be related to the first content. In one embodiment, the second artificial intelligence model (810) may be stored in the memory (210).

[0138] According to one embodiment, the electronic device (201) may input a plurality of third images including a subject corresponding to me and a plurality of fourth images related to the second content to the artificial intelligence model (320) based on verifying a user input for an indicator (711) representing me (e.g., 711 of FIG. 7) and a user input for an indicator (722) corresponding to the second content (e.g., a B movie).

[0139] In one embodiment, the electronic device (201) may fine-tune the artificial intelligence model (320) to obtain a third artificial intelligence model (830). The third artificial intelligence model (not shown) may represent an artificial intelligence model for transforming a subject corresponding to me to be related to the second content. In one embodiment, the third artificial intelligence model (830) may be stored in the memory (210).

[0140] According to one embodiment, the electronic device (201) may input a plurality of first images including a subject corresponding to the daughter and a plurality of fourth images related to the second content to the artificial intelligence model (320) based on a user input for an indicator (713) representing the daughter (e.g., 713 of FIG. 7) and a user input for an indicator (722) corresponding to the second content (e.g., a B movie).

[0141] In one embodiment, the electronic device (201) may fine-tune the artificial intelligence model (320) to obtain a fourth artificial intelligence model (840). The fourth artificial intelligence model (840) may represent an artificial intelligence model for transforming a subject corresponding to the daughter to be related to the second content. In one embodiment, the fourth artificial intelligence model (840) may be stored in the memory (210).

[0142] In one embodiment, the electronic device (201) may not fine-tune the artificial intelligence model (320) even if a user input is confirmed for an indicator corresponding to content without a purchase history.

[0143] Referring to (b) of FIG. 8, according to one embodiment, the electronic device (201) can obtain a fine-tuned artificial intelligence model as a result of training the artificial intelligence model (320).

[0144] According to one embodiment, the electronic device (201) may obtain a first artificial intelligence model (820) (e.g., the first artificial intelligence model (330) of FIG. 3) that obtains an image in which a subject corresponding to the daughter is transformed to be related to the first content (e.g., movie A).

[0145] According to one embodiment, the electronic device (201) may obtain a second artificial intelligence model (810) that obtains an image in which a subject corresponding to me is transformed to be related to the first content (e.g., movie A).

[0146] According to one embodiment, the electronic device (201) may obtain a third artificial intelligence model (830) that obtains an image in which the subject corresponding to the artificial intelligence model (320) is transformed to be related to second content (e.g., a B movie).

[0147] According to one embodiment, the electronic device (201) may obtain a fourth artificial intelligence model (840) that obtains an image in which a subject corresponding to the daughter is transformed to be related to a second content (e.g., a B movie).

[0148] FIG. 9 is a diagram illustrating an operation of an electronic device to confirm an input for generating a text prompt to transform a subject into a content-related object, according to one embodiment.

[0149] Referring to (a) of FIG. 9, according to one embodiment, an electronic device (201) (e.g., the electronic device (201) of FIG. 2) may display a plurality of first indicators (911, 912, 913, 914) corresponding to a plurality of subjects. According to one embodiment, the plurality of first indicators (911, 912, 913, 914) may include indicators corresponding to different subjects. For example, the plurality of first indicators (911, 912, 913, 914) may display an indicator (911) representing me, an indicator (912) representing my wife, an indicator (913) representing my daughter, and an indicator (914) representing my dog.

[0150] According to one embodiment, the electronic device (201) can verify a user input for an indicator (913) representing a daughter among a plurality of first indicators (711, 712, 713, 714).

[0151] Referring to (b) of FIG. 9, according to one embodiment, the electronic device (201) may display a plurality of second indicators (921, 922, 923, 924) corresponding to a plurality of contents in which the artificial intelligence model (320) is fine-tuned using images of a subject corresponding to the daughter based on an input of an indicator (913) representing the daughter. For example, the plurality of second indicators (921, 922, 923, 924) may display an indicator (921) corresponding to a first content (e.g., movie A), an indicator (922) corresponding to a second content (e.g., movie B), an indicator (923) corresponding to a third content (e.g., music video C), and an indicator (924) corresponding to a fourth content (e.g., game D). For example, A and B may represent identifiers (e.g., titles) of movies. For example, C may represent an identifier for a song (e.g., the song title). For example, D may represent an identifier for a game (e.g., the game title).

[0152] According to one embodiment, the electronic device (201) can verify a user input for an indicator (921) corresponding to a first content (e.g., movie A).

[0153] According to one embodiment, the electronic device (201) may apply a visual effect to the plurality of second indicators (921, 922, 923, 924) so ​​that the plurality of second indicators (921, 922, 923, 924) are visually distinguishable from each other. According to one embodiment, the electronic device (201) may check whether there is a purchase history for each of the plurality of contents. According to one embodiment, the electronic device (201) may display, through the display (260), an indicator corresponding to a content without a purchase history among the plurality of second indicators (921, 922, 923, 924) so ​​as to be visually distinguishable from an indicator corresponding to a content with a purchase history among the plurality of second indicators (921, 922, 923, 924).

[0154] According to one embodiment, the electronic device (201) can identify a first artificial intelligence model (330) stored in the memory (210) based on a user input for an indicator (913) representing a daughter and an indicator (921) corresponding to a first content (e.g., movie A).

[0155] For example, when a user input for an indicator (913) representing a daughter and an indicator (922) corresponding to second content (e.g., a B movie) is confirmed, the electronic device (201) can check the fourth artificial intelligence model (840) stored in the memory (210) (e.g., the fourth artificial intelligence model (840) of FIG. 8).

[0156] For example, when a user input for an indicator (913) representing me and an indicator (921) corresponding to the first content (e.g., movie A) is confirmed, the electronic device (201) can check the second artificial intelligence model (810) (e.g., the second artificial intelligence model (810) of FIG. 8) stored in the memory (210).

[0157] For example, when a user input for an indicator (913) representing me and an indicator (922) corresponding to second content (e.g., B movie) is confirmed, the electronic device (201) can check a third artificial intelligence model (830) stored in the memory (210) (e.g., the third artificial intelligence model (830) of FIG. 8).

[0158] FIG. 10 is a diagram illustrating an operation of an electronic device obtaining a text command according to one embodiment.

[0159] Referring to FIG. 10, according to one embodiment, when a user input for an indicator (913) representing a daughter (e.g., 913 of FIG. 9) and a user input for an indicator (921) representing first content (e.g., A movie) (e.g., 921 of FIG. 9) are confirmed, the electronic device (201) (e.g., the electronic device (201) of FIG. 2) may check a first artificial intelligence model (330) (e.g., the first artificial intelligence model (330) of FIG. 3) stored in a memory (210) (e.g., the memory (210) of FIG. 2).

[0160] In one embodiment, the electronic device (201) may obtain a text command to transform an object corresponding to the daughter so as to be associated with a first content (e.g., a movie). For example, the text command may include the object to be transformed and a specific scene or a specific action.

[0161] For example, text command (1011) may include generating a scene of the daughter fighting a villain. For example, text command (1012) may include generating a scene of the daughter climbing the outer wall of a building. For example, text command (1013) may include generating a scene of the daughter shooting a spider web.

[0162] According to one embodiment, the electronic device (201) can display text commands (1011, 1012, 1013) through a display (260) (e.g., the display (260) of FIG. 2). According to one embodiment, the electronic device (201) can confirm a user input for a text command (1012) among the text commands (1011, 1012, 1013).

[0163] Depending on the implementation, according to one embodiment, the electronic device (201) may randomly acquire any one of the text commands (1011, 1012, 1013) without displaying the text commands (1011, 1012, 1013) through the display (260).

[0164] According to one embodiment, the electronic device (201) may obtain text input from a user for a specific scene or a specific action through the display (260).

[0165] FIG. 11 is a diagram illustrating an image acquired by an electronic device using a first artificial intelligence model according to one embodiment.

[0166] Referring to (a) of FIG. 11, according to one embodiment, an electronic device (201) (e.g., the electronic device (201) of FIG. 2) may input a text command (1012) (e.g., 1012 of FIG. 10) into a first artificial intelligence model (330) (e.g., the first artificial intelligence model (330) of FIG. 3) stored in a memory (210) (e.g., the memory (210) of FIG. 2).

[0167] Referring to (b) of FIG. 11, according to one embodiment, the electronic device (201) may obtain a third image in which the first subject is transformed to be related to movie A using the first artificial intelligence model (330). According to one embodiment, the electronic device (201) may obtain a third image of a scene in which the subject corresponding to the daughter climbs the outer wall of a building.

[0168] According to one embodiment, the electronic device (201) can obtain a transformed subject by transforming the first subject so that it is identical to at least one of the style, color, composition, and background of the subject included in the second image (e.g., the subject of movie A).

[0169] According to one embodiment, an electronic device may include a display, memory, and a processor.

[0170] According to one embodiment, the electronic device can identify an input for transforming a first subject included in a plurality of first images stored in the memory to be related to a specific scene or a specific action of the first content stored in the memory.

[0171] According to one embodiment, the electronic device, in response to the input, may obtain at least one image related to the specific scene or the specific action from among a plurality of second images related to the first content.

[0172] According to one embodiment, the electronic device may obtain a text command to transform the first subject to relate to the specific scene or the specific action.

[0173] According to one embodiment, the electronic device can input the text command to the first artificial intelligence model stored in the memory to obtain an image including a second subject modified to relate the first subject to the specific scene or the specific action.

[0174] According to one embodiment, the electronic device can acquire the plurality of first images including the first subject stored in the memory.

[0175] According to one embodiment, the electronic device can obtain the plurality of second images related to the first content stored in the memory.

[0176] According to one embodiment, the electronic device may input the plurality of first images and the plurality of second images as learning data into an artificial intelligence model stored in the memory, thereby obtaining the first artificial intelligence model for transforming the first subject to be related to the specific scene or the specific action.

[0177] According to one embodiment, the electronic device can obtain the first artificial intelligence model by fine-tuning the artificial intelligence model.

[0178] According to one embodiment, the electronic device may transform the first subject to be related to the specific scene or the specific action by using at least one of the subject, composition, background, or color included in the at least one image.

[0179] According to one embodiment, the electronic device can display, through the display, a plurality of first indicators corresponding to the plurality of subjects and a plurality of second indicators corresponding to the plurality of contents.

[0180] According to one embodiment, the electronic device can identify a first input for a first indicator corresponding to the first subject among the plurality of first indicators, and a second input for a second indicator corresponding to the first content among the plurality of second indicators.

[0181] According to one embodiment, the electronic device can identify an input for transforming the first subject to be related to the specific scene or the specific action, based on identifying the first input and the second input.

[0182] In one embodiment, the electronic device, in response to the input, can determine whether there is a purchase history for the first content.

[0183] According to one embodiment, the electronic device can obtain the at least one image based on determining that there is a purchase history for the first content.

[0184] According to one embodiment, the electronic device may input the plurality of second images as learning data to the artificial intelligence model based on determining that there is a purchase history for the first content.

[0185] According to one embodiment, the electronic device may display, through the display, at least one first indicator corresponding to at least one first content without a purchase history among the plurality of second indicators so as to be visually distinct from at least one second indicator corresponding to at least one second content with a purchase history among the plurality of second indicators.

[0186] According to one embodiment, the electronic device can obtain the image including the second subject, which includes a background identical to the background of the specific scene, and the first subject is transformed to take the action of the subject of the first content included in the specific scene.

[0187] According to one embodiment, the electronic device may, when the first content is an animation, transform the first subject into the second subject based on a character of the animation.

[0188] According to one embodiment, a method of operating an electronic device may include an operation of identifying an input for transforming a first subject included in a plurality of first images stored in a memory of the electronic device to be related to a specific scene or a specific action of first content stored in the memory.

[0189] According to one embodiment, a method of operating an electronic device may include, in response to the input, obtaining at least one image related to the specific scene or the specific action from among a plurality of second images related to the first content.

[0190] According to one embodiment, a method of operating an electronic device may include obtaining a text command for transforming the first subject to be related to the specific scene or the specific action.

[0191] According to one embodiment, a method of operating an electronic device may include inputting the text command to a first artificial intelligence model stored in the memory, thereby obtaining an image including a second subject in which the first subject is transformed to be related to the specific scene or the specific action.

[0192] According to one embodiment, a method of operating an electronic device may include an operation of acquiring a plurality of first images including the first subject stored in the memory.

[0193] According to one embodiment, a method of operating an electronic device may include obtaining a plurality of second images related to the first content stored in the memory.

[0194] According to one embodiment, the method of operating an electronic device may include an operation of inputting the plurality of first images and the plurality of second images as learning data into an artificial intelligence model stored in the memory, thereby obtaining the first artificial intelligence model for transforming the first subject to be related to the first content.

[0195] According to one embodiment, a method of operating an electronic device may include an operation of fine-tuning the artificial intelligence model to obtain the first artificial intelligence model.

[0196] According to one embodiment, a method of operating an electronic device may include an operation of transforming the first subject to be related to the specific scene or the specific action by using at least one of at least one subject, composition, background, or color included in the at least one image.

[0197] According to one embodiment, a method of operating an electronic device may include an operation of displaying, through a display included in the electronic device, a plurality of first indicators corresponding to the plurality of subjects and a plurality of second indicators corresponding to the plurality of contents.

[0198] According to one embodiment, a method of operating an electronic device may include an operation of confirming a first input for a first indicator corresponding to the first subject among the plurality of first indicators, and a second input for a second indicator corresponding to the first content among the plurality of second indicators.

[0199] According to one embodiment, a method of operating an electronic device may include an operation of identifying an input for transforming the first subject to be related to the specific scene or the specific action, based on identifying the first input and the second input.

[0200] According to one embodiment, the method of operating an electronic device may include, in response to the input, an operation of checking whether there is a purchase history for the first content.

[0201] According to one embodiment, a method of operating an electronic device may include an operation of acquiring the at least one image based on determining that there is a purchase history for the first content.

[0202] According to one embodiment, a method of operating an electronic device may include inputting the plurality of second images as learning data into the artificial intelligence model based on determining that there is a purchase history for the first content.

[0203] According to one embodiment, the method of operating an electronic device may include an operation of displaying, through the display, at least one first indicator corresponding to at least one first content without a purchase history among the plurality of second indicators so as to be visually distinct from at least one second indicator corresponding to at least one second content with a purchase history among the plurality of second indicators.

[0204] According to one embodiment, a method of operating an electronic device may include obtaining an image including a background identical to a background of the specific scene and including a first deformed subject that is deformed to assume a motion of a subject included in the specific scene.

[0205] According to one embodiment, the method of operating an electronic device may include, when the first content is an animation, an operation of transforming the first subject into the second subject based on a character of the animation.

[0206] According to one embodiment, a non-transitory recording medium may store instructions that can execute an operation of identifying an input for transforming a first subject included in a plurality of first images stored in a memory of an electronic device to be related to a specific scene or a specific action of first content stored in the memory.

[0207] According to one embodiment, the non-transitory recording medium may store instructions that, in response to the input, may execute an operation of acquiring at least one image related to the specific scene or the specific action from among a plurality of second images related to the first content.

[0208] According to one embodiment, the non-transitory recording medium may store instructions that can execute an operation to obtain a text command for transforming the first subject to be related to the specific scene or the specific action.

[0209] According to one embodiment, the non-transitory recording medium may store instructions that can cause the first artificial intelligence model stored in the memory to perform an operation of obtaining an image including a second subject that is transformed so as to be related to the specific scene or the specific action by inputting the text command.

[0210] According to one embodiment, the non-transitory recording medium can store instructions that can execute an operation of acquiring the plurality of first images including the first subject stored in the memory.

[0211] According to one embodiment, the non-transitory recording medium can store instructions that can execute an operation of obtaining the plurality of second images related to the first content stored in the memory.

[0212] According to one embodiment, the non-transitory recording medium may store instructions that can execute an operation of obtaining the first artificial intelligence model for transforming the first subject to be related to the first content by inputting the plurality of first images and the plurality of second images as learning data to the artificial intelligence model stored in the memory.

[0213] According to one embodiment, the non-transitory recording medium can store instructions that can execute an operation to obtain the first artificial intelligence model by fine-tuning the artificial intelligence model.

[0214] According to one embodiment, the non-transitory recording medium may store instructions that can execute an operation of transforming the first subject to be related to the specific scene or the specific action by using at least one of the subject, composition, background, or color included in the at least one image.

[0215] According to one embodiment, a non-transitory recording medium may store instructions that can execute an operation of displaying a plurality of first indicators corresponding to the plurality of subjects and a plurality of second indicators corresponding to the plurality of contents through a display included in the electronic device.

[0216] According to one embodiment, the non-transitory recording medium may store instructions that can execute an operation of confirming a first input for a first indicator corresponding to the first subject among the plurality of first indicators, and a second input for a second indicator corresponding to the first content among the plurality of second indicators.

[0217] According to one embodiment, the non-transitory recording medium may store instructions that, based on identifying the first input and the second input, may execute an operation of identifying an input for transforming the first subject to be related to the specific scene or the specific action.

[0218] According to one embodiment, the non-transitory recording medium may store instructions that, in response to the input, may perform an operation of checking whether there is a purchase history for the first content.

[0219] According to one embodiment, the non-transitory recording medium may store instructions that can execute an operation of acquiring the at least one image based on determining that there is a purchase history for the first content.

[0220] According to one embodiment, the non-transitory recording medium may store instructions that can execute an operation of inputting the plurality of second images as training data to the artificial intelligence model based on determining that there is a purchase history for the first content.

[0221] According to one embodiment, the non-transitory recording medium may store instructions that can execute an operation to obtain the image, which includes a background identical to the background of the specific scene and includes the first deformed subject that is deformed to assume the motion of the subject included in the specific scene.

[0222] According to one embodiment, the non-transitory recording medium may store instructions that, when the first content is an animation, can execute an operation of transforming the first subject into the second subject based on a character of the animation.

[0223] According to one embodiment, the non-transitory recording medium may store instructions that can execute an operation of displaying, through the display, at least one first indicator corresponding to at least one first content without a purchase history among the plurality of second indicators so as to be visually distinct from at least one second indicator corresponding to at least one second content with a purchase history among the plurality of second indicators.

[0224] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments disclosed in this document are not limited to the aforementioned devices.

[0225] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0226] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0227] Various embodiments of the present document may be implemented as software (e.g., program (140)) including one or more commands stored in a storage medium (e.g., built-in memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101, 201)). For example, a processor (e.g., processor (120, 220)) of a machine (e.g., electronic device (101, 201)) may call at least one command among the one or more commands stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the called at least one command. The one or more commands may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means a device in which the storage medium is tangible, It simply means that it does not contain signals (e.g. electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on a storage medium.

[0228] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0229] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (201), display (260); Memory (210) for storing instructions; and Contains a processor (220), The above instructions, when executed by the processor, cause the electronic device to: Identifying an input for transforming a first subject included in a plurality of first images stored in the memory to be related to a specific scene or specific action of the first content stored in the memory, In response to the input, at least one image related to the specific scene or the specific action is acquired from among a plurality of second images related to the first content, Obtaining a text command to transform the first subject to be related to the specific scene or the specific action, and By inputting the text command to the first artificial intelligence model (330) stored in the memory, an image including the second subject transformed so that the first subject is related to the specific scene or the specific action is obtained. Electronic devices.

2. In paragraph 1, The above instructions, when executed by the processor, cause the electronic device to: Acquire the plurality of first images including the first subject stored in the memory, Obtaining the plurality of second images related to the first content stored in the memory, and By inputting the plurality of first images and the plurality of second images as learning data into the artificial intelligence model (320) stored in the memory, the first artificial intelligence model is obtained for transforming the first subject to be related to the specific scene or the specific action. Electronic devices.

3. In any one of paragraphs 1 and 2, The above instructions, when executed by the processor, cause the electronic device to: To obtain the first artificial intelligence model by fine-tuning the artificial intelligence model, Electronic devices.

4. In any one of paragraphs 1 to 3, The above instructions, when executed by the processor, cause the electronic device to: Using at least one of the subject, composition, background, or color included in the at least one image, transforming the first subject to be related to the specific scene or the specific action. Electronic devices.

5. In any one of paragraphs 1 to 4, The above instructions, when executed by the processor, cause the electronic device to: Through the above display, a plurality of first indicators corresponding to the plurality of subjects and a plurality of second indicators corresponding to the plurality of contents are displayed, Confirming a first input for a first indicator corresponding to the first subject among the plurality of first indicators, and a second input for a second indicator corresponding to the first content among the plurality of second indicators, and Based on the verification of the first input and the second input, an input is verified to transform the first subject to be related to the specific scene or the specific action. Electronic devices.

6. In any one of paragraphs 1 to 5, The above instructions, when executed by the processor, cause the electronic device to: In response to the above input, check whether there is a purchase history for the first content, and An electronic device that causes the acquisition of at least one image based on determining that there is a purchase history for said first content.

7. In any one of paragraphs 1 to 6, The above instructions, when executed by the processor, cause the electronic device to: Based on the confirmation that there is a purchase history for the above first content, inputting the plurality of second images as learning data into the artificial intelligence model. Electronic devices.

8. In any one of paragraphs 1 to 7, The above instructions, when executed by the processor, cause the electronic device to: Acquire the image including the second subject, which includes a background identical to the background of the specific scene, and the first subject is transformed to take the action of the subject of the first content included in the specific scene. Electronic devices.

9. In any one of paragraphs 1 to 8, The above instructions, when executed by the processor, cause the electronic device to: If the first content is an animation, transforming the first subject into the second subject based on a character of the animation. Electronic devices.

10. In any one of paragraphs 1 to 9, The above instructions, when executed by the processor, cause the electronic device to: Through the above display, at least one first indicator corresponding to at least one first content without a purchase history among the plurality of second indicators is displayed so as to be visually distinguished from at least one second indicator corresponding to at least one second content with a purchase history among the plurality of second indicators. Electronic devices.

11. In the operating method of an electronic device (201), An operation of confirming an input for transforming a first subject included in a plurality of first images stored in a memory (210) of the electronic device to be related to a specific scene or specific action of the first content stored in the memory; In response to said input, an action of obtaining at least one image related to said specific scene or said specific action from among a plurality of second images related to said first content; An action of obtaining a text command for transforming said first subject to be related to said specific scene or said specific action; and An operation of inputting the text command to the first artificial intelligence model (330) stored in the memory to obtain an image including the second subject modified to be related to the specific scene or the specific action of the first subject. How an electronic device operates.

12. In paragraph 11, An operation of acquiring the plurality of first images including the first subject stored in the memory; An operation of obtaining a plurality of second images related to the first content stored in the memory; and Further comprising an operation of inputting the plurality of first images and the plurality of second images as learning data into the artificial intelligence model (320) stored in the memory to obtain the first artificial intelligence model for transforming the first subject to be related to the first content. How an electronic device operates.

13. In any one of paragraphs 11 to 12, Further comprising at least one operation of an electronic device according to any one of claims 3 to 10; How an electronic device operates.

Images