Electronic device, method, and recording medium for supporting image generation

Abstract

An embodiment of the present disclosure provides an electronic device, a method, and a recording medium for supporting image generation. The electronic device according to an embodiment may display an image on a display. In response to a first input for selecting a portion corresponding to a face image in the image, the electronic device may display an interface for editing the face image. On the basis of a second input for selecting an emoji included in the interface, the electronic device may identify information related to the selected emoji and including an index and / or semantic information related to the emoji. The electronic device may generate a text prompt indicating a feature of an emoji to be applied to the face image so as to perform, on the basis of information related to the emoji, inpainting and / or outpainting on the face image of the selected portion. The electronic device may acquire a result image in relation to the text prompt and display the result image on the display. Various embodiments are possible.

Description

Electronic devices, methods, and recording media supporting image generation

[0001] Embodiments of the present disclosure provide an electronic device, an operating method thereof, and a recording medium that support image generation based on artificial intelligence (AI) (e.g., generative AI).

[0002] With the advancement of digital technology, various types of electronic devices, such as smartphones, digital cameras, and / or wearable devices, are becoming widely used. The hardware and / or software components of these electronic devices are continuously being developed to support and enhance their functionality.

[0003] For example, portable electronic devices (hereinafter referred to as "electronic devices"), such as smartphones, can now be equipped with a variety of functions. Electronic devices include touchscreen-based displays that allow users to easily access various functions, and can display screens for various applications through these displays.

[0004] Recently, with the rapid development of big data and deep learning technologies, artificial intelligence (AI) has been applied to electronic devices. It is also being applied to intelligent personal services that analyze specific data and integrate and utilize information from various fields tailored to the user. For example, users can control electronic devices through voice conversations, and a deep learning-based knowledge base enables them to search for, query, and respond to specific information. Recently, generative AI has been implemented as AI technology evolves. Generative AI can refer to AI technology that generates similar content using existing content, such as text, audio, and / or images. For example, generative AI can refer to AI technology that can generate content (e.g., text, audio, images, and / or video) that responds to a given input.

[0005] Meanwhile, in order to generate content in generative AI, a prompt (or instruction) that instructs the generative AI to generate content is required, and the prompt can be generated based on input provided by the user. For example, in order to obtain the content desired by the user through generative AI, it is necessary to configure an optimal prompt.

[0006] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above-described matters constitute prior art related to the present disclosure.

[0007] In one embodiment of the present disclosure, an electronic device, an operating method thereof, and a recording medium supporting image generation (e.g., image reproduction or reconstruction) based on generative AI are provided.

[0008] In one embodiment of the present disclosure, an electronic device, a method of operating the same, and a recording medium are provided that generate a prompt (e.g., a text prompt) related to image generation based on a graphic object (e.g., an emoji), and generate (e.g., regenerate or reconstruct) an image from a server or on-device based on the prompt.

[0009] The technical problems to be achieved in this document are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0010] An electronic device according to an embodiment of the present disclosure may include a display, at least one processor including processing circuitry, and a memory storing instructions (or commands). In one embodiment, the memory may store instructions that, when individually and / or collectively executed by the at least one processor, cause the electronic device to perform operations.

[0011] In one embodiment, the instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to display an image on the display. The instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to receive a first input selecting a portion of the image corresponding to a face image. The instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to, in response to the first input selecting the portion of the image corresponding to the face image, display an interface for editing the face image. In one embodiment, the interface may include a plurality of user interface (UI) items including emojis. The instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to identify, based on a second input selecting the emoji included in the interface, information related to the selected emoji, including an index and / or semantic information related to the selected emoji. The instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to generate a text prompt indicating a feature of the emoji to be applied to the facial image, such that inpainting and / or outpainting of the facial image of the selected portion is performed based on the information related to the emoji.The instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to obtain a resulting image in relation to the text prompt. The instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to display the resulting image on the display.

[0012] A method of operating an electronic device according to an embodiment of the present disclosure may include an operation of displaying an image on the display. The method may include an operation of receiving a first input for selecting a portion corresponding to a face image in the image. The method may include an operation of displaying an interface for editing the face image in response to the first input for selecting the portion corresponding to the face image in the image. According to one embodiment, the interface may include a plurality of UI (user interface) items including emojis. The method may include an operation of identifying information related to the selected emoji, the information including an index and / or semantic information related to the emoji, based on a second input for selecting the emoji included in the interface. The method may include generating a text prompt indicating a feature of the emoji to be applied to the facial image, based on information related to the emoji, to perform inpainting and / or outpainting on the facial image of the selected portion. The method may include obtaining a resulting image in relation to the text prompt. The method may include displaying the resulting image on the display.

[0013] In order to solve the above-described problem, various embodiments of the present disclosure may include a computer-readable recording medium having recorded thereon a program for executing the method on at least one processor.

[0014] According to one embodiment, a non-transitory computer-readable recording medium (or storage medium or computer program product) storing one or more programs is described. According to one embodiment, one or more programs are configured to: display an image on the display; receive a first input for selecting a portion of the image corresponding to a face image; display, in response to the first input for selecting the portion of the image corresponding to the face image, an interface for editing the face image, the interface including a plurality of UI (user interface) items including emojis; identify, based on a second input for selecting the emoji included in the interface, information related to the selected emoji including an index and / or semantic information related to the emoji; generate, based on the information related to the emoji, a text prompt indicating a feature of the emoji to be applied to the face image so as to perform inpainting and / or outpainting on the face image of the selected portion; obtain a resulting image in relation to the text prompt; and may include a command (or instructions) that performs an action of displaying the result image on the display.

[0015] Further scope of the applicability of the present disclosure will become apparent from the detailed description below. However, since various modifications and variations within the spirit and scope of the present disclosure will readily become apparent to those skilled in the art, it should be understood that the detailed description and specific examples, such as preferred embodiments of the present disclosure, are given by way of example only.

[0016] According to an embodiment of the present disclosure, an electronic device, an operating method thereof, and a recording medium thereof, it is possible to support generation of an optimal prompt (or instruction) (e.g., a text prompt) that enables a user to obtain desired content (e.g., an image) through generative artificial intelligence.

[0017] According to one embodiment, an electronic device can easily change / apply various facial expressions of a person in an image according to the user's intention using emojis. According to one embodiment, the electronic device can intuitively and quickly change a designated object (e.g., a facial image) within a profile image or an image (e.g., a gallery image or a photo) to a desired facial expression and mood. According to one embodiment, the user can select an emoji related to changing the facial image through an interface including emojis, and easily generate a prompt (e.g., a text prompt) of a generative artificial intelligence based on an index (e.g., Unicode) and / or semantic information (e.g., shape (or form) information, type information, expression information, emotion information, and / or action information related to the emoji) related to the selected emoji. According to one embodiment, the electronic device can transmit the prompt to the generative artificial intelligence to generate an image (e.g., a result image) desired by the user.

[0018] According to one embodiment, an electronic device can intuitively provide information necessary for generating a prompt through a graphical object, thereby providing convenience in prompt generation. According to one embodiment, when generating an image (e.g., regenerating or reconstructing) based on generative artificial intelligence, a user can quickly and easily configure a prompt using a graphical object, and obtain an optimal result image desired by the user. According to one embodiment, when generating a prompt for image generation based on generative artificial intelligence, an electronic device can provide a new user experience (UX) that allows the user to quickly and easily generate a desired image by providing easy and convenient input using a graphical object.

[0019] In addition, various effects may be directly or indirectly realized through this document. The effects obtained through this disclosure are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art to which this disclosure pertains, based on the description below.

[0020] In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar components.

[0021] FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment of the present disclosure.

[0022] FIG. 2 is a block diagram illustrating an integrated intelligence system according to one embodiment of the present disclosure.

[0023] FIG. 3 is a diagram schematically illustrating the configuration of an electronic device according to one embodiment of the present disclosure.

[0024] FIG. 4 is a flowchart illustrating a method of operating an electronic device according to one embodiment of the present disclosure.

[0025] FIG. 5 is a diagram illustrating an example of an execution screen of an application in an electronic device according to one embodiment of the present disclosure.

[0026] FIG. 6 is a diagram illustrating an example of an interface that supports input related to prompt generation in an electronic device according to one embodiment of the present disclosure.

[0027] FIG. 7 is a diagram illustrating an example of an interface that supports execution of image generation in an electronic device according to one embodiment of the present disclosure.

[0028] FIG. 8 is a diagram illustrating an example of an interface during image generation in an electronic device according to one embodiment of the present disclosure.

[0029] FIG. 9 is a diagram illustrating an example of providing a result image in an electronic device according to one embodiment of the present disclosure.

[0030] FIG. 10 is a diagram illustrating an example of providing a result image in an electronic device according to one embodiment of the present disclosure.

[0031] FIG. 11 is a flowchart illustrating a method of operating an electronic device according to one embodiment of the present disclosure.

[0032] FIGS. 12A, 12B, and 12C are diagrams illustrating an example of an operation of image generation in an electronic device according to one embodiment of the present disclosure.

[0033] FIGS. 13A, 13B, 13C, and 13D are diagrams illustrating an example of an operation of image generation in an electronic device according to one embodiment of the present disclosure.

[0034] FIG. 14 is a flowchart illustrating a method of operating an electronic device according to one embodiment of the present disclosure.

[0035] FIG. 15 is a diagram illustrating an example of generating an image based on characteristics of a graphic object in an electronic device according to one embodiment of the present disclosure.

[0036] FIG. 16 is a flowchart illustrating a method of operating an electronic device according to one embodiment of the present disclosure.

[0037] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0038] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to one embodiment of the present disclosure.

[0039] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0040] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit (CPU) or an application processor (AP)) or an auxiliary processor (123) (e.g., a graphic processing unit (GPU), a neural processing unit (NPU), an image signal processor (ISP), a sensor hub processor, or a communication processor (CP)) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0041] The auxiliary processor (123) may control at least a part of functions or states related to at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0042] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0043] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (OS) (142), middleware (144), or an application (146).

[0044] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0045] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0046] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0047] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0048] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0049] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, or an audio interface.

[0050] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0051] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0052] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0053] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least a part of a power management integrated circuit (PMIC).

[0054] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0055] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0056] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). NR access technology can support high-speed transmission of high-capacity data (eMBB, enhanced mobile broadband), minimizing terminal power and connecting multiple terminals (mMTC, massive machine type communications), or high reliability and low latency communications (URLLC, ultra-reliable and low-latency communications). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0057] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas by, for example, the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0058] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

[0059] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0060] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0061] FIG. 2 is a block diagram illustrating an integrated intelligence system according to one embodiment of the present disclosure.

[0062] Referring to FIG. 2, an integrated intelligent system of one embodiment may include an electronic device (201) (e.g., the electronic device (101) of FIG. 1), an intelligent server (300), and a service server (399).

[0063] According to the illustrated embodiment, the electronic device (201) may include a communication interface (210), an input / output (I / O) interface (220), a processor (230), and / or a memory (240). The components listed above may be operatively or electrically connected to each other. For example, the electronic device (201) may include at least some of the components of the electronic device (101) of FIG. 1.

[0064] The communication interface (210) can be connected to an external device (e.g., an intelligent server (300) and / or a service server (399)) via a network (299) (e.g., any network including a cellular network and / or a wireless local area network (WLAN)) to transmit and receive data. For example, the communication interface (210) can correspond to the CP and / or communication circuit of FIG. 1. The I / O interface (220) can receive user input, process received user input, and / or output a result processed by the processor (230) using an input / output device (not shown) (e.g., a microphone, a speaker, and / or a display (e.g., a display of FIG. 1).

[0065] The processor (230) may be operatively or electrically connected to the communication interface (210), the I / O interface (220), and / or the memory (240) (e.g., the memory of FIG. 1) to perform a designated operation. For example, the processor (230) may correspond to the processor (120) of FIG. 1. The processor (230) may execute a program (or one or more instructions) stored in the memory (240) to perform a designated operation. For example, the processor (230) may receive a user's voice input (e.g., a user's speech) through the I / O interface (220) or from an external electronic device. The processor (230) may transmit the voice input received through the communication interface (210) to the intelligent server (300). For example, the processor (230) may include one or more processors.

[0066] The processor (230) may receive a result corresponding to the voice input from the intelligent server (300). For example, the processor (230) may receive a plan corresponding to the voice input and / or a result calculated using the plan from the intelligent server (300). For example, the plan may include, but is not limited to, information regarding a plurality of sequential operations to be executed by the electronic device (201) and / or another electronic device in relation to the voice input. The processor (230) may receive a request from the intelligent server (300) to obtain information (e.g., entities, slots, and / or parameters) necessary to generate a plan corresponding to the voice input. The processor (230) may transmit the necessary information to the intelligent server (300) in response to the request.

[0067] The processor (230) may visually, tactilely, and / or audibly output the results of executing the operations specified according to the plan via the I / O interface (220). For example, the processor (230) may sequentially display the execution results of multiple operations on the display. As an example, the processor (230) may display only the execution results of executing multiple operations (e.g., the execution results of one of the multiple operations or the last operation) on the display.

[0068] The processor (230) can recognize voice input. For example, the processor (230) can execute an intelligent app (or a voice recognition app) to process the voice input in response to a specified voice input (e.g., "Wake up!"). The processor (230) can provide a voice recognition service through the intelligent app. The processor (230) can transmit the voice input to the intelligent server (300) through the intelligent app and receive a result corresponding to the voice input from the intelligent server (300).

[0069] An intelligent server (300) of one embodiment can receive a user's voice input from an electronic device (201) via a network (299). The intelligent server (300) can convert audio data corresponding to the received voice input into text data. The intelligent server (300) can generate at least one plan for performing a task corresponding to the user's voice input based on the text data. The intelligent server (300) can transmit the generated plan or a result according to the generated plan to the electronic device (201) via the network (299).

[0070] An intelligent server (300) of one embodiment may include a front end (310), a natural language platform (320), a capsule database (330), an execution engine (340), and / or an end user interface (350).

[0071] The front end (310) can receive a voice input received by the electronic device (201) from the electronic device (201). The front end (310) can transmit a response corresponding to the voice input to the electronic device (201).

[0072] The natural language platform (320) may include an automatic speech recognition (ASR) module (321), a natural language understanding (NLU) module (323), a planner module (325), a natural language generator (NLG) module (327), and / or a text-to-speech (TTS) module (329).

[0073] The automatic speech recognition module (321) can convert the voice input received from the electronic device (201) into text data. The natural language understanding module (323) can identify the user's intent and / or parameters (e.g., entities and / or slots) based on the text data of the voice input. The user's intent corresponds to the voice input and may include information indicating an action (or function) that the user wishes to perform using the device. The slot may be detailed information related to the user's intent. The slot may be acquired based on a domain corresponding to the utterance. The slot may be variable information required to perform the action. In one embodiment, the variable information constituting the slot may include a named entity.

[0074] The planner module (325) can generate a plan using the intent and / or parameters determined by the natural language understanding module (323). For example, the planner module (325) can determine at least one domain necessary to perform a task based on the determined intent. The planner module (325) can determine a plurality of operations included in each of the at least one domain determined based on the intent. The domain may correspond to a category (or service) associated with an operation (or function) that the user wishes to perform using the device. The domain may be classified according to a service (e.g., an app) related to the text. The domain may be related to the user's intent corresponding to the text. The domain may be classified according to, for example, the application that received the voice input and / or the type of service to be provided based on the voice input, but is not limited thereto. In one example, the determination of the domain may be performed by another module (e.g., the natural language understanding module (323)). The planner module (325) can determine parameters required to execute a plurality of determined actions or result values ​​output by the execution of the plurality of actions. The parameters and result values ​​can be defined as concepts of a specified format (or class). For example, the plan can include a plurality of actions and / or a plurality of concepts determined by the user's intention. The planner module (325) can determine the relationship between the plurality of actions and / or the plurality of concepts in a step-by-step (or hierarchical) manner. For example, the planner module (325) can identify the execution order of the plurality of actions (e.g., the plurality of actions determined based on the user's intention) based on the plurality of concepts (e.g., parameters required to execute the plurality of actions and results output by the execution of the plurality of actions). The planner module (325) can generate a plan including association information (e.g., ontology) between the plurality of actions and the plurality of concepts.The planner module (325) can create a plan using information (e.g., at least one capsule) stored in a capsule database (330) in which a set of relationships between concepts and actions is stored.

[0075] The planner module (325) can generate a plan based on an artificial intelligence (AI) system. For example, the AI ​​system can include one or more electronic devices and / or one or more processing circuits to execute a rule-based system, a neural network-based system (e.g., a feedforward neural network (FNN), a recurrent neural network (RNN)), or a combination thereof. The AI ​​system described above is exemplary, and the AI ​​system can be an AI system based on any machine learning-based model. The planner module (325) can select a plan corresponding to a user request from a set of predefined plans, or generate a plan in real time in response to a user request.

[0076] The natural language generation module (327) can convert specified information into text format. The information converted into text format may be in the form of natural language speech. The text-to-speech conversion module (329) can convert information in text format into information in speech format.

[0077] The capsule database (330) can store information on the relationship between multiple concepts and actions corresponding to multiple domains (e.g., applications). The capsule database (330) can store at least one capsule (e.g., capsule (331) and / or capsule (333)) in the form of a concept action network (CAN). For example, the capsule database (330) can store actions for processing tasks corresponding to a user's voice input and parameters required for the actions in the form of a CAN. A capsule can include multiple action objects (or action information) and / or concept objects (or concept information) included in a plan. For example, capsules (331, 333) can be created for each domain and stored in the capsule database (330), but are not limited thereto.

[0078] The execution engine (340) can produce results using the generated plan. The end user interface (350) can transmit the produced results to the electronic device (201).

[0079] According to one embodiment, some functions (e.g., the natural language platform (320)) or all functions of the intelligent server (300) may be implemented in the electronic device (201). For example, the electronic device (201) may execute one or more programs including the natural language platform (e.g., the natural language platform (320) of FIG. 3) separately from the intelligent server (300). For example, the electronic device (201) may directly perform at least some of the operations of the natural language platform (320) of the intelligent server (300) (e.g., the automatic speech recognition module (321), the natural language understanding module (323), the planner module (325), the natural language generation module (327), and / or the text-to-speech module (329)).

[0080] In one embodiment, a service server (399) may provide a service (e.g., food ordering or hotel reservation) designated to an electronic device (201). The service server (399) may be a server operated by a different operator than the intelligent server (300). The service server (399) may communicate with the intelligent server (300) and / or the electronic device (201) via a network (299). The service server (399) may communicate with the intelligent server (300) via a separate connection (not shown). The service server (399) may provide the intelligent server (300) with information for generating a plan corresponding to a voice input received in the electronic device (201) (e.g., operation information and / or concept information for providing a designated service). The provided information may be stored in a capsule database (330). The service server (399) may provide the intelligent server (300) with result information according to the plan received from the electronic device (201).

[0081] FIG. 3 is a diagram schematically illustrating the configuration of an electronic device according to one embodiment of the present disclosure.

[0082] Referring to FIG. 3, an electronic device (101) according to one embodiment of the present disclosure may include a display (490) (e.g., the display module (160) of FIG. 1 or the I / O interface (220) of FIG. 2), a memory (130) (e.g., the memory (130, 240) of FIG. 1 or 2), a communication circuit (495) (e.g., the communication module (190) or the communication interface (210) of FIG. 1), and / or a processor (120) (e.g., the processor (120, 230) of FIG. 1 or 2). According to one embodiment, the electronic device (101) may include all or at least a part of the components of the electronic device (101, 201) described in the description with reference to FIG. 1 or 2. For example, in various embodiments of the present document, some of the illustrated configurations may be omitted or replaced. The electronic device (101) may include at least some of the configurations and / or functions of the electronic device (101) of FIG. 1 and / or the electronic device (201) of FIG. 2. At least some of the respective configurations of the illustrated (or not illustrated) electronic devices (101, 201) may be operatively, functionally, and / or electrically connected to each other.

[0083] According to one embodiment, the display (490) may include a configuration identical or similar to the display module (160) of FIG. 1. According to one embodiment, the display (490) may display various images provided from the processor (120). According to one embodiment, the display (490) may visually provide, under the control of the processor (120), an application being executed (e.g., the application (146) of FIG. 1) and various screens related to its use (e.g., a contents screen, an application execution screen, a menu screen, and / or a function execution screen).

[0084] According to one embodiment, the display (490) may be combined with a touch sensor, a pressure sensor capable of measuring the intensity of a touch, and / or a touch panel (e.g., a digitizer) that detects a magnetic stylus pen. According to one embodiment, the display (490) may detect a touch input, an air gesture input, and / or a hovering input (or a proximity input) by measuring a change in a signal (e.g., voltage, light intensity, resistance, electromagnetic signal, and / or charge) for a specific location of the display (490) based on the touch sensor, the pressure sensor, and / or the touch panel. For example, the display (490) may include a touchscreen that detects a touch and / or a proximity touch (or a hovering) input using a part of a user's body (e.g., a finger) or an input device (e.g., a stylus pen). The display (490) may include at least some of the configuration and / or functions of the display module (160) of FIG. 1 and / or the I / O interface (220) of FIG. 2.

[0085] In one embodiment, the display (490) may include, but is not limited to, a liquid crystal display (LCD), a light-emitting diode (LED), an organic light-emitting diode (OLED) display, and / or an active matrix OLED (AMOLED) display, a micro electro mechanical systems (MEMS) display, or an electronic paper display. In one embodiment, the display (490) may include a flexible display.

[0086] According to one embodiment, the memory (130) includes at least a portion of the configuration and / or function of the memory (130) of FIG. 1 and / or the memory (240) of FIG. 2, and may store software (e.g., the program (140) of FIG. 1). The memory (130) may store various applications (e.g., the application (146) of FIG. 1) and program modules (e.g., client modules) that support intelligent services.

[0087] According to one embodiment, the memory (130) may store various data used by at least one component (e.g., processor (120)) of the electronic device (101). In one embodiment, the data may include, for example, software (e.g., program (140) of FIG. 1), and input data or output data for commands related to the software.

[0088] According to one embodiment, the memory (130) may include volatile memory (e.g., volatile memory (132) of FIG. 1) or non-volatile memory (134) (e.g., non-volatile memory (134) of FIG. 1). According to one embodiment, the memory (130) may store instructions or data received from the processor (120) in the volatile memory (132), and may store result data of instructions or data stored in the volatile memory (132) being processed by the processor (120) in the non-volatile memory (134).

[0089] In one embodiment, the data may include various data (e.g., training data, prompt data, context, and / or learning models) to support the electronic device (101) in editing and generating (e.g., regenerating or reconstructing) AI-based data (e.g., images). In one embodiment, the data may include information regarding various settings to support the electronic device (101) in controlling the operation of editing and / or generating AI-based data (e.g., images).

[0090] In one embodiment, the data may include various learning data and / or parameters acquired based on the user's learning through interaction with the user. In one embodiment, the data may include various schemas (or algorithms, models, networks, or functions) for supporting AI-based image editing and / or creation operations.

[0091] For example, a scheme for supporting artificial intelligence-based image editing and / or creation operations in an electronic device (101) may include a neural network. In one embodiment, the neural network may include a neural network model based on at least one of an artificial neural network (ANN), a convolution neural network (CNN), a region with convolution neural network (R-CNN), a region proposal network (RPN), a recurrent neural network (RNN), a stacking-based deep neural network (S-DNN), a state-space dynamic neural network (S-SDNN), a deconvolution network, a deep belief network (DBN), a restricted Boltzman machine (RBM), a long short-term memory (LSTM) network, a classification network, a plain residual network, a dense network, a hierarchical pyramid network, and / or a fully convolutional network. According to one embodiment, the type of the neural network model is not limited to the examples described above.

[0092] According to one embodiment, the memory (130) may store instructions that, when executed, cause the processor (120) to operate. The memory (130) may store instructions that, when individually and / or collectively executed by the processor (120), cause the electronic device (101) to perform operations.

[0093] According to one embodiment, the memory (130) comprises instructions that, when individually and / or collectively executed by the processor (120), cause the electronic device (101) to display an image on the display (490), receive a first input for selecting a portion of the image corresponding to a face image, display an interface for editing the face image (e.g., the interface includes a plurality of UI (user interface) items including emojis), and, based on a second input for selecting an emoji included in the interface, identify information related to the selected emoji including an index and / or semantic information related to the emoji, and, based on the information related to the emoji, perform inpainting and / or outpainting on the face image of the selected portion. Instructions can be stored to generate, obtain a resulting image in relation to a text prompt, and display the resulting image on the display (490).

[0094] According to one embodiment, the memory (130) comprises instructions that, when individually and / or collectively executed by the processor (120), cause the electronic device (101) to display an image on a display, receive a first input for selecting a portion corresponding to at least one object in the image, select the portion corresponding to at least one object in the image based on the first input, display an interface on the display (490) including at least one emoji related to editing of the at least one object in the selected portion of the image, receive a second input for selecting at least one emoji to apply to the at least one object in the selected portion based on the interface, determine (or identify) at least one piece of information related to an index, a shape, and / or a feature point (e.g., semantic information) related to the at least one emoji selected based on the second input, and generate a prompt based on a third input for performing inpainting and / or outpainting of the at least one object in the selected portion based on the at least one emoji and the at least one piece of information, obtain a result image in relation to the prompt, and display the result image on the display (490). Instructions can be stored.

[0095] According to one embodiment, the memory (130) may store instructions that, when individually and / or collectively executed by the processor (120), cause the electronic device (101) to display an execution screen of an application on the display (490), receive a first input for selecting an information input portion related to image generation on the execution screen, display an interface for inputting a graphic object (e.g., an emoji) on the execution screen based on the first input, receive a second input for selecting at least one graphic object based on the interface, select at least one graphic object (e.g., an emoji) based on the second input, and generate a prompt (e.g., a text prompt) for performing image generation based on at least one graphic object corresponding to the second input based on a third input for generating a result image, obtain a result image in relation to the prompt, and display the result image on the display (490).

[0096] For example, the instructions may be stored as software (e.g., program (140) of FIG. 1) on the memory (130) and may be executable by the processor (120). For example, the instructions may include control commands such as arithmetic and logical operations, data movement, and / or input / output that may be recognized by the processor (120). According to one embodiment, the software may include various applications (e.g., application (146) of FIG. 1) that may provide various functions (or services) (e.g., routine functions, call functions, message functions, messenger functions, e-mail functions, SNS (social networking service) functions, search functions, media (e.g., video and / or music) playback functions, game functions, and / or wireless communication functions) in the electronic device (101).

[0097] According to one embodiment, the communication circuit (495) may support the establishment of a designated wireless communication channel (e.g., short-range communication such as Bluetooth communication and / or BLE communication) and the performance of communication through the established wireless communication channel. For example, the communication circuit (495) may perform designated communication (e.g., Bluetooth communication and / or BLE communication) with an external device. According to one embodiment, the communication circuit (495) may support wireless communication with an external device using cellular wireless communication (e.g., 4G LTE, 5G NR) and / or short-range wireless communication (e.g., Wi-Fi). For example, the electronic device (101) may communicate with an external server (e.g., a generative artificial intelligence server) that provides an artificial intelligence-based function (or service) through a network using the communication circuit (495). According to one embodiment, the communication circuit (495) may transmit data (e.g., images and / or prompts) generated in the electronic device (101) to the external server and receive data (e.g., images) transmitted from the external server. According to one embodiment, the communication circuit (495) may include at least some of the configuration and / or functionality of the communication module (190) of FIG. 1 and / or the communication interface (210) of FIG. 2.

[0098] According to one embodiment, the processor (120) may perform an application layer processing function requested by a user of the electronic device (101). According to one embodiment, the processor (120) may provide control and commands of functions for various blocks of the electronic device (101). According to one embodiment, the processor (120) may perform operations or data processing related to control and / or communication of each component of the electronic device (101). For example, the processor (120) may include at least some of the configurations and / or functions of the processor (120) of FIG. 1. According to one embodiment, the processor (120) may be operatively connected to the components of the electronic device (101). According to one embodiment, the processor (120) may load commands or data received from other components of the electronic device (101) into the memory (130), process the commands or data stored in the memory (130), and store result data.

[0099] According to one embodiment, the processor (120) may include at least one processor including processing circuitry and / or executable program elements. According to one embodiment, the processor (120) may control (or process) the overall operation related to supporting the function of the electronic device (101) (e.g., generating a prompt (e.g., a text prompt) based on a graphic object (or graphic element) (e.g., an emoji) and generating an image through the generated prompt) based on the processing circuitry and / or the executable program elements.

[0100] In one embodiment, the processor (120) may display an image on the display (490). In one embodiment, the processor (120) may receive a first input for selecting a portion corresponding to a face image in the image. In one embodiment, the processor (120) may display an interface for editing the face image in response to the first input for selecting a portion corresponding to the face image in the image. In one embodiment, the interface may include a plurality of UI (user interface) items including emojis.

[0101] According to one embodiment, the processor (120) may receive a second input for selecting an emoji included in the interface. According to one embodiment, the processor (120) may identify, based on the second input for selecting an emoji included in the interface, an index related to the emoji, and / or information related to the selected emoji, including semantic information. In one embodiment, the semantic information may include shape (or form) information (e.g., shape information or image), type information (e.g., information about a person, animal, or object), facial expression information (or expression information) (e.g., information such as wink, sleepy, open-mouthed, grinning, and / or crying), emotion information (e.g., information such as joy, happiness, sadness, depression, and / or anger), and / or action information (e.g., action information such as praying, running, waving, meditating, and / or saluting) related to the graphical object (or graphical element) (e.g., emoji). According to one embodiment, the processor (120) may determine meaningful semantic information of a graphic object based on object analysis (e.g., emoji analysis) of the graphic object and / or an index (e.g., Unicode) of the graphic object. For example, the processor (120) may determine features (e.g., semantic information) of the graphic object (e.g., emoji) based on object analysis from the graphic object, and may determine (or infer) meaningful information of the graphic object based on the features. In one embodiment, the features may include meaningful features (e.g., emotion, action, and / or shape) that can be extracted from the graphic object (e.g., emoji).

[0102] In one embodiment, the processor (120) may generate a text prompt indicating a feature of the emoji to be applied to the facial image to perform inpainting and / or outpainting on the selected portion of the facial image based on information related to the emoji. In one embodiment, the processor (120) may obtain a resulting image in relation to the text prompt. In one embodiment, the processor (120) may display the resulting image on the display (490).

[0103] According to one embodiment, the processor (120) may display an image on the display (490). According to one embodiment, the processor (120) may receive a first input for selecting a portion corresponding to at least one object in the image. According to one embodiment, the processor (120) may select a portion corresponding to at least one object in the image based on the first input. According to one embodiment, the processor (120) may display an interface on the display (490) that includes at least one emoji related to editing of at least one object in the selected portion of the image. According to one embodiment, the processor (120) may receive a second input for selecting at least one emoji to be applied to at least one object in the selected portion based on the interface. According to one embodiment, the processor (120) may determine at least one piece of information related to an index, a shape, and / or a feature point (e.g., semantic information) related to the at least one emoji selected based on the second input. In one embodiment, the processor (120) may generate a prompt to perform inpainting and / or outpainting of at least one object of a selected portion based on at least one emoji and at least one piece of information, based on a third input. In one embodiment, the processor (120) may obtain a result image in relation to the prompt and display the result image on the display (490).

[0104] According to one embodiment, the processor (120) may display an execution screen of an application on the display (490). According to one embodiment, the processor (120) may receive a first input for selecting an information input section related to image generation on the execution screen. According to one embodiment, the processor (120) may display an interface for inputting graphic objects on the execution screen based on the first input. According to one embodiment, the processor (120) may receive a second input for selecting at least one graphic object based on the interface. According to one embodiment, the processor (120) may select at least one graphic object based on the second input. According to one embodiment, the processor (120) may generate a prompt for performing image generation based on at least one graphic object corresponding to the second input based on a third input for generating a result image. According to one embodiment, the processor (120) may obtain a result image in relation to the prompt. According to one embodiment, the processor (120) may display the result image on the display (490).

[0105] In one embodiment, the processor (120) may receive information input. In one embodiment, the processor (120) may detect the execution of image generation. In one embodiment, the processor (120) may analyze (or extract) information for prompt generation. In one embodiment, the processor (120) may determine whether a graphic object is included. In one embodiment, if a graphic object is not included, the processor (120) may generate a prompt to execute image generation based on the extracted information. In one embodiment, if a graphic object is included, the processor (120) may determine whether a target image is included. In one embodiment, if a target image is included, the processor (120) may determine the graphic object, the target image, and parameter information as prompt inputs. In one embodiment, the processor (120) may generate a prompt to inpaint and / or outpaint the target image based on the graphic object. In one embodiment, if a target image is not included, the processor (120) may determine the graphic object and parameter information as prompt inputs. In one embodiment, the processor (120) may generate a prompt to inpaint and / or outpaint based on a graphical object. In one embodiment, the processor (120) may obtain a result image in connection with the prompt.

[0106] According to one embodiment, the detailed operation of the processor (120) of the electronic device (101) is described with reference to the drawings described below.

[0107] According to one embodiment, the processors (120) can operate individually and / or collectively.

[0108] According to one embodiment, the processor (120) may include an application processor (AP) and / or a communication processor. According to one embodiment, the communication processor may be included in and operate in the communication circuit (495). According to one embodiment, the processor (120) may be an application processor. For example, the processor (120) may be a system semiconductor that is responsible for various functions (e.g., calculation and multimedia driving functions) of the electronic device (101). According to one embodiment, the processor (120) may be configured in the form of a system-on-chip (SoC), and may include a technology-intensive semiconductor chip (e.g., an application processor) that integrates various semiconductor technologies into one and implements system blocks into a single chip.

[0109] According to one embodiment, the system blocks of the processor (120) may include components such as a graphics processing unit (GPU) (410), an image signal processor (ISP) (420), a central processing unit (CPU) (430), a neural processing unit (NPU) (440), a digital signal processor (DSP) (450), a modem (460), connectivity (470), and / or security (480), as illustrated in FIG. 3.

[0110] In one embodiment, the GPU (410) may be responsible for graphics processing. In one embodiment, the GPU (410) may receive commands from the CPU (430) and perform graphics processing to express the shape, position, color, shading, movement, and / or texture of objects (or objects) on the display (490).

[0111] In one embodiment, the ISP (420) may be responsible for image processing and correction of images and videos. In one embodiment, the ISP (420) may correct raw data (e.g., raw data) transmitted from an image sensor of a camera (e.g., the camera module (180) of FIG. 1) to generate an image in a form more preferred by the user. In one embodiment, the ISP (420) may perform post-processing, such as adjusting partial brightness of an image and emphasizing detailed parts. For example, the ISP (420) may independently perform a process of tuning and correcting the image quality of an image acquired through a camera to generate a result preferred by the user.

[0112] According to one embodiment, the ISP (420) may support artificial intelligence (AI)-based image processing technology. According to one embodiment, the ISP (420) may support scene segmentation (e.g., image segmentation) technology that recognizes and / or classifies parts of a scene being captured in conjunction with the NPU (440). For example, the ISP (420) may include a function that applies different parameters to objects such as the sky, bushes, and / or skin and processes them. According to one embodiment, the ISP (420) may detect and display a human face during image capture using the AI ​​function, or adjust the brightness, focus, and / or color of the image using the coordinates and information of the face.

[0113] According to one embodiment, the CPU (430) can perform operations corresponding to the processor (120). According to one embodiment, the CPU (430) can decode a user's command, perform arithmetic and logical operations, and / or data processing operations. For example, the CPU (430) can be responsible for memory, interpretation, calculation, and control functions. According to one embodiment, the CPU (430) can control the overall function of the electronic device (101). For example, the CPU (430) can execute all software (e.g., application (146) of FIG. 1) of the electronic device (101) on an operating system (OS) (e.g., operating system (142) of FIG. 1) and control hardware devices. According to one embodiment, the CPU (430) can execute an application and control the overall operation of the processor (120) to perform neural network-based tasks required according to the execution of the application.

[0114] According to one embodiment, the CPU (430) may store instructions or data in volatile memory (e.g., volatile memory (132) of FIG. 1) of the memory (130), process the instructions or data stored in the volatile memory, and store resultant data in nonvolatile memory (e.g., nonvolatile memory (134) of FIG. 1) of the memory (130) as at least part of data processing or calculation.

[0115] According to one embodiment, the CPU (430) may include a single processor core or multiple processor cores (multi-core). According to one embodiment, the CPU (430) may be a programmable processor that stores executable instructions (e.g., instructions capable of performing operations of the CPU (430)) and executes the instructions.

[0116] According to one embodiment, the CPU (430) can operate in a multi-domain environment. According to one embodiment, the CPU (430) can operate in a multi-domain environment of a normal world (e.g., a non-secure world, a framework, or a non-secure environment) and a secure world (e.g., a secure framework or a secure environment). In one embodiment, a domain of the secure world can include one or more domains (e.g., a trusted OS, a trust zone, and / or a virtualization framework).

[0117] According to one embodiment, the NPU (440) can perform processing optimized for artificial intelligence deep-learning algorithms. According to one embodiment, the NPU (440) is a processor optimized for deep-learning algorithm operations (e.g., artificial intelligence operations) and can process big data quickly and efficiently like a human neural network. For example, the NPU (440) can be mainly used for artificial intelligence operations. According to one embodiment, the NPU (440) can recognize objects, environments, and / or people in the background when taking a video through a camera and automatically adjust the focus, automatically switch the shooting mode of the camera module (180) to food mode when taking a picture of food, and / or perform processing to erase only unnecessary subjects from the captured results. According to one embodiment, the NPU (440) can perform processing to generate (e.g., regenerate or reconstruct) an image based on given information (e.g., an image and / or a prompt).

[0118] According to one embodiment, the electronic device (101) can support integrated machine learning processing by interacting with all processors such as the GPU (410), the ISP (420), the CPU (430), and the NPU (440).

[0119] In one embodiment, the DSP (450) may represent an integrated circuit that facilitates rapid processing of digital signals. In one embodiment, the DSP (450) may perform the function of converting analog signals into digital signals and performing high-speed processing.

[0120] According to one embodiment, the modem (460) may perform operations that enable the use of various communication functions in the electronic device (101). For example, the modem (460) may support communications such as telephone and data transmission and reception by exchanging signals with a base station. According to one embodiment, the modem (460) may include an integrated modem (e.g., a cellular modem, an LTE modem, a 5G modem, a 5G-Advanced modem, and a 6G modem) that supports communication technologies such as long term evolution (LTE) and 2G to 5G. According to one embodiment, the modem (460) may include an artificial intelligence modem that applies an artificial intelligence algorithm.

[0121] In one embodiment, the connectivity (470) may support wireless data transmission based on IEEE 802.11. In one embodiment, the connectivity (470) may support communication services based on IEEE 802.11 (e.g., Wi-Fi) and / or 802.15 (e.g., Bluetooth, ZigBee, UWB). For example, the connectivity (470) may support communication services targeting an unspecified number of people in a localized area, such as indoors, using an unlicensed band.

[0122] According to one embodiment, security (480) can provide an independent security execution environment between data or services stored in the electronic device (101). According to one embodiment, security (480) can perform an operation to prevent external hacking through software and hardware security during the process of user authentication when providing services such as biometrics, mobile identification, and / or payment of the electronic device (101). For example, security (480) can provide an independent security execution environment in device security for reinforcing the security of the electronic device (101) itself and in security services based on user information such as mobile identification, payment, and car keys in the electronic device (101).

[0123] According to one embodiment, the operations performed by the processor (120) may be implemented by executing instructions stored in a recording medium (or a computer program product or storage medium). For example, the recording medium may include a non-transitory computer-readable recording medium having recorded thereon a program for executing various operations performed by the processor (120).

[0124] The embodiments described in the present disclosure may be implemented in a computer-readable recording medium using software, hardware, or a combination thereof. In a hardware implementation, the operations described in one embodiment may be implemented using at least one of application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, and / or other electrical units for performing functions.

[0125] In one embodiment, a computer-readable recording medium (or computer program product or storage medium) is provided, which records a program for causing an electronic device (101) to perform (or execute) various operations.

[0126] The above operations may include: displaying an image on a display; receiving a first input for selecting a portion of the image corresponding to a face image; displaying, in response to the first input for selecting the portion of the image corresponding to the face image, an interface for editing the face image (e.g., the interface includes a plurality of UI items including emojis); receiving a second input for selecting an emoji included in the interface; identifying, based on the second input for selecting an emoji included in the interface, information related to the selected emoji including an index and / or semantic information related to the emoji; generating, based on the information related to the emoji, a text prompt indicating features of an image to be applied to the face image so as to perform inpainting and / or outpainting on the face image of the selected portion; obtaining, in relation to the text prompt, a result image; and displaying the result image on the display.

[0127] The above operations may include: displaying an image on a display (490); receiving a first input for selecting a portion corresponding to at least one object in the image; selecting a portion corresponding to at least one object in the image based on the first input; displaying an interface on the display (490) that includes at least one emoji related to editing of at least one object in the selected portion of the image; receiving a second input for selecting at least one emoji to be applied to at least one object in the selected portion based on the interface; determining (or identifying) at least one piece of information related to an index, a shape, and / or a feature point (e.g., semantic information) related to the at least one emoji selected based on the second input; generating, based on a third input, a prompt for performing inpainting and / or outpainting of at least one object in the selected portion based on the at least one emoji and the at least one piece of information; obtaining a result image in relation to the prompt; and displaying the result image on the display (490).

[0128] The above operations may include an operation of displaying an execution screen of an application on a display (490), an operation of receiving a first input for selecting an information input section related to image generation on the execution screen, an operation of displaying an interface for inputting graphic objects on the execution screen based on the first input, an operation of receiving a second input for selecting at least one graphic object based on the interface, an operation of selecting at least one graphic object based on the second input, an operation of generating a prompt for performing image generation based on at least one graphic object corresponding to the second input based on a third input for generating a result image, an operation of obtaining a result image in relation to the prompt, and an operation of displaying the result image on the display (490).

[0129] The above operations may include an operation of receiving information input, an operation of detecting execution of image generation, an operation of analyzing (or extracting) information for generating a prompt, an operation of determining whether a graphic object is included, an operation of generating a prompt to execute image generation based on the extracted information if a graphic object is not included, an operation of determining whether a target image is included if a graphic object is included, an operation of determining a graphic object, a target image, and parameter information as prompt inputs if a target image is included, an operation of generating a prompt to inpaint and / or outpaint the target image based on the graphic object, an operation of determining a graphic object and parameter information as prompt inputs if a target image is not included, an operation of generating a prompt to inpaint and / or outpaint based on the graphic object, and an operation of obtaining a result image in relation to the prompt.

[0130] An electronic device (e.g., an electronic device (101, 201) of FIGS. 1 to 3) according to one embodiment of the present disclosure may include a display (e.g., a display module (160) of FIG. 1 or a display (490) of FIG. 3), at least one processor including processing circuitry (e.g., a processor (120, 230) of FIGS. 1 to 3), and a memory (e.g., a memory (130, 240) of FIGS. 1 to 3). In one embodiment, the memory may store instructions that, when individually and / or collectively executed by at least one processor, cause the electronic device (101) to perform operations.

[0131] In one embodiment, the instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to display an image on a display. The instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to receive a first input selecting a portion of the image corresponding to a face image. The instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to display an interface for editing a face image in response to the first input selecting a portion of the image corresponding to a face image. In one embodiment, the interface may include a plurality of UI items including emojis. The instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to receive a second input selecting an emoji included in the interface. The instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to identify, based on a second input selecting an emoji included in the interface, information related to the selected emoji, including an index and / or semantic information related to the emoji. The instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to perform inpainting and / or outpainting on a selected portion of a facial image based on the information related to the emoji, thereby generating a text prompt indicating characteristics of an emoji to be applied to the facial image. The instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to obtain a result image in relation to the text prompt.The above instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to display a resulting image on a display.

[0132] According to one embodiment, the instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to generate the text prompt based on at least one of the image, the facial image, the emoji, information related to the emoji, application information of an application providing the image, parameter information related to the application, and / or additional information input from a user.

[0133] According to one embodiment, the instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to analyze semantic information associated with the emoji and use the semantic information as additional input information in generating the text prompt.

[0134] According to one embodiment, the semantic information may include shape information, type information, facial expression information, emotion information, and / or action information related to the emoji.

[0135] In one embodiment, the resulting image may include an image reconstructed by the inpainting and / or the outpainting performed based on the emoji in relation to the facial image of the image.

[0136] According to one embodiment, the result image may include an image generated such that the facial image of the image corresponds to the emoji.

[0137] In one embodiment, the interface may be provided as a float or pop-up on the display.

[0138] According to one embodiment, the instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to provide the text prompt to an on-device and / or server-side generative artificial intelligence (AI) to execute an image generation process based on the text prompt.

[0139] According to one embodiment, the instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to provide the text prompt and the image of the emoji to the generative AI.

[0140] According to one embodiment, the instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to display an execution screen of an application on the display, receive a first input for inputting at least one emoji based on an information input portion of the execution screen, display the at least one emoji on the information input portion based on the first input, generate a text prompt for performing inpainting and / or outpainting based on the at least one emoji based on a second input for generating a result image, obtain a plurality of result images in relation to the text prompt, display the plurality of result images on the display, receive a third input for selecting one of the plurality of result images, and display the result image selected based on the third input as a user image.

[0141] According to one embodiment, the information input section may include a profile information input field or a schedule information input field.

[0142] According to one embodiment, the instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to receive an input for selecting the information input portion, display an interface including the at least one emoji on the display based on the input, receive an input for selecting the at least one emoji based on the interface, and display the at least one emoji selected based on the input on the information input portion.

[0143] In one embodiment, the instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to display an image on the display. In one embodiment, the instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to receive a first input selecting a portion corresponding to at least one object in the image. In one embodiment, the instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to select a portion corresponding to the at least one object in the image based on the first input. In one embodiment, the instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to display an interface on the display that includes at least one emoji associated with editing the at least one object in the selected portion of the image. In one embodiment, the instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to receive a second input selecting at least one emoji to apply to the at least one object of the selected portion based on the interface. In one embodiment, the instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to determine at least one piece of information related to an index, a form, and / or a feature associated with the at least one emoji selected based on the second input.In one embodiment, the instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to generate a prompt that causes the electronic device to perform inpainting and / or outpainting of the at least one object of the selected portion based on the at least one emoji and the at least one piece of information, based on a third input. In one embodiment, the instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to obtain a result image in relation to the prompt. In one embodiment, the instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to display the result image on the display.

[0144] According to one embodiment, the instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to generate the prompt based on at least one of the image, the selected portion of the image, the at least one emoji, the at least one piece of information related to the at least one emoji, application information of an application providing the image, parameter information related to the application, and / or additional information input from a user.

[0145] According to one embodiment, the instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to analyze semantic information related to the emoji and use the semantic information as additional input information for generating the prompt. According to one embodiment, the semantic information may include facial expression information, emotional information, action information, and / or shape information (e.g., an emoji image) related to the emoji.

[0146] In one embodiment, the resulting image may include an image reconstructed by the inpainting and / or the outpainting performed based on the at least one emoji in relation to the selected portion of the image.

[0147] According to one embodiment, the resulting image may include an image generated such that the at least one object of the selected portion of the image corresponds to the at least one emoji.

[0148] In one embodiment, the interface may be provided as a float or pop-up on the display.

[0149] In one embodiment, the third input may include an input for selecting a generation object that causes image generation to be performed.

[0150] In one embodiment, the instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to provide the prompt to an on-device and / or server-generated artificial intelligence (AI) to execute an image generation process based on the prompt.

[0151] According to one embodiment, the instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to display an execution screen of an application on the display, receive a first input for inputting at least one emoji based on an information input portion of the execution screen, display the at least one emoji on the information input portion based on the first input, generate a prompt for performing inpainting and / or outpainting based on the at least one emoji based on a second input for generating a result image, obtain a plurality of result images in relation to the prompt, display the plurality of result images on the display, receive a third input for selecting one of the plurality of result images, and display the result image selected based on the third input as a user image.

[0152] According to one embodiment, the information input section may include a profile information input field or a schedule information input field.

[0153] According to one embodiment, the instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to receive an input for selecting the information input portion, display an interface including the at least one emoji on the display based on the input, receive an input for selecting the at least one emoji based on the interface, and display the at least one emoji selected based on the input on the information input portion.

[0154] According to one embodiment, the instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to display an execution screen of an application on the display. The instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to receive a first input selecting an information input portion related to image generation on the execution screen. The instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to display an interface for inputting a graphical object on the execution screen based on the first input. The instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to receive a second input selecting at least one graphical object based on the interface. The instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to select the at least one graphical object based on the second input. The instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to generate a prompt that causes the electronic device to perform image generation based on the at least one graphic object corresponding to the second input, based on a third input that causes the electronic device to generate a result image. The instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to obtain the result image in relation to the prompt. The instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to display the result image on the display.

[0155] According to one embodiment, the execution screen may include a first screen related to profile creation, a second screen displaying an image, or a third screen that allows the user to input information.

[0156] According to one embodiment, the first screen and the third screen may include screens that include or do not include images.

[0157] In one embodiment, the second screen may include a screen including the image.

[0158] According to one embodiment, the information input portion may include a profile information input field on the first screen, at least one object in an image on the second screen, or a schedule information input field on the third screen.

[0159] According to one embodiment, the interface may include at least one graphical object.

[0160] According to one embodiment, the at least one graphic object may include at least one graphic object such as an emoji, an emoticon, an icon, a sticker, and an image.

[0161] According to one embodiment, the interface may be provided as a float or pop-up at any location on the execution screen.

[0162] In one embodiment, the third input may include an input for selecting an object designated to command image generation.

[0163] According to one embodiment, the instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to generate a prompt that causes the electronic device to perform inpainting and / or outpainting based on the at least one graphical object, in response to the third input.

[0164] In one embodiment, the instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to provide the prompt to an on-device and / or server-based generative AI to execute an image generation process based on the prompt.

[0165] According to one embodiment, the instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to generate the prompt based on the at least one graphical object, an index associated with the at least one graphical object, a target image displayed via the application, and parameter information associated with the application.

[0166] According to one embodiment, the instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device, in response to the third input, to identify a graphical object input by the user in the input information for the prompt.

[0167] According to one embodiment, the instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to generate the prompt based on the graphical object, based on identifying the graphical object in the input information.

[0168] According to one embodiment, the instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to obtain, based on identifying the graphic object, the graphic object, an index associated with the graphic object, a target image displayed through a running application, and parameter information associated with the running application.

[0169] According to one embodiment, the instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to generate the prompt based on at least one of the graphical object, the index, the target image, and the parameter information.

[0170] According to one embodiment, the instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to analyze semantic information for the prompt based at least on the graphical object, the index, the target image, and the parameter information.

[0171] According to one embodiment, the instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to use the semantic information as additional input information in generating the prompt.

[0172] Hereinafter, an operating method of an electronic device (e.g., the electronic device (101, 201) of FIGS. 1 to 3) (hereinafter, the electronic device (101)) according to various embodiments will be described in detail. Operations performed in the electronic device (101) according to various embodiments may be executed by a processor (e.g., the processor (120, 230) of FIGS. 1 to 3) including various processing circuitry and / or executable program elements of the electronic device (101). According to one embodiment, the operations performed in the electronic device (101) may be stored as instructions in a memory (130) and individually and / or collectively performed (or executed) by the processor (120, 220).

[0173] FIG. 4 is a flowchart illustrating a method of operating an electronic device according to one embodiment of the present disclosure.

[0174] FIG. 4, according to one embodiment, may illustrate an example of a method for generating a prompt (e.g., a text prompt) based on a graphic object (or graphic element) (e.g., an emoji) in an electronic device (101) and supporting image generation through the generated prompt.

[0175] A method for supporting image generation in an electronic device (101) according to one embodiment of the present disclosure may be performed, for example, according to a flowchart illustrated in FIG. 4. The flowchart illustrated in FIG. 4 is an example according to one embodiment of an operation of the electronic device (101), and the order of at least some operations may be changed or performed in parallel, performed as independent operations, or at least some other operations may be performed complementarily to at least some operations. According to one embodiment of the present disclosure, operations 401 to 417 may be performed in at least one processor of the electronic device (101) (e.g., processors 120 and 230 of FIGS. 1 to 3 ).

[0176] As illustrated in FIG. 4, an operation method performed by an electronic device (101) according to an embodiment may include an operation of displaying an execution screen (operation 401), an operation of receiving a first input for selecting an information input portion (operation 403), an operation of displaying an interface for inputting a graphic object based on the first input (operation 405), an operation of receiving a second input for selecting a graphic object based on the interface (operation 407), an operation of selecting a graphic object based on the second input (operation 409), an operation of receiving a third input for generating a result image (operation 411), an operation of generating a prompt (or instruction) for performing image generation based on a graphic object corresponding to the second input based on the third input (operation 413), an operation of obtaining a result image in relation to the prompt (operation 415), and an operation of displaying the result image (operation 417).

[0177] Referring to FIG. 4, in operation 401, the processor (120) of the electronic device (101) may display an execution screen. According to one embodiment, the processor (120) may display an execution screen of an application on the display. According to one embodiment, the processor (120) may receive an input related to execution of an application (e.g., a gallery application, an account application, a messenger application, a contact application, a memo application, and / or a calendar application) from a user. According to one embodiment, the processor (120) may execute the application and display the execution screen of the application on the display in response to the input related to execution of the application.

[0178] In one embodiment, the execution screen may include a first screen related to profile creation (e.g., a profile screen), a second screen displaying images (e.g., a gallery screen), and / or a third screen allowing the user to input information (e.g., a calendar screen). In one embodiment, the first and third screens may include screens that either include or do not include images. In one embodiment, the second screen may include a screen that includes images.

[0179] In operation 403, the processor (120) may receive a first input for selecting an information input portion. According to one embodiment, the processor (120) may receive a first input for selecting an information input portion related to image generation on an execution screen. According to one embodiment, the information input portion may include a profile information input field on the first screen, at least one object (e.g., a face object) in an image on the second screen, and / or a schedule information input field on the third screen. According to one embodiment, the user may select a designated portion for information input (e.g., an information input field, at least one object in an image, a schedule information input field, or a menu for activating an information input window) while the execution screen of the application is displayed. According to one embodiment, the processor (120) may select and display the information input portion based on the first input received from the user on the execution screen of the application.

[0180] For example, a user may select an information input section (e.g., an account input field) in which information related to the profile (e.g., a name, a nickname, and / or an ID) can be input from a launch screen of an application (e.g., an account application, a contact application) that allows the user to create (or enter or edit) a user profile. In one embodiment, the processor (120) may, in response to input detection based on the information input section, activate the information input section to enable user input of information (e.g., activate an input mode based on the account input field).

[0181] For example, a user may select an information input portion (e.g., a target object portion to be edited among objects included in an image) on the execution screen of an application (e.g., a gallery application) capable of displaying (or editing) images. According to one embodiment, the processor (120) may, in response to an input based on the information input portion, activate (e.g., display an information input window) the information input portion (e.g., the target object portion) so that the user can input information.

[0182] In operation 405, the processor (120) may display an interface for inputting a graphic object based on the first input. According to one embodiment, the processor (120) may control the display to display the interface for inputting a graphic object on the execution screen based on the first input.

[0183] According to one embodiment, the interface may include at least one graphical object. According to one embodiment, the interface may be provided as a float or pop-up at any location on the execution screen. According to one embodiment, the graphical object (or graphic element or content) may be used as a prompt source for image generation. In one embodiment, the graphical object may include image-based content. According to one embodiment, the graphical object may include an emoji, an emoticon, an icon, a sticker, and / or an image. According to one embodiment, the graphical object (e.g., an emoji) may include an index for each graphical object (e.g., a Unicode or a graphical object source (or content source)). In one embodiment, the index may include a unique code (e.g., a Unicode) that represents a characteristic of the graphical object.

[0184] In operation 407, the processor (120) may receive a second input for selecting a graphical object based on the interface. In one embodiment, the processor (120) may receive a second input for selecting at least one graphical object based on the interface.

[0185] In operation 409, the processor (120) may select a graphic object based on the second input. According to one embodiment, if the information input portion is in a format that allows information input (e.g., an information input field), the processor (120) may display the selected graphic object in the information input portion and set it as an input (e.g., a prompt source) of a prompt for image generation. According to one embodiment, if the information input portion is in a format that does not allow information input (e.g., a format other than an information input field, e.g., an object within an image), the processor (120) may not display the selected graphic object and set it as an input (e.g., a prompt source) of a prompt for image generation.

[0186] In operation 411, the processor (120) may receive a third input to generate a result image. In one embodiment, the user may select a graphic object and complete information input. In one embodiment, the completion of the information input may be performed by a designated command input. For example, the user may request completion of the information input based at least on selection of a designated object (or creation object) (e.g., a software button provided on an execution screen) for completing information input (or image generation), a designated voice command, and / or a designated motion gesture (e.g., shaking the electronic device (101). In one embodiment, the processor (120) may determine entry into an operation to generate an image based on detecting a designated command input (e.g., the third input) while performing information input based on interaction with the user.

[0187] In operation 413, the processor (120) may generate a prompt (or instruction) to perform image generation based on a graphic object corresponding to the second input based on the third input. In one embodiment, the processor (120) may generate a prompt to perform inpainting and / or outpainting based on the graphic object based on the third input. In one embodiment, the processor (120) may perform an image generation process related to generating a new image (e.g., a user profile image) based on the prompt based on the third input.

[0188] In one embodiment, the image generation process may include an operation to perform inpainting and / or outpainting based on a specified prompt. For example, the processor (120) may generate a prompt (or instruction) to perform inpainting and / or outpainting based on a graphical object. In one embodiment, the processor (120) may provide a prompt to a generative AI to generate an image based on the prompt. In one embodiment, the prompt may be provided to an on-device generative AI and / or a server-side generative AI. In one embodiment, the image generation process may include an operation to generate (e.g., regenerate) an image based on a graphical object using artificial intelligence (AI) (e.g., a generative AI). In one embodiment, the image generation process may be provided on-device and / or server-side.

[0189] In operation 415, the processor (120) may obtain a result image in relation to the prompt (or instruction). In one embodiment, the processor (120) may obtain (or generate) a result image according to an image generation process (e.g., inpainting and / or outpainting based on a graphic object) executed in relation to the prompt (or instruction) in the on-device artificial intelligence. In one embodiment, the processor (120) may obtain (or receive) a result image from a server according to an image generation process (e.g., inpainting and / or outpainting based on a graphic object) executed in relation to the prompt (or instruction) in the server artificial intelligence. In one embodiment, the processor (120) may determine completion of image generation based on the generation and / or acquisition of the result image.

[0190] In operation 417, the processor (120) may display the result image on the display. In one embodiment, the processor (120) may control the display to display the result image obtained in relation to the prompt (or instruction). In one embodiment, the processor (120) may display the obtained result image on the execution screen through the display. In one embodiment, the processor (120) may control the display to display the result image through a designated portion of the execution screen that was previously being displayed (e.g., a portion or the entire portion of the execution screen).

[0191] FIGS. 5 to 10 are drawings illustrating a user interface that supports image generation in an electronic device according to one embodiment and an example of an operation of image generation using the same.

[0192] FIGS. 5 to 10 according to one embodiment may illustrate an example of an operation that supports image generation based on artificial intelligence in an electronic device (101). According to one embodiment, the artificial intelligence may include generative AI. Generative AI may refer to an AI technology that newly creates similar content using existing content such as text, audio, and / or images. For example, generative AI may refer to an AI technology that can generate content (e.g., text, audio, image, and / or video) corresponding to an input based on a given input. According to one embodiment, the electronic device (101) may generate (e.g., regenerate or reconstruct) and provide an image based on generative AI (e.g., on-device AI). According to one embodiment, the electronic device (101) may request generation of an image from a server, and receive and provide an image generated based on the generative AI of the server from the server. According to one embodiment, the electronic device (101) may provide a prompt (or instruction or generative AI prompt) to the generative AI requesting image generation (e.g., a question or instruction to be entered into the generative AI).

[0193] In one embodiment, the image generation may be generated based on a graphic object specified by a user. In one embodiment, the graphic object (or graphic element or content) may be used as a prompt source for image generation. In one embodiment, the graphic object may include image-based content. In one embodiment, the graphic object may include emojis, emoticons, icons, stickers, and / or images. In one embodiment, the graphic object may include an index for each graphic object (e.g., Unicode or a graphic object source (or content source)). In one embodiment, the index may include a unique code (e.g., Unicode) that represents a characteristic of the graphic object.

[0194] According to one embodiment, as illustrated in FIGS. 5 to 10, the method of operation performed by the electronic device (101) according to one embodiment may include an operation of receiving a prompt source (e.g., a graphic object) related to image generation based on interaction with a user, and generating (e.g., regenerating or reconstructing) and providing an image on a server or on-device based on the prompt source.

[0195] FIG. 5 is a diagram illustrating an example of an execution screen (or user interface) of an application in an electronic device according to one embodiment of the present disclosure.

[0196] Referring to FIG. 5, a user may use an electronic device (101) to execute an application (e.g., a contact application) and edit (or modify) a user profile on the application's execution screen. According to one embodiment, in response to receiving an input for editing the user's profile, the electronic device (101) may display a related execution screen (500) (or user interface) on the display.

[0197] In one embodiment, in the example of FIG. 5, the execution screen (500) of the application may include a user profile editing screen. In one embodiment, in the example of FIG. 5, the execution screen (500) may include an image (510) pre-designated by the user (e.g., a profile image). For example, the user may perform an input related to the execution of a contact application. In response to the input related to the execution of the application, the electronic device (101) may execute the application and display the execution screen (500) of the application on the display.

[0198] According to one embodiment, a user may perform an input (525) (e.g., a touch gesture) to select an information input portion (520) on an execution screen (500). For example, the electronic device (101) may receive a first input (e.g., an input (525) based on the information input portion (520)) to select an information input portion (520) on the execution screen (500). According to one embodiment, the user may select (525) an information input portion (520) (e.g., an information input field) for information input while the execution screen (500) of the application is displayed. According to one embodiment, the electronic device (101) may enter a profile editing mode based on the first input received from the user on the execution screen (500) of the application, and may activate and display an interface (e.g., an interface (600) of FIG. 6) related to editing the information input portion (520) on the display.

[0199] For example, a user may select an information input section (520) (e.g., an account input field or a name input window) in which information related to the profile (e.g., a name, nickname, and / or ID) may be input from an execution screen (500) of an application (e.g., a contact application) that allows the user to create (or input or edit) a user profile. In one embodiment, the electronic device (101) may, in response to detecting an input (525) based on the information input section (520), activate the information input section (520) to enable input of information by the user (e.g., activate an input mode based on the account input field). An example of this is illustrated in FIG. 6.

[0200] FIG. 6 is a diagram illustrating an example of an interface that supports input related to prompt generation in an electronic device according to one embodiment of the present disclosure.

[0201] Referring to FIG. 6, the electronic device (101) may control the display to display an interface (600) for inputting a prompt source (e.g., a graphic object) on the execution screen (500) based on the first input (e.g., input (525) based on the information input portion (520)) illustrated through FIG. 5. According to one embodiment, the interface (600) for inputting a prompt source may be an interface that is switched to a state in which input by a user is possible through activation of the information input portion (520) according to the first input. In one embodiment, although the interface (600) is highlighted in FIG. 6 for convenience of explanation, the interface (600) may be a form that is switched to a state in which user input is possible in a form corresponding to the information input portion (520) of FIG. 5.

[0202] According to one embodiment, a user may input first information (e.g., name, nickname, and / or ID) related to the user's profile based on the interface (600), and / or second information (e.g., graphic object). According to one embodiment, FIG. 6 illustrates an example in which a user inputs both first information (610) (e.g., “Pink Kim Da-sol”) and second information (620) (e.g., graphic object (e.g., emoji)), but is not limited thereto. For example, the user may input second information (620) while omitting first information (610), or may input first information (610) while omitting second information (620). In one embodiment, the second information (620) (e.g., graphic object) may include at least one graphic object selected by the user through an object interface that provides the second information. For example, a user may perform a second input to select at least one graphical object based on an object interface (not shown). In one embodiment, the electronic device (101) may receive a second input to select at least one graphical object based on the object interface.

[0203] According to one embodiment, the electronic device (101) may select a graphic object (620) based on the second input and display it in the information input portion (520). According to one embodiment, if the information input portion (520) is in a format that allows information input (e.g., an information input field), the electronic device (101) may display the selected graphic object (620) in the information input portion (520) and set it as an input of a prompt for image generation (e.g., a prompt source).

[0204] FIG. 7 is a diagram illustrating an example of an interface that supports execution of image generation in an electronic device according to one embodiment of the present disclosure.

[0205] Referring to FIG. 7, the electronic device (101) may provide an object (700) (e.g., a creation object (or button)) for executing (or requesting) image generation based on a designated area of ​​the execution screen (500). According to one embodiment, the user may complete information input based on the information input portion (520). According to one embodiment, the user may input information related to the user's profile (e.g., first information and / or second information) based on an interface (600) as illustrated in FIG. 6, and complete the information input operation based on a designated command (e.g., a completion button, a designated motion gesture for completion (e.g., shaking the electronic device (101)), and / or a command based on a designated voice).

[0206] According to one embodiment, the electronic device (101) may complete an information input operation based on detecting a specified command input while performing information input based on the information input portion (520) (e.g., during activation of the information input portion (520)). According to one embodiment, the electronic device (101) may provide an object (700) capable of executing image generation through a specified area (e.g., an image display portion or a menu provision portion) of the execution screen (500) based on the specified command input (or based on completion of the information input operation).

[0207] In one embodiment, a user may request the electronic device (101) to generate an image (e.g., a result image) through an input (e.g., a touch) based on an object (700). In one embodiment, the electronic device (101) may receive a third input to generate a result image (e.g., execute image generation) based on the object (700). In one embodiment, the electronic device (101) may determine to enter into an operation to generate an image based on detecting a specified command input (e.g., a third input) while performing information input based on interaction with the user.

[0208] According to one embodiment, when determining that information input is complete, the electronic device (101) may determine to enter into an operation to generate an image if there is a history of using a graphic object in the information input portion (520), for example, if the information input portion (520) includes a graphic object.

[0209] According to one embodiment, when the electronic device (101) determines to enter an operation to generate an image (e.g., when detecting a third input based on an object (700) and / or recognizing a graphic object in the information input section (520), the electronic device (101) may provide the user with an interface (750) for confirming the image generation. According to one embodiment, the user may execute or cancel the image generation based on the interface (750). For example, when the user wishes to execute the image generation, the user may select (701) an object designated to execute the image generation (e.g., a confirmation object). According to one embodiment, the operation of providing the interface (750) for confirming the image generation may not be performed (or omitted), and the operation related to the image generation may be directly executed.

[0210] According to one embodiment, when the electronic device (101) determines to enter an operation to generate an image, the electronic device (101) may generate a prompt (or instruction) to perform image generation based on information of the information input portion (520) (e.g., the graphic object (620) corresponding to the second input of FIG. 6). According to one embodiment, the electronic device (101) may generate a prompt to perform inpainting and / or outpainting based on the graphic object (620). According to one embodiment, the electronic device (101) may analyze the graphic object (620), obtain information related to the graphic object (620) based on the analysis (e.g., feature points (e.g., semantic information) and / or index of the graphic object (620), and provide the obtained information as an input for a prompt source.

[0211] According to one embodiment, the electronic device (101) may perform an image generation process related to generating a new image (e.g., a user profile image) based on a prompt. According to one embodiment, the image generation process may include an operation to perform inpainting and / or outpainting based on a specified prompt. For example, the electronic device (101) may generate a prompt (or instruction) to perform inpainting and / or outpainting based on a graphical object (620). According to one embodiment, the electronic device (101) may provide a prompt to a generative AI to generate an image based on the prompt. According to one embodiment, the prompt may be provided to an on-device generative AI and / or a server-side generative AI. According to one embodiment, the image generation process may include an operation to generate (e.g., regenerate) an image based on a graphical object using artificial intelligence (AI) (e.g., a generative AI). In one embodiment, the image generation process may be provided on-device and / or server-based.

[0212] FIG. 8 is a diagram illustrating an example of an interface during image generation in an electronic device according to one embodiment of the present disclosure.

[0213] According to one embodiment, as in the example of FIG. 8, the electronic device (101) may provide a related interface (800) to the user while the image generation process is in progress. For example, the related interface (800) may include a guide object (850) that notifies that the image generation process is in progress. According to one embodiment, the related interface (800) may be provided by switching from the execution screen (500) to the related interface (800), or may be provided by dimming the execution screen (500) and providing the guide object (850) on the dimmed screen.

[0214] In one embodiment, the guide object (850) may be provided based on various indicator objects indicating the progress status of the image generation process. In one embodiment, the indicator object may include an indicator (or icon, item, identifier, or object) (e.g., a moving icon (or animated icon) whose status changes) and / or text (e.g., a progress guide phrase). In one embodiment, the text (e.g., the progress guide phrase) may be provided by changing various contents along with changes in the indicator while the process is in progress. For example, the electronic device (101) may change and provide the guide phrase according to changes in the indicator (e.g., the moving icon). For example, the text (e.g., the progress guide phrase) may be provided as “Drawing an image...” In one embodiment, the guide phrase may include various phrases that are predefined or automatically generated based on artificial intelligence.

[0215] FIG. 9 is a diagram illustrating an example of providing a result image in an electronic device according to one embodiment of the present disclosure.

[0216] Referring to FIG. 9, the electronic device (101) may obtain a result image (900) in relation to a prompt (or instruction). According to one embodiment, the electronic device (101) may obtain (or generate) a result image according to an image generation process (e.g., inpainting and / or outpainting based on a graphic object) executed in relation to the prompt (or instruction) in the on-device artificial intelligence. According to one embodiment, the electronic device (101) may obtain (or receive) a result image according to an image generation process (e.g., inpainting and / or outpainting based on a graphic object) executed in relation to the prompt (or instruction) in the server artificial intelligence from the server.

[0217] According to one embodiment, the electronic device (101) may display the result image (900) on the display. According to one embodiment, the electronic device (101) may control the display to display the result image (900) obtained in relation to a prompt (or instruction). According to one embodiment, the electronic device (101) may display the obtained result image on an execution screen through the display. According to one embodiment, the electronic device (101) may control the display to display the result image (900) through a designated portion (e.g., a portion or the entire portion of the execution screen) of a previously displayed execution screen (e.g., the execution screen (500) of FIG. 5). For example, the electronic device (101) may provide the result image (900) on the execution screen through a pop-up.

[0218] According to one embodiment, the result image (900) may be provided with N or more result images (e.g., multiple result images). For example, the electronic device (101) may obtain multiple result images (e.g., a first result image (901), a second result image (903), a third result image (905), and a fourth result image (907)) based on generative artificial intelligence, and may provide a preview (e.g., a thumbnail) corresponding to each of the multiple result images (901, 903, 905, 907). According to one embodiment, when multiple result images (901, 903, 905, 907) are provided, the user may sequentially switch between the multiple result images by sliding the preview (e.g., inputting a flick or swipe gesture). For example, the electronic device (101) may sequentially switch and display multiple result images (901, 903, 905, 907) in response to an input of switching the user's preview.

[0219] FIG. 10 is a diagram illustrating an example of providing a result image in an electronic device according to one embodiment of the present disclosure.

[0220] In one embodiment, a user may select a desired image from among multiple result images (901, 903, 905, 907), as illustrated in FIG. 9. For example, a user may select any one of the displayed result images (901, 903, 905, 907) to use as a new image (1000).

[0221] According to one embodiment, the electronic device (101) may apply (or set) the result image (1000) selected by the user as an image (e.g., a representative image, a profile image, a decoration image) to be used in a running application and display it. For example, as illustrated in FIG. 10, the electronic device (101) may set and provide the result image (1000) as a user's profile image used in a running application (e.g., a contact application). For example, the electronic device (101) may set (or apply) the result image (1000) as a representative image of the application and provide it based on a specified format of various interfaces (e.g., a first interface (1010), a second interface (1020)) provided by the application.

[0222] FIG. 11 is a flowchart illustrating a method of operating an electronic device according to one embodiment of the present disclosure.

[0223] FIG. 11 according to one embodiment may illustrate an example of a method for generating a prompt based on a graphic object (or graphic element) in an electronic device (101) and supporting image generation (e.g., regeneration or reprocessing) through the generated prompt.

[0224] A method for supporting image generation in an electronic device (101) according to one embodiment of the present disclosure may be performed, for example, according to a flowchart illustrated in FIG. 11. The flowchart illustrated in FIG. 11 is an example according to one embodiment of an operation of the electronic device (101), and the order of at least some operations may be changed or performed in parallel, performed as independent operations, or at least some other operations may be performed complementarily to at least some operations. According to one embodiment of the present disclosure, operations 1101 to 1121 may be performed in at least one processor of the electronic device (101) (e.g., processors 120 and 230 of FIGS. 1 to 3 ).

[0225] According to one embodiment, the operations described in FIG. 11 may be heuristically performed in combination with the operations described in FIGS. 4 to 10, for example, or heuristically performed as a replacement for at least some of the operations described and combined with at least some other operations, or heuristically performed as a detailed operation of at least some of the operations described.

[0226] As illustrated in FIG. 11, an operation method performed by an electronic device (101) according to an embodiment includes an operation of receiving an information input (operation 1101), an operation of detecting the execution of image generation (operation 1103), an operation of analyzing (or extracting) information for generating a prompt (operation 1105), an operation of determining whether a graphic object is included (operation 1107), an operation of generating a prompt to execute image generation based on the extracted information if the graphic object is not included (operation 1109), an operation of determining whether a target image is included if the graphic object is included (operation 1111), an operation of determining the graphic object, the target image, and parameter information as prompt inputs if the target image is included (operation 1113), an operation of generating a prompt to inpaint and / or outpaint the target image based on the graphic object (operation 1115), an operation of determining the graphic object and parameter information as prompt inputs if the target image is not included (operation 1117), and an operation of inpainting and / or outpainting based on the graphic object. An action may include generating a prompt to perform outpainting (action 1119), and an action may include obtaining a result image in relation to the prompt (action 1121).

[0227] Referring to FIG. 11, in operation 1101, the processor (120) of the electronic device (101) may receive an information input. According to one embodiment, the processor (120) may receive an information input related to image generation based on an information input portion of the execution screen. In one embodiment, the information input may include inputting at least one piece of information from among text and / or graphic objects.

[0228] In operation 1103, the processor (120) may detect an input that causes image generation to be executed (or requested). In one embodiment, the processor (120) may receive a designated input from a user that causes image generation to be executed (e.g., an input based on a creation object, voice, and / or motion gesture related to the execution of image generation).

[0229] In operation 1105, the processor (120) may analyze (or extract) information for prompt generation. In one embodiment, the processor (120) may extract information related to prompt generation based on a specified input. For example, the processor (120) may analyze a prompt source, such as parameter information related to a running application, information input based on an information input portion (e.g., text and / or graphic objects), and / or a target image (e.g., an original image to be used (or provided) as a prompt input). In one embodiment, the parameter information may be information used (or set) in relation to an image in a running application. For example, the parameter information may include at least one of a photo aspect ratio (e.g., 1:1, 9:16, or 16:9), a background (e.g., pastel, no background, or photo realistic), a camera shot (e.g., close up shot, full body shot, random shot, or extreme wide shot), a resolution (e.g., FHD, 4K, or 8K), a pattern, and / or a style (e.g., sticker, line drawing, collage, or artistic). In one embodiment, the parameter information may vary depending on user preferences used (or set) in the application.

[0230] In operation 1107, the processor (120) may determine whether a graphic object is included. According to one embodiment, the processor (120) may determine, based on information analysis, whether the analyzed information includes a graphic object.

[0231] At operation 1107, if the processor (120) does not include a graphic object (e.g., 'No' at operation 1107), at operation 1109, the processor (120) may generate a prompt to execute image generation based on the analyzed information. In one embodiment, the processor (120) may generate a prompt to generate an image corresponding to the input based on a given input (e.g., an image, user input information, and / or parameter information).

[0232] In operation 1107, if the processor (120) includes a graphic object (e.g., 'yes' in operation 1107), in operation 1111, the processor (120) may determine whether the target image is included. In one embodiment, the processor (120) may determine, based on information analysis, whether the analyzed information includes an original image to be used (or provided) as a prompt input. For example, the processor (120) may determine whether an image is displayed on the execution screen.

[0233] In operation 1111, if the target image is included (e.g., 'yes' in operation 1111), in operation 1113, the processor (120) may determine the graphic object, the target image, and the parameter information as prompt input.

[0234] In operation 1115, the processor (120) may generate a prompt to inpaint and / or outpaint a target image based on a graphical object. In one embodiment, the processor (120) may generate a prompt to generate an image based on at least one combination of at least one graphical object, an index related to at least one graphical object, a target image displayed through an application, parameter information related to the application, and / or semantic information. In one embodiment, the processor (120) may analyze semantic information for the prompt based at least on a graphical object (e.g., an emoji), an index (e.g., Unicode), text (e.g., text mapped to an emoji or Unicode), a target image (e.g., a profile image or a gallery image), and / or the parameter information (e.g., application information). In one embodiment, the processor (120) may use the semantic information as additional input information for generating the prompt.

[0235] In one embodiment, the semantic information may include shape (or form) information (e.g., shape information or image) related to a graphical object (or graphic element) (e.g., an emoji), type information (e.g., information about a person, animal, or object), facial expression information (or expression information) (e.g., information such as wink, sleepy, open-mouthed, grinning, and / or crying), emotion information (e.g., information such as joy, happiness, sadness, depression, and / or anger), and / or action information (e.g., action information such as praying, running, waving, meditating, and / or saluting). According to one embodiment, the processor (120) may determine meaningful semantic information of a graphical object based on object analysis (e.g., emoji analysis) for the graphical object and / or an index (e.g., Unicode) of the graphical object. For example, the processor (120) may determine features (e.g., semantic information) of a graphic object (e.g., an emoji) based on object analysis from the graphic object, and determine (or infer) meaningful information of the graphic object based on the features. In one embodiment, the features may include meaningful features (e.g., emotion, action, and / or shape) that can be extracted from the graphic object (e.g., an emoji).

[0236] In operation 1111, if the target image is not included (e.g., 'No' in operation 1111), in operation 1117, the processor (120) may determine the graphic object and parameter information as prompt input.

[0237] In operation 1119, the processor (120) may generate a prompt (or instruction) to inpaint and / or outpaint based on a graphical object. In one embodiment, the processor (120) may generate a prompt to generate an image corresponding to the input based on a given input (e.g., a graphical object and parameter information).

[0238] In operation 1121, the processor (120) may obtain a result image in relation to a prompt (or instruction). In one embodiment, the processor (120) may obtain (or generate) a result image according to an image generation process (e.g., inpainting and / or outpainting based on a graphic object) executed in relation to the prompt (or instruction) in the on-device artificial intelligence. In one embodiment, the processor (120) may obtain (or receive) a result image from a server according to an image generation process (e.g., inpainting and / or outpainting based on a graphic object) executed in relation to the prompt (or instruction) in the server artificial intelligence. In one embodiment, the processor (120) may determine completion of image generation based on the generation and / or acquisition of the result image.

[0239] FIGS. 12A, 12B, and 12C are diagrams illustrating an example of an operation of image generation in an electronic device according to one embodiment of the present disclosure.

[0240] FIGS. 12A, 12B, and 12C according to one embodiment may illustrate a user interface that supports image generation in an electronic device (101) and an example of an operation of image generation using the same.

[0241] According to one embodiment, FIGS. 12A, 12B, and 12C may perform an image generation operation corresponding to the operation described in the description with reference to FIGS. 5 to 10 described above. For example, the electronic device (101) may generate a prompt (or instruction) to perform image generation (e.g., perform inpainting and / or outpainting based on the graphic object) based on a user input, and obtain and provide a result image in relation to the prompt. According to one embodiment, FIGS. 12A, 12B, and 12C may illustrate an example of generating a prompt based on a graphic object input by a user when there is no target image (e.g., an original image to be used as a prompt input) as information for prompt generation (e.g., a prompt source). For example, FIGS. 5 to 10 may illustrate examples of generating a prompt in which information for generating a prompt (e.g., a prompt source) includes a target image and a graphic object, and a prompt is generated to inpaint and / or outpaint the target image based on the graphic object. For example, FIGS. 12a, 12b, and 12c may illustrate examples of generating a prompt in an operation corresponding to FIGS. 5 to 7, but in which the information for generating the prompt includes a graphic object without a target image, and a prompt is generated to inpaint and / or outpaint based on the graphic object.

[0242] Referring to Figure 12a, an example <1201> , example <1203> , and examples <1205> As illustrated, a user may activate (1220) an information input section (1210) related to information input on an execution screen of an application (e.g., a contact application) to input at least one graphic object (1230). According to one embodiment, the user may complete the information input based on the information input section (1210) and cause the electronic device (101) to execute an image generation process for generating an image (e.g., a result image) based on a designated object (1240) (e.g., a generation object) that executes image generation.

[0243] Referring to Figure 12b, an example <1207> , example <1209> , and examples <1211> As illustrated, the electronic device (101) may provide an interface (1250) for confirming image generation to the user based on receiving an image generation request based on a designated object (1240). According to one embodiment, the user may execute or cancel image generation based on the interface (1250). For example, if the user wishes to execute image generation, the user may select an object (e.g., a confirmation object) designated to execute image generation. According to one embodiment, the operation of providing the interface (1250) for confirming image generation may not be performed (or omitted), and an operation related to image generation may be directly executed.

[0244] According to one embodiment, when the electronic device (101) determines to execute an image generation process, the electronic device (101) may generate a prompt (or instruction) to perform image generation based on information in an information input section (e.g., a graphic object (1230)). According to one embodiment, the electronic device (101) may generate a prompt to perform inpainting and / or outpainting based on the graphic object (1230). According to one embodiment, the electronic device (101) may analyze the graphic object (1230), obtain information related to the graphic object (1230) based on the analysis (e.g., feature points (e.g., semantic information) and / or an index of the graphic object (1230), and provide the obtained information as input for the prompt.

[0245] According to one embodiment, the electronic device (101) may perform inpainting and / or outpainting (e.g., an image generation process) based on a prompt. According to one embodiment, the electronic device (101) may provide a prompt to a generative AI to generate an image based on the prompt. According to one embodiment, the prompt may be provided to the generative AI of the on-device and / or the generative AI of the server. According to one embodiment, the image generation process may include an operation of generating (e.g., regenerating) an image based on a graphic object (1230) using artificial intelligence (AI) (e.g., a generative AI). In one embodiment, the image generation process may be provided based on the on-device and / or the server.

[0246] According to one embodiment, the electronic device (101) may provide a related interface (1260) to the user while the image generation process is in progress. In one embodiment, the related interface (1260) may be provided based on various indicator objects indicating the progress status of the image generation process. In one embodiment, the indicator object may include an indicator (or icon, item, identifier, or object) (e.g., a moving icon (or animated icon) whose state changes) and / or text (e.g., a progress guide phrase).

[0247] According to one embodiment, the electronic device (101) can obtain a result image (1270) in relation to a prompt (or instruction). According to one embodiment, the electronic device (101) can obtain (or generate) a result image (1270) according to an image generation process (e.g., inpainting and / or outpainting based on a graphic object (1230)) executed in relation to a prompt (or instruction) in an on-device artificial intelligence. According to one embodiment, the electronic device (101) can obtain (or receive) a result image (1270) according to an image generation process (e.g., inpainting and / or outpainting based on a graphic object (1230)) executed in relation to a prompt (or instruction) in a server artificial intelligence from a server.

[0248] According to one embodiment, the electronic device (101) can display the result image (1270) on the display. According to one embodiment, the electronic device (101) can display the acquired result image (1270) on the execution screen through the display. According to one embodiment, the electronic device (101) can control the display to display the result image (1270) through a portion or the entire portion of the execution screen. For example, the electronic device (101) can provide the result image (1270) on the execution screen through a pop-up.

[0249] According to one embodiment, the result image (1270) may be provided with N or more result images (1270) (e.g., multiple result images). For example, the electronic device (101) may obtain multiple result images (1270) based on generative artificial intelligence and provide a preview (e.g., thumbnail) corresponding to each of the multiple result images (1270). According to one embodiment, the user may select a desired image from among the multiple result images (1270). For example, the user may select one result image (1280) from the displayed result images (1270) to be used as a new image.

[0250] Referring to Figure 12c, an example <1213> As illustrated, the electronic device (101) can display the result image (1280) selected by the user by applying (or setting) it as an image (e.g., a representative image, a profile image, a decoration image) to be used in the running application. For example, the electronic device (101) can provide the result image (1280) by setting it as a user's profile image used in the running application (e.g., a contact application). For example, the electronic device (101) can set (or apply) the result image (1280) as a representative image of the application and provide it based on a specified format of various interfaces (e.g., a first interface (1285), a second interface (1295)) provided in the application.

[0251] FIGS. 13A, 13B, 13C, and 13D are diagrams illustrating an example of an operation of image generation in an electronic device according to one embodiment of the present disclosure.

[0252] FIGS. 13a, 13b, 13c, and 13d according to one embodiment may illustrate a user interface that supports image generation in an electronic device (101) and an example of an operation of image generation using the same.

[0253] According to one embodiment, in FIGS. 13A, 13B, 13C, and 13D, an image generation operation may be performed in a manner corresponding to the operation described in the description with reference to FIGS. 5 to 10 described above. For example, the electronic device (101) may generate a prompt (or instruction) to perform image generation (e.g., perform inpainting and / or outpainting based on the graphic object) based on a user input graphic object, and obtain and provide a result image in relation to the prompt. According to one embodiment, in FIGS. 13A, 13B, 13C, and 13D, an example of generating a prompt that includes a target image (e.g., an original image to be used as a prompt input) and a graphic object input (or selected) by a user as information for prompt generation (e.g., a prompt source) and generates an image based on the graphic object (e.g., re-generating the target image) may be illustrated. For example, FIGS. 12a, 12b, and 12c illustrate examples of generating a prompt in which information for generating a prompt (e.g., a prompt source) includes a graphic object without a target image, and generates a prompt for inpainting and / or outpainting based on the graphic object. For example, FIGS. 13a, 13b, 13c, and 13d illustrate examples of generating a prompt in which information for generating a prompt includes a target image and a graphic object, and generates a prompt for inpainting and / or outpainting the target image based on the graphic object.

[0254] Referring to Figure 13a, an example <1301> As illustrated, a user may display a running screen of an application (e.g., a gallery application, a camera application, or an image editing application). In one embodiment, the running screen of the application may include an image (1310) (e.g., a target image). For example, the image (1310) may include an image captured by the user using a camera application, an image previously stored in the memory (130) of the electronic device (101), and / or an image downloaded from an external source (e.g., a client server or a content server).

[0255] example <1303> and examples <1305> As illustrated, the electronic device (101) may perform object recognition (e.g., face recognition or person recognition) from the image (1310) based on the display of the image (1310). According to one embodiment, when a specified object (e.g., face or person) is recognized in the image (1310) based on object recognition, the electronic device (101) may activate an image generation function and provide a generation object (1320) capable of executing image generation.

[0256] example <1305> As illustrated, the electronic device (101) may display a recognized result based on an object (e.g., a face portion) recognized in the image (1310). For example, the electronic device (101) may provide a selection object (1315) indicating a recognized state (e.g., a recognition result) based on the surroundings of the object recognized in the image (1310).

[0257] According to one embodiment, the electronic device (101) is an example <1303> and examples <1305> may not perform the action according to the example. For example, the user may <1301> As illustrated in , on an execution screen including an image (1310), the image creation function may be activated by an input that directly selects an object to be edited in the image (1310) (e.g., a tap and hold gesture). In this case, the electronic device (101) may be an example <1303> and examples <1305> The operation of can be omitted and the relevant interface can be provided directly to the user by entering the operation according to Fig. 13b.

[0258] Referring to Figure 13b, an example <1307> and examples <1309> As illustrated, a user may select a portion of an image (1310) to be regenerated (e.g., synthesized with a graphic object). According to one embodiment, the electronic device (101) may receive an input for selecting a portion corresponding to an object in the image (1310). According to one embodiment, the user may select at least one object in the image (1310) through a user input while the image (1310) is displayed. For example, the user may select (or designate) at least one target object (or target image) for image generation in the image (1310). According to one embodiment, the electronic device (101) may select at least one object corresponding to the user input in the image (1310) as a target object (or target image) for image generation.

[0259] According to one embodiment, the electronic device (101) may provide an information input portion (1330) (e.g., an interface or menu supporting graphical object-based information input) on the execution screen based on an input for selecting an object within an image (1310). According to one embodiment, the user may perform information input based on the information input portion (1330). For example, the user may select a graphical object (1340) in the information input portion (1330).

[0260] According to one embodiment, the electronic device (101) may receive an input to generate a result image (e.g., execute image generation) based on a graphic object (1340) and / or a generation object (1320). For example, a user may request the electronic device (101) to execute image (e.g., result image) generation by selecting a graphic object (1340) in an information input portion (1330). For example, a user may request the electronic device (101) to execute image (e.g., result image) generation by input (e.g., touch) based on a generation object (1320) after selecting a graphic object (1340). According to one embodiment, the electronic device (101) may determine to enter an operation to generate an image based on detecting a specified command input while performing information input based on interaction with a user.

[0261] Referring to Figure 13c, an example <1311> As illustrated, when the electronic device (101) determines to enter an operation to generate an image (e.g., upon detecting an input based on a generation object (1320) and / or detecting a selection of a graphic object (1340) in an information input portion (1330), the electronic device (101) may provide the user with an interface (1350) for confirmation of image generation. According to one embodiment, the user may execute or cancel image generation based on the interface (1350). For example, when the user wishes to execute image generation, the user may select an object designated to execute image generation (e.g., a confirmation object). According to one embodiment, the operation of providing the interface (1350) for confirmation of image generation may not be performed (or omitted), and an operation related to image generation may be directly executed.

[0262] example <1311> and examples <1313> As illustrated, the electronic device (101) may support input of additional information (e.g., a user's additional description of the image to be regenerated) related to a prompt input for image generation via an interface (1350) (e.g., an additional information input field). In one embodiment, the user may select the interface (1350) to input additional information (1360) (or additional description) related to image generation (e.g., “Moustache”).

[0263] According to one embodiment, the electronic device (101) may provide a prompt input based at least on the image (1310), the graphic object (1340) of the information input portion (1330), the target object (or target image), the additional information (1360), and parameter information related to the application when deciding to execute the image generation process. For example, the electronic device (101) may generate a prompt (or instruction) to perform image generation (e.g., regeneration) based on the image (1310), the graphic object (1340), the target object, the additional information (1360), and the parameter information when deciding to execute the image generation process. According to one embodiment, the electronic device (101) may generate a prompt to perform inpainting and / or outpainting on the target object of the image (1310) based on the graphic object (1340). According to one embodiment, the electronic device (101) may analyze a graphic object (1340) and additional information (1360), obtain information related to the graphic object (1340) based on the analysis (e.g., features (e.g., semantic information) of the graphic object (1340), and / or an index), and provide the obtained information as input for a prompt.

[0264] According to one embodiment, the electronic device (101) may perform inpainting and / or outpainting (e.g., an image generation process) based on a prompt. According to one embodiment, the electronic device (101) may provide a prompt to a generative AI to generate an image based on the prompt. According to one embodiment, the prompt may be provided to the generative AI of the on-device and / or the generative AI of the server. According to one embodiment, the image generation process may include an operation of generating (e.g., regenerating) an image based on a graphic object (1340) using artificial intelligence (AI) (e.g., a generative AI). In one embodiment, the image generation process may be provided based on the on-device and / or the server.

[0265] According to one embodiment, the electronic device (101) may provide a user with a relevant interface while the image generation process is in progress. In one embodiment, the relevant interface may be provided based on various indicator objects indicating the progress status of the image generation process. In one embodiment, the indicator object may include an indicator (or icon, item, identifier, or object) (e.g., a moving icon (or animated icon) whose state changes) and / or text (e.g., a progress guide phrase).

[0266] Referring to Fig. 13d, an example <1315> As illustrated, the electronic device (101) may obtain a result image (1370) in relation to a prompt (or instruction). According to one embodiment, the electronic device (101) may obtain (or generate) a result image (1370) according to an image generation process (e.g., inpainting and / or outpainting based on a graphic object (1340)) executed in relation to the prompt (or instruction) in the on-device artificial intelligence. According to one embodiment, the electronic device (101) may obtain (or receive) a result image (1370) according to an image generation process (e.g., inpainting and / or outpainting based on a graphic object (1340)) executed in relation to the prompt (or instruction) in the server artificial intelligence from the server.

[0267] According to one embodiment, the electronic device (101) may display the result image (1370) on the display. According to one embodiment, the electronic device (101) may display the acquired result image (1370) on the execution screen through the display. According to one embodiment, the electronic device (101) may control the display to display the result image (1370) through a portion or the entire portion of the execution screen. According to one embodiment, the result image (1370) may be a new image (e.g., a regenerated image) to which a relevant feature (e.g., a beard) of a graphic object (1340) is applied (or synthesized) based on the target object of the image (1310), such as an extended example of element 1380.

[0268] FIG. 14 is a flowchart illustrating a method of operating an electronic device according to one embodiment of the present disclosure.

[0269] FIG. 14, according to one embodiment, may illustrate an example of a method for generating a prompt based on a graphic object (or graphic element) (e.g., an emoji) in an electronic device (101) and supporting image generation (e.g., regeneration or reprocessing) through the generated prompt.

[0270] A method for supporting image generation in an electronic device (101) according to one embodiment of the present disclosure may be performed, for example, according to a flowchart illustrated in FIG. 14. The flowchart illustrated in FIG. 14 is an example according to one embodiment of an operation of the electronic device (101), and the order of at least some operations may be changed or performed in parallel, performed as independent operations, or at least some other operations may be performed complementarily to at least some operations. According to one embodiment of the present disclosure, operations 1401 to 1417 may be performed in at least one processor of the electronic device (101) (e.g., processors 120 and 230 of FIGS. 1 to 3 ).

[0271] According to one embodiment, the operations described in FIG. 14 may be heuristically performed in combination with the operations described in FIGS. 4 to 13d, for example, or heuristically performed as a replacement for at least some of the operations described and combined with at least some other operations, or heuristically performed as a detailed operation of at least some of the operations described.

[0272] As illustrated in FIG. 14, an operation method performed by an electronic device (101) according to an embodiment may include an operation of displaying an image (operation 1401), an operation of receiving a first input for selecting a portion corresponding to at least one object in the image (operation 1403), an operation of selecting a portion corresponding to at least one object in the image based on the first input (operation 1405), an operation of displaying an interface including at least one emoji related to editing of at least one object (operation 1407), an operation of receiving a second input for selecting at least one emoji in the interface (operation 1409), an operation of determining at least one information related to an index, a shape, and / or a feature related to at least one emoji (operation 1411), an operation of generating a prompt to perform inpainting and / or outpainting of at least one object based on the emoji and the information (operation 1413), an operation of obtaining a result image in relation to the prompt (operation 1415), and an operation of displaying the result image (operation 1417).

[0273] Referring to FIG. 14, in operation 1401, the processor (120) of the electronic device (101) may display an image. According to one embodiment, the processor (120) may execute an application and control the display to display an image selected by the user on the execution screen of the application.

[0274] In operation 1403, the processor (120) may receive a first input for selecting a portion corresponding to at least one object in the image. In one embodiment, the user may select at least a portion corresponding to a face image (or face object) in the image, and the processor (120) may receive (e.g., detect) the first input that at least a portion in the image is selected.

[0275] In operation 1405, the processor (120) may select a portion corresponding to at least one object in the image based on a first input. In one embodiment, the processor (120) may identify (or recognize) a portion (e.g., a portion of a face image) corresponding to a face image in the image (e.g., a portion corresponding to the first input) in response to the first input.

[0276] In operation 1407, the processor (120) may display an interface including at least one emoji related to editing at least one object. In one embodiment, the processor (120) may display an interface including at least one emoji related to editing at least one object (e.g., a face image) of a selected portion of an image on a display. In one embodiment, the interface may include a plurality of UI items including emojis. In one embodiment, the processor (120) may provide the interface as a float or pop-up on the display.

[0277] At operation 1409, the processor (120) may receive a second input selecting at least one emoji from the interface. In one embodiment, the user may select an emoji to apply to at least one object (e.g., a face image) in a selected portion of the image based on the interface, and the processor (120) may receive (e.g., detect) a second input selecting an emoji from the interface.

[0278] In operation 1411, the processor (120) may determine at least one piece of information related to an index, a shape, and / or a feature (e.g., semantic information) associated with at least one emoji. According to one embodiment, the processor (120) may identify various pieces of information related to the emoji based on object analysis (or object recognition) and / or text analysis from the emoji. For example, the processor (120) may infer information about the shape (e.g., form) of the emoji, the type of the emoji, the facial expression of the emoji, the emotion of the emoji, and / or the action of the emoji. For example, the processor (120) may determine (or infer) meaningful information related to the emoji based on the image of the emoji, the index, and / or the text mapped to the emoji (or index).

[0279] In operation 1413, the processor (120) may generate a prompt to perform inpainting and / or outpainting of at least one object based on an emoji and information. In one embodiment, the processor (120) may generate a text prompt to perform inpainting and / or outpainting of at least one object (e.g., a face image) of a selected portion of an image based on an emoji and information based on a third input. In one embodiment, the processor (120) may transmit the prompt to a generative artificial intelligence of an on-device and / or a server to execute an image generation process based on the prompt. In one embodiment, the processor (120) may transmit the prompt and an image of the emoji to the generative artificial intelligence.

[0280] In operation 1415, the processor (120) may obtain a result image in relation to the prompt. In one embodiment, the processor (120) may obtain (or generate) a result image according to an image generation process (e.g., inpainting and / or outpainting) executed in relation to the prompt (or instruction) in the on-device artificial intelligence. In one embodiment, the processor (120) may obtain (or receive) a result image according to an image generation process (e.g., inpainting and / or outpainting) executed in relation to the prompt (or instruction) in the server artificial intelligence from the server.

[0281] In operation 1417, the processor (120) may display a result image. In one embodiment, the processor (120) may control the display to display the result image obtained in relation to the prompt (or instruction). In one embodiment, the processor (120) may display the obtained result image on the execution screen through the display. In one embodiment, the processor (120) may control the display to display the result image through a designated portion of the execution screen that was previously being displayed (e.g., a portion or the entire portion of the execution screen).

[0282] FIG. 15 is a diagram illustrating an example of generating an image based on characteristics of a graphic object in an electronic device according to one embodiment of the present disclosure.

[0283] According to one embodiment, a graphic object (or graphic element or content) can be used as a prompt source (or input of a prompt) for image generation. In one embodiment, the graphic object can include image-based content. In one embodiment, the graphic object can include emojis, emoticons, icons, stickers, and / or images. In one embodiment, the graphic object can include an index for each graphic object (e.g., Unicode or a graphic object source (or content source)). In one embodiment, the index can include a unique code (e.g., Unicode) that represents a characteristic of the graphic object.

[0284] Referring to FIG. 15, the electronic device (101) can analyze (or infer) the feature points of the graphic object (1500) when the graphic object (1500) is given as input of a prompt. For example, the electronic device (101) may analyze (or infer) features (e.g., semantic information) such as the shape (or form) of the graphic object (1500), the type of the graphic object (1500) (e.g., information about a person, an animal, or an object), the facial expression information (or expression information) of the graphic object (1500) (e.g., facial expression information such as wink, sleepy face, open mouth, grin, and / or crying), the emotion information of the graphic object (1500) (e.g., emotion information such as joy, happiness, sadness, depression, and / or anger), and / or action information (e.g., action information such as praying, running, shaking, meditating, and / or saluting), based on the unique code of the graphic object (1500).

[0285] According to one embodiment, the electronic device (101) may generate a prompt for image generation based on the feature points of the graphic object (1500). For example, the electronic device (101) may generate a prompt to generate a result image (1505) reflecting the feature points of the graphic object (1500), as illustrated in FIG. 15 .

[0286] In one embodiment, an example <1510> The graphic object (1500) may represent an example of a face with an open mouth. According to one embodiment, the electronic device (101) may generate a prompt to generate a result image (1505) corresponding to a feature of the graphic object (1500) (e.g., a face with an open mouth). For example, the electronic device (101) may include “face with open mouth” as an input of the prompt, and obtain a result image (1505) of a face with an open mouth, in which feature points related to the graphic object (1500) are reflected in the target image. According to one embodiment, the input of the prompt may further include additional information to reflect more accurate features of the graphic object (1500). For example, the electronic device (101) may generate a prompt based on a graphical object (1500) by including additional information related to the characteristics of the graphical object (1500), such as “face with closed eyes, face with both eyes open, or winking face” in addition to “face with open mouth”.

[0287] In one embodiment, an example <1520> The graphic object (1500) may represent an example of an object of a sleepy face (e.g., a sleepy face). According to one embodiment, the electronic device (101) may generate a prompt to generate a result image (1505) corresponding to the features (e.g., a sleepy face) of the graphic object (1500). For example, the electronic device (101) may include “sleepy face” as an input of the prompt, and may obtain a result image (1505) of a sleepy face in which features related to the graphic object (1500) are reflected in the target image. According to one embodiment, the input of the prompt may further include additional information to reflect more accurate features of the graphic object (1500). For example, the electronic device (101) may generate a prompt by further including additional information related to features of the graphic object (1500), such as “a face with closed eyes, a face with an open mouth, a face yawning,” in “sleepy face” based on the graphic object (1500).

[0288] In one embodiment, an example <1530> The graphic object (1500) may represent an example of an object of an angry face (e.g., an angry face). According to one embodiment, the electronic device (101) may generate a prompt to generate a result image (1505) corresponding to a feature (e.g., an angry face) of the graphic object (1500). For example, the electronic device (101) may include “angry face” as an input of the prompt, and may obtain a result image (1505) of an angry face in which feature points related to the graphic object (1500) are reflected in the target image. According to one embodiment, the input of the prompt may further include additional information to reflect more accurate features of the graphic object (1500). For example, the electronic device (101) may generate a prompt by further including additional information, such as “a frowning face or a face with pursed lips” related to the feature of the graphic object (1500), in addition to “angry face” based on the graphic object (1500).

[0289] In one embodiment, an example <1540> The graphic object (1500) may represent an example of an object of a grinning face with smiling eyes. According to one embodiment, the electronic device (101) may generate a prompt to generate a result image (1505) corresponding to the features of the graphic object (1500) (e.g., a grinning face with smiling eyes). For example, the electronic device (101) may include “grinning face with smiling eyes” as an input of the prompt, and obtain a result image (1505) of a grinning face with smiling eyes, in which features related to the graphic object (1500) are reflected in the target image. According to one embodiment, the input of the prompt may further include additional information to reflect more accurate features of the graphic object (1500). For example, the electronic device (101) may generate a prompt based on a graphical object (1500) by including additional information related to the characteristics of the graphical object (1500), such as “grinning face with smiling eyes,” “face with teeth showing, face with wide open eyes, smiling face.”

[0290] In one embodiment, an example <1550> The graphic object (1500) may represent an example of a face with a tongue sticking out and a winking face. According to one embodiment, the electronic device (101) may generate a prompt to generate a result image (1505) corresponding to the feature of the graphic object (1500) (e.g., a face with a tongue sticking out and a winking face). For example, the electronic device (101) may include “winking face with tongue” as an input of the prompt, and obtain a result image (1505) of a face with a tongue sticking out and a winking face, in which feature points related to the graphic object (1500) are reflected in the target image. According to one embodiment, the input of the prompt may further include additional information to reflect more accurate features of the graphic object (1500). For example, the electronic device (101) may generate a prompt based on a graphic object (1500) by including additional information related to the characteristics of the graphic object (1500), such as “winking face with tongue”, “winking with left eye closed, winking with right eye closed, winking with both eyes closed, face with tongue sticking out and turned to the left.”

[0291] In one embodiment, when configuring a prompt for image generation, a user may utilize graphical objects that facilitate the user's communication. In one embodiment, the electronic device (101) may generate a prompt by adding various combinations based on a given graphical object.

[0292] According to one embodiment, a prompt source for inputting a prompt may include a graphic object (e.g., an emoji), an image (e.g., a target image displayed through a running application), an index related to the graphic object, additional input information of the user, and parameter information related to the application. According to one embodiment, when generating a prompt, the electronic device (101) may generate a prompt by combining at least one prompt source among the various prompt sources with a graphic object. For example, the electronic device (101) may generate a prompt for generative artificial intelligence by combining a graphic object (e.g., an emoji), first information (e.g., a profile image, a gallery image), second information (e.g., a Unicode of the graphic object), and third information (e.g., parameter information). For example, the electronic device (101) may generate a prompt for generative artificial intelligence by combining a graphic object, third information (e.g., parameter information), and fourth information (e.g., additional user input information).

[0293] According to one embodiment, different result images may be provided depending on a combination of prompt sources for inputting a prompt. For example, it may be assumed that the prompt sources include a first source (e.g., an image), a second source (e.g., user input text), a third source (e.g., a graphic object), and a fourth source (e.g., parameter information). According to one embodiment, the electronic device (101) may generate a first prompt based on a combination of the first source and the third source, and obtain a first result image in relation to the first prompt. According to one embodiment, the electronic device (101) may generate a second prompt based on a combination of the first source, the second source, and the third source, and obtain a second result image in relation to the second prompt. According to one embodiment, the electronic device (101) may generate a third prompt based on a combination of the first source, the second source, the third source, and the fourth source, and obtain a third result image in relation to the third prompt.

[0294] In one embodiment, the electronic device (101) may obtain a more detailed result image reflecting the user's intent based on the degree of combination of prompt sources (e.g., the amount of input information). For example, the information combined in the prompts for the first and third result images may differ, and the specificity or information expressed in the result images may differ in response to the different information. For example, the third result image may be more specific in its representation of the user's intent and include more relevant information compared to the first result image.

[0295] FIG. 16 is a flowchart illustrating a method of operating an electronic device according to one embodiment of the present disclosure.

[0296] FIG. 16, according to one embodiment, may illustrate an example of a method for generating a text prompt based on an emoji in an electronic device (101) and supporting image generation (e.g., regeneration or reprocessing) through the generated text prompt.

[0297] A method for supporting image generation in an electronic device (101) according to one embodiment of the present disclosure may be performed, for example, according to a flowchart illustrated in FIG. 16. The flowchart illustrated in FIG. 16 is an example according to one embodiment of an operation of the electronic device (101), and the order of at least some operations may be changed, performed in parallel, performed as independent operations, or at least some other operations may be performed complementarily to at least some operations. According to one embodiment of the present disclosure, operations 1601 to 1615 may be performed in at least one processor of the electronic device (101) (e.g., processors 120 and 230 of FIGS. 1 to 3 ).

[0298] According to one embodiment, the operations described in FIG. 16 may be heuristically performed in combination with the operations described in FIGS. 4 to 15, for example, or heuristically performed as a replacement for at least some of the operations described and combined with at least some other operations, or heuristically performed as a detailed operation of at least some of the operations described.

[0299] As illustrated in FIG. 16, an operation method performed by an electronic device (101) according to an embodiment may include an operation of displaying an image on a display (operation 1601), an operation of receiving a first input for selecting a portion corresponding to a face image from the image (operation 1603), an operation of displaying an interface for editing the face image in response to the first input (operation 1605), an operation of receiving a second input for selecting an emoji included in the interface (operation 1607), an operation of identifying information related to the selected emoji based on the second input (operation 1609), an operation of generating a text prompt indicating characteristics of an emoji to be applied to the face image so as to perform inpainting and / or outpainting on the face image of the selected portion based on the information related to the emoji (operation 1611), an operation of obtaining a result image in relation to the text prompt (operation 1613), and an operation of displaying the result image on the display (operation 1615).

[0300] Referring to FIG. 16, in operation 1601, the processor (120) of the electronic device (101) may display an image on the display. According to one embodiment, the processor (120) may execute an application and control the display to display an image selected by the user on the execution screen of the application.

[0301] In operation 1603, the processor (120) may receive a first input for selecting a portion corresponding to a face image from an image. In one embodiment, the user may select at least a portion corresponding to a face image (or face object) from the image. The processor (120) may receive (e.g., detect) the first input that at least a portion from the image is selected.

[0302] In operation 1605, the processor (120) may display an interface for editing a facial image in response to the first input. In one embodiment, the processor (120) may display an interface on the display that includes at least one emoji related to editing a facial image of a selected portion of the image. In one embodiment, the interface may include a plurality of UI items including emojis. In one embodiment, the processor (120) may provide the interface as a float or pop-up on the display.

[0303] At operation 1607, the processor (120) may receive a second input selecting an emoji included in the interface. In one embodiment, the user may select an emoji to apply to a facial image of a selected portion of the image based on the interface. The processor (120) may receive (e.g., detect) the second input selecting an emoji from the interface.

[0304] In operation 1609, the processor (120) may identify information related to the selected emoji based on a second input. In one embodiment, the processor (120) may identify information related to the selected emoji, including an index and / or semantic information related to the emoji, based on the second input of selecting an emoji included in the interface. In one embodiment, the processor (120) may identify various information related to the emoji based on object analysis (or object recognition) and / or text analysis from the emoji. For example, the processor (120) may infer information about the shape (e.g., shape) of the emoji, the type of the emoji, the facial expression of the emoji, the emotion of the emoji, and / or the action of the emoji. For example, the processor (120) may determine (or infer) meaningful information related to the emoji based on the image of the emoji, the index, and / or the text mapped to the emoji (or index).

[0305] According to one embodiment, the semantic information may include shape (or form) information (e.g., shape information or image) related to the emoji, type information (e.g., information about a person, animal, or object), facial expression information (or expression information) (e.g., information such as wink, sleepy, open mouth, grin, and / or crying), emotion information (e.g., information such as joy, happiness, sadness, depression, and / or anger), and / or action information (e.g., action information such as praying, running, waving, meditating, and / or saluting). According to one embodiment, the processor (120) may determine meaningful semantic information of the graphic object based on object analysis (e.g., emoji analysis) for the graphic object and / or an index (e.g., Unicode) of the graphic object. For example, the processor (120) may determine features (e.g., semantic information) of a graphic object (e.g., an emoji) based on object analysis from the graphic object, and determine (or infer) meaningful information of the graphic object based on the features. In one embodiment, the features may include meaningful features (e.g., emotion, action, and / or shape) that can be extracted from the graphic object (e.g., an emoji).

[0306] In operation 1611, the processor (120) may generate a text prompt indicating characteristics of an emoji to be applied to a facial image to perform inpainting and / or outpainting on a selected portion of the facial image based on information related to the emoji. In one embodiment, the processor (120) may generate a text prompt to perform inpainting and / or outpainting on a facial image of a selected portion of the image based on information related to the emoji. In one embodiment, the processor (120) may transmit the text prompt to a generative artificial intelligence of an on-device and / or a server to edit the corresponding facial image in the image based on the text prompt.

[0307] According to one embodiment, the processor (120) may generate a text prompt based on at least one of an image, a facial image, an emoji, information related to an emoji, application information of an application providing the image, parameter information related to the application, and additional information input by a user. According to one embodiment, the processor (120) may also transmit the text prompt and the image of the emoji to the generative artificial intelligence.

[0308] In operation 1613, the processor (120) may obtain a result image in relation to the text prompt. In one embodiment, the processor (120) may obtain (or generate) a result image according to an image generation process (e.g., inpainting and / or outpainting) executed in relation to the prompt (or instruction) in the on-device artificial intelligence. In one embodiment, the processor (120) may obtain (or receive) a result image according to an image generation process (e.g., inpainting and / or outpainting) executed in relation to the prompt (or instruction) in the server artificial intelligence from the server.

[0309] In operation 1615, the processor (120) may display the result image on the display. According to one embodiment, the processor (120) may display the acquired result image on the execution screen through the display. According to one embodiment, the processor (120) may control the display to display the result image through a designated portion of the execution screen that was previously being displayed (e.g., a portion or the entire portion of the execution screen).

[0310] An operation method performed in an electronic device (101) according to one embodiment of the present disclosure may include an operation of displaying an image on a display. The operation method may include an operation of receiving a first input for selecting a portion corresponding to a face image in the image. The operation method may include an operation of displaying an interface for editing the face image in response to the first input for selecting the portion corresponding to the face image in the image. In one embodiment, the interface may include a plurality of UI (user interface) items including emojis. The operation method may include an operation of identifying information related to the selected emoji, the information including an index and / or semantic information related to the emoji, based on a second input for selecting the emoji included in the interface. The method may include generating a text prompt indicating a feature of the emoji to be applied to the facial image to perform inpainting and / or outpainting on the facial image of the selected portion based on information related to the emoji. The method may include obtaining a resulting image in relation to the text prompt. The method may include displaying the resulting image on the display.

[0311] According to one embodiment, the operation of generating the text prompt may include an operation of generating the text prompt based on at least one of the image, the face image, the emoji, information related to the emoji, application information of an application providing the image, parameter information related to the application, and additional information input from a user.

[0312] In one embodiment, the act of generating the text prompt may include the act of analyzing semantic information related to the emoji, and the act of using the semantic information as additional input information for generating the text prompt.

[0313] According to one embodiment, the semantic information may include shape information, type information, facial expression information, emotion information, and / or action information related to the emoji.

[0314] In one embodiment, the resulting image may include an image reconstructed by the inpainting and / or the outpainting performed based on the emoji in relation to the facial image of the image.

[0315] In one embodiment, the resulting image may include an image generated such that the facial image of the image corresponds to the emoji.

[0316] According to one embodiment, the operating method may include an operation of displaying an execution screen of an application on a display, an operation of receiving a first input for inputting at least one emoji based on an information input portion of the execution screen, an operation of displaying the at least one emoji in the information input portion based on the first input, an operation of generating a text prompt for performing inpainting and / or outpainting based on the at least one emoji based on a second input for generating a result image, an operation of obtaining a plurality of result images in relation to the text prompt, an operation of displaying the plurality of result images on the display, an operation of receiving a third input for selecting one of the plurality of result images, and an operation of setting and displaying the result image selected based on the third input as a user image.

[0317] According to one embodiment, the information input section may include a profile information input field or a schedule information input field.

[0318] According to one embodiment, the operating method may include: receiving an input for selecting the information input portion; displaying an interface including the at least one emoji on the display based on the input; receiving an input for selecting the at least one emoji based on the interface; and displaying the selected at least one emoji on the information input portion based on the input.

[0319] An operation method performed in an electronic device (101) according to an embodiment of the present disclosure may include an operation of displaying an image on the display. The operation method may include an operation of receiving a first input for selecting a portion corresponding to at least one object in the image. The operation method may include an operation of selecting a portion corresponding to the at least one object in the image based on the first input. The operation method may include an operation of displaying an interface on the display including at least one emoji related to editing of the at least one object in the selected portion in the image. The operation method may include an operation of receiving a second input for selecting at least one emoji to be applied to the at least one object in the selected portion based on the interface. The operation method may include an operation of determining at least one piece of information related to an index, a form, and / or a feature related to the at least one emoji selected based on the second input. The method may include generating a prompt based on a third input to perform inpainting and / or outpainting of at least one object of the selected portion based on the at least one emoji and the at least one piece of information. The method may include obtaining a result image in relation to the prompt. The method may include displaying the result image on the display.

[0320] In one embodiment, the action of generating the prompt may include an action of generating the prompt based on at least one of the image, the selected portion of the image, the at least one emoji, the at least one piece of information related to the at least one emoji, application information of an application providing the image, parameter information related to the application, and additional information input from a user.

[0321] In one embodiment, the act of generating the prompt may include an act of analyzing semantic information related to the emoji, and an act of using the semantic information as additional input information for generating the prompt.

[0322] According to one embodiment, the semantic information may include emotional information, action information, and / or shape information related to the emoji.

[0323] In one embodiment, the resulting image may include an image reconstructed by the inpainting and / or the outpainting performed based on the at least one emoji in relation to the selected portion of the image.

[0324] According to one embodiment, the resulting image may include an image generated such that the at least one object of the selected portion of the image corresponds to the at least one emoji.

[0325] According to one embodiment, the operating method may include an operation of displaying an execution screen of an application on a display, an operation of receiving a first input for inputting at least one emoji based on an information input portion of the execution screen, an operation of displaying the at least one emoji on the information input portion based on the first input, an operation of generating a prompt for performing inpainting and / or outpainting based on the at least one emoji based on a second input for generating a result image, an operation of obtaining a plurality of result images in relation to the prompt, an operation of displaying the plurality of result images on the display, an operation of receiving a third input for selecting one of the plurality of result images, and an operation of setting and displaying the result image selected based on the third input as a user image.

[0326] According to one embodiment, the information input section may include a profile information input field or a schedule information input field.

[0327] According to one embodiment, the operating method may include: receiving an input for selecting the information input portion; displaying an interface including the at least one emoji on the display based on the input; receiving an input for selecting the at least one emoji based on the interface; and displaying the selected at least one emoji on the information input portion based on the input.

[0328] An operating method performed in an electronic device (101) according to one embodiment of the present disclosure may include an operation of displaying an execution screen of an application on a display. According to one embodiment, the operating method may include an operation of receiving a first input for selecting an information input portion related to image generation on the execution screen. According to one embodiment, the operating method may include an operation of displaying an interface for inputting a graphic object on the execution screen based on the first input. According to one embodiment, the operating method may include an operation of receiving a second input for selecting at least one graphic object based on the interface. According to one embodiment, the operating method may include an operation of selecting the at least one graphic object based on the second input. According to one embodiment, the operating method may include an operation of generating a prompt for performing image generation based on the at least one graphic object corresponding to the second input based on a third input for generating a result image. According to one embodiment, the operating method may include an operation of acquiring a result image in relation to the prompt. According to one embodiment, the method may include an operation of displaying the result image on the display.

[0329] According to one embodiment, the execution screen may include a first screen related to profile creation, a second screen displaying an image, or a third screen that allows the user to input information.

[0330] According to one embodiment, the first screen and the third screen may include screens that include or do not include images.

[0331] In one embodiment, the second screen may include a screen including the image.

[0332] According to one embodiment, the information input portion may include a profile information input field on the first screen, at least one object in an image on the second screen, or a schedule information input field on the third screen.

[0333] According to one embodiment, the interface may be provided as a float or pop-up at any location on the execution screen and may include at least one graphic object.

[0334] According to one embodiment, the at least one graphic object may include at least one graphic object such as an emoji, an emoticon, an icon, a sticker, and an image.

[0335] In one embodiment, the act of generating the prompt may include generating a prompt to perform inpainting and / or outpainting based on the at least one graphical object, in response to the third input.

[0336] In one embodiment, the act of generating the prompt may include providing the prompt to a generative AI on-device and / or on a server to execute an image generation process based on the prompt.

[0337] In one embodiment, the act of generating the prompt may include, in response to the third input, identifying a graphical object input by the user in the input information for the prompt.

[0338] In one embodiment, the act of generating the prompt may include an act of obtaining, based on identifying the graphic object in the input information, the graphic object, an index related to the graphic object, a target image displayed through a running application, and parameter information related to the running application.

[0339] In one embodiment, the act of generating the prompt may include an act of generating the prompt based on at least one of the graphic object, the index, the target image, and the parameter information.

[0340] According to one embodiment, the method may include analyzing semantic information for the prompt based at least on the graphic object, the index, the target image, and the parameter information.

[0341] According to one embodiment, the method may include an operation of using the semantic information as additional input information for generating the prompt.

[0342] A non-transitory computer-readable medium storing instructions that, when individually and / or collectively executed by a processor (120) of an electronic device (101) according to one embodiment of the present disclosure, cause the processor (120) to perform operations, wherein the instructions, when individually and / or collectively executed by the processor, cause the electronic device to display an image on a display, receive a first input for selecting a portion corresponding to a face image in the image, display an interface for editing the face image, the interface including a plurality of UI (user interface) items including emojis, and identify information related to the selected emoji including an index and / or semantic information related to the emoji based on a second input for selecting the emoji included in the interface. The recording medium may include an operation for generating a text prompt indicating a feature of the image to be applied to the facial image based on information related to the emoji, to perform inpainting and / or outpainting on the facial image of the selected portion, an operation for obtaining a resulting image in relation to the text prompt, and an operation for displaying the resulting image on the display.

[0343] According to one embodiment, the instructions, when individually and / or collectively executed by the processor, cause the electronic device to perform the following actions: displaying an image on a display; receiving a first input for selecting a portion corresponding to at least one object in the image; selecting a portion corresponding to the at least one object in the image based on the first input; displaying an interface on the display including at least one emoji related to editing of the at least one object in the selected portion of the image; receiving a second input for selecting at least one emoji to be applied to the at least one object in the selected portion based on the interface; determining at least one information related to an index, a form, and / or a feature associated with the at least one emoji selected based on the second input; generating a prompt for performing inpainting and / or outpainting of the at least one object in the selected portion based on the at least one emoji and the at least one information based on a third input; A recording medium may be included that causes the user to perform an action of obtaining a result image in relation to a prompt, and an action of displaying the result image on the display.

[0344] According to one embodiment, the instructions may include a recording medium that, when individually and / or collectively executed by the processor, causes the electronic device to perform the following operations: displaying an execution screen of an application on a display; receiving a first input for selecting an information input portion related to image generation on the execution screen; displaying an interface for inputting a graphic object on the execution screen based on the first input; receiving a second input for selecting at least one graphic object based on the interface; selecting the at least one graphic object based on the second input; receiving a third input for generating a result image; generating a prompt for performing image generation based on the at least one graphic object corresponding to the second input based on the third input; acquiring a result image in relation to the prompt; and displaying the result image on the display.

[0345] It will be appreciated that the above-described embodiments and their technical features may be combined with each other in any and all combinations, as long as there is no potential conflict between the two embodiments or features. For example, any and all combinations of two or more of the above-described embodiments may be envisioned and incorporated within the present disclosure. One or more features from any embodiment may be incorporated into any other embodiment, providing a corresponding advantage or advantages.

[0346] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0347] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0348] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0349] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more commands stored in a storage medium (or recording medium) (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one command among the one or more commands stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one command called. The one or more commands may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g. electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0350] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as a computer program product. The computer program product may be traded between sellers and buyers as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or may be provided through an application store (e.g., Play Store). TM ) or directly between two user devices (e.g., smart phones), online distribution (e.g., downloading or uploading). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium (or recording medium), such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0351] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and arranged in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0352] The various embodiments of the present disclosure disclosed in this specification and drawings are intended to provide specific examples to facilitate easy explanation of the technical content of the present disclosure and to aid understanding of the present disclosure, and are not intended to limit the scope of the present disclosure. Therefore, the scope of the present disclosure should be interpreted to include all modifications or variations derived based on the technical concepts of the present disclosure, in addition to the embodiments disclosed herein.

Claims

1. In the electronic device (101, 201), display(160, 490); At least one processor (120, 230) comprising processing circuitry; and Includes a memory (130, 240) for storing instructions, The above instructions, when individually and / or collectively executed by the at least one processor (120, 230), cause the electronic device (101, 201) to: Display the image on the above display (160, 490), Receiving a first input for selecting a portion corresponding to a face image in the above image, In response to the first input selecting the portion corresponding to the face image in the image, displaying an interface for editing the face image, the interface including a plurality of UI (user interface) items including emojis, Based on a second input selecting the emoji included in the interface, identifying information related to the selected emoji, including at least one of an index and semantic information related to the emoji, Based on information related to the emoji, generate a text prompt indicating a feature of the emoji to be applied to the face image to perform at least one of inpainting and outpainting on the face image of the selected portion, Obtaining a resulting image in relation to the above text prompt, and An electronic device that displays the above result image on the display (160, 490).

2. In paragraph 1, The above instructions, when individually and / or collectively executed by the at least one processor (120, 230), cause the electronic device (101, 201) to: An electronic device that generates the text prompt based on at least one of the image, the face image, the emoji, information related to the emoji, application information of an application providing the image, parameter information related to the application, and additional information input from a user.

3. In paragraph 1, The above instructions, when individually and / or collectively executed by the at least one processor (120, 230), cause the electronic device (101, 201) to: Analyze the semantic information related to the above emoji, and To use the above semantic information as additional input information in the generation of the above text prompt, An electronic device wherein the semantic information includes at least one of shape information, type information, facial expression information, emotional information, and behavioral information related to the emoji.

4. In paragraph 1, The above result image is, An electronic device comprising at least one of an image reconstructed by at least one of the inpainting and the outpainting performed based on the emoji in relation to the facial image of the image, or an image generated so that the facial image of the image corresponds to the emoji.

5. In paragraph 1, An electronic device characterized in that the above interface is provided as a float or pop-up on the display (160, 490).

6. In paragraph 1, The above instructions, when individually and / or collectively executed by the at least one processor (120, 230), cause the electronic device (101, 201) to: An electronic device that provides the text prompt to an on-device and / or server generative artificial intelligence (AI) to execute an image generation process based on the text prompt.

7. In paragraph 1, The above instructions, when individually and / or collectively executed by the at least one processor (120, 230), cause the electronic device (101, 201) to: An electronic device that provides the above text prompt and the image of the above emoji to a generative AI.

8. In paragraph 1, The above instructions, when individually and / or collectively executed by the at least one processor (120, 230), cause the electronic device (101, 201) to: The application's execution screen is displayed on the display (160, 490), Based on the information input section of the above execution screen, a first input for inputting at least one emoji is received, Based on the first input, displaying at least one emoji in the information input portion; Generate a text prompt to perform at least one of inpainting and outpainting based on the at least one emoji, based on a second input that causes the resulting image to be generated; Obtaining multiple result images in relation to the above text prompt, Display the above multiple result images on the display (160, 490), Receiving a third input for selecting one of the plurality of result images, and Display the selected result image based on the third input as a user image, An electronic device characterized in that the above information input section includes a profile information input field or a schedule information input field.

9. In paragraph 8, The above instructions, when individually and / or collectively executed by the at least one processor (120, 230), cause the electronic device (101, 201) to: Receive input for selecting the above information input section, Based on the above input, display an interface including at least one emoji on the display (160, 490), Based on the above interface, receiving an input for selecting at least one emoji, and An electronic device that displays at least one emoji selected based on the input in the information input section.

10. In the operating method of an electronic device (101, 201), The action of displaying an image on the display (160, 490); An action of receiving a first input for selecting a portion corresponding to a face image in the above image; In response to the first input selecting the portion corresponding to the face image in the image, an operation of displaying an interface for editing the face image, the interface including a plurality of UI (user interface) items including emojis; An operation of identifying information related to the selected emoji, including at least one of an index and semantic information related to the emoji, based on a second input selecting the emoji included in the interface; An action of generating a text prompt indicating a feature of the image to be applied to the face image (a feature of the emoji) to perform at least one of inpainting and outpainting on the face image of the selected portion based on information related to the emoji; An action of obtaining a resulting image in relation to the above text prompt; and A method comprising the action of displaying the above result image on the display (160, 490).

11. In paragraph 10, The action that generates the above text prompt is: A method comprising an action of generating the text prompt based on at least one of the image, the face image, the emoji, information related to the emoji, application information of an application providing the image, parameter information related to the application, and additional information input from a user.

12. In paragraph 11, The action that generates the above text prompt is: An action of analyzing semantic information related to the above emoji; and Including an operation of using the above semantic information as additional input information in generating the above text prompt, The above semantic information includes at least one of shape information, type information, facial expression information, emotion information, and action information related to the emoji, The above result image is an image reconstructed by at least one of the inpainting and the outpainting performed based on the emoji in relation to the face image of the image, or A method comprising: generating at least one image of the face image of the image to correspond to the emoji.

13. In paragraph 10, The action of displaying the application's execution screen on the display (160, 490). An action of receiving a first input for entering at least one emoji based on an information input portion of the above execution screen; An action of displaying at least one emoji in the information input portion based on the first input; An action of generating a text prompt that causes at least one of inpainting and outpainting to be performed based on said at least one emoji, based on a second input that causes the resulting image to be generated; An action to obtain multiple result images in relation to the above text prompt; An operation of displaying the plurality of result images on the display (160, 490); An operation of receiving a third input for selecting any one of the plurality of result images; and Including an action of displaying the selected result image based on the third input as a user image, A method characterized in that the above information input section includes a profile information input field or a schedule information input field.

14. In paragraph 13, An action of receiving an input for selecting the above information input section; Based on the input, an operation of displaying an interface including at least one emoji on the display (160, 490); An operation of receiving an input for selecting at least one emoji based on the above interface; and A method further comprising an action of displaying at least one selected emoji in the information input portion based on the input.

15. A non-transitory computer-readable medium storing instructions that, when individually and / or collectively executed by at least one processor (120, 230) of an electronic device (101, 201), cause the at least one processor (120, 230) to perform operations, The above instructions, when individually and / or collectively executed by the at least one processor (120, 230), cause the electronic device (101, 201) to: The action of displaying an image on the display (160, 490); An action of receiving a first input for selecting a portion corresponding to a face image in the above image; In response to the first input selecting the portion corresponding to the face image in the image, an operation of displaying an interface for editing the face image, the interface including a plurality of UI (user interface) items including emojis; An operation of identifying information related to the selected emoji, including at least one of an index and semantic information related to the emoji, based on a second input selecting the emoji included in the interface; An action of generating a text prompt indicating a feature of the image to be applied to the face image (a feature of the emoji) to perform at least one of inpainting and outpainting on the face image of the selected portion based on information related to the emoji; An action of obtaining a resulting image in relation to the above text prompt; and A recording medium that performs an operation of displaying the above result image on the display (160, 490).

Images