Electronic device for generating image and control method thereof

Abstract

An electronic device is disclosed. The electronic device of the present disclosure comprises: a memory that stores one or more computer programs; a display; and one or more processors communicatively coupled to the memory and the display, wherein the one or more computer programs may store computer-executable instructions that, when executed individually or collectively by the one or more processors, cause the electronic device to: display an execution screen of a message application; extract at least one keyword text on the basis of conversation information having a history of being displayed through the execution screen; on the basis of receiving a user input for generating an image that can be displayed on the execution screen, identify an image corresponding to the at least one keyword text from among a plurality of images stored in the memory; obtain at least one first layer on the basis of the identified image; obtain at least one second layer on the basis of at least one of the conversation information, the at least one keyword text, or information related to the identified image; and generate an image by overlapping the at least one first layer and the at least one second layer.

Description

Electronic device for generating images and method for controlling the same

[0001] The present disclosure relates to an electronic device for generating an image and a method for controlling the same.

[0002] The variety of services and additional features offered through electronic devices, such as smartphones, is steadily increasing. To enhance the utility of these devices and satisfy the diverse needs of users, telecommunications service providers and electronic device manufacturers are competitively developing electronic devices to offer a variety of features and differentiate themselves from competitors. Consequently, the various functions offered through electronic devices are also becoming increasingly sophisticated.

[0003] Additionally, electronic devices provide various GUIs (graphical user interfaces) for user interaction through displays.

[0004] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0005] According to one embodiment, an electronic device may include a memory storing one or more computer programs, a display, and one or more processors communicatively connected to the memory and the display.

[0006] In one embodiment, the one or more computer programs may store computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to perform the following operations:

[0007] According to one embodiment, the one or more computer programs, when individually or collectively executed by the one or more processors, may cause the electronic device to display an execution screen of a message application.

[0008] According to one embodiment, the one or more computer programs, when individually or collectively executed by the one or more processors, may cause the electronic device to extract at least one keyword text based on conversation information with a history displayed through the execution screen.

[0009] According to one embodiment, the one or more computer programs, when individually or collectively executed by the one or more processors, may cause the electronic device to identify an image corresponding to the at least one keyword text among a plurality of images stored in the memory based on a user input received for generating an image displayable on the execution screen.

[0010] In one embodiment, the one or more computer programs, when individually or collectively executed by the one or more processors, may cause the electronic device to obtain at least one first layer based on the identified image.

[0011] In one embodiment, the one or more computer programs, when individually or collectively executed by the one or more processors, may cause the electronic device to obtain at least one second layer based on at least one of the conversation information, the at least one keyword text, or information related to the identified image.

[0012] In one embodiment, the one or more computer programs, when individually or collectively executed by the one or more processors, may cause the electronic device to generate the image by superimposing the at least one first layer and the at least one second layer.

[0013] According to one embodiment, a method of controlling an electronic device may include an action of displaying an execution screen of a message application.

[0014] According to one embodiment, a method for controlling an electronic device may include an operation of extracting at least one keyword text based on conversation information having a history displayed through the execution screen.

[0015] According to one embodiment, a method for controlling an electronic device may include an operation of identifying an image corresponding to at least one keyword text among a plurality of images stored in a memory of the electronic device based on receiving a user input for generating an image displayable on the execution screen.

[0016] According to one embodiment, a method of controlling an electronic device may include an operation of obtaining at least one first layer based on the identified image.

[0017] According to one embodiment, a method for controlling an electronic device may include an operation of obtaining at least one second layer based on at least one of the conversation information, the at least one keyword text, or information related to the identified image.

[0018] According to one embodiment, a method of controlling an electronic device may include an operation of generating the image by overlapping the at least one first layer and the at least one second layer.

[0019] According to one embodiment, a non-transitory computer-readable recording medium storing one or more programs may include instructions that cause an electronic device to display an execution screen of a message application.

[0020] According to one embodiment, the one or more programs may include instructions that cause the electronic device to extract at least one keyword text based on conversation information displayed through the execution screen.

[0021] According to one embodiment, the one or more programs may include instructions that cause the electronic device to identify an image corresponding to the at least one keyword text among a plurality of images stored in the memory based on receiving a user input for generating an image displayable on the execution screen.

[0022] In one embodiment, the one or more programs may include instructions that cause the electronic device to obtain at least one first layer based on the identified image.

[0023] According to one embodiment, the one or more programs may include instructions that cause the electronic device to obtain at least one second layer based on at least one of the conversation information, the at least one keyword text, or information related to the identified image.

[0024] In one embodiment, the one or more programs may include instructions that cause the electronic device to generate the image by overlapping the at least one first layer and the at least one second layer.

[0025] FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment.

[0026] FIG. 2 is a flowchart illustrating an image generation operation of an electronic device according to one embodiment.

[0027] FIG. 3A is a diagram illustrating an operation of generating an image based on conversation information in an electronic device according to one embodiment.

[0028] FIG. 3b is a diagram for explaining an operation of generating an image based on conversation information in an electronic device according to one embodiment.

[0029] FIG. 4 is a flowchart illustrating an operation of generating an image by creating a set number of layers based on conversation information in an electronic device according to one embodiment.

[0030] FIG. 5 is a flowchart illustrating an operation of determining the number of layers based on conversation information and generating an image by creating the determined number of layers in an electronic device according to one embodiment.

[0031] FIG. 6 is a diagram for explaining an operation of generating an image based on generative AI in an electronic device according to one embodiment.

[0032] FIG. 7 is a flowchart illustrating a system for generating an image based on conversation information, according to one embodiment.

[0033] FIG. 8 is a diagram for explaining an operation of generating an image using multiple layers generated based on conversation information in an electronic device according to one embodiment.

[0034] FIG. 9 is a flowchart illustrating an operation of generating multiple layers based on conversation information in an electronic device according to one embodiment.

[0035] FIG. 10 is a diagram for explaining an operation of generating multiple layers based on conversation information in an electronic device according to one embodiment.

[0036] FIG. 11 is a diagram for explaining an operation of generating multiple layers based on conversation information in an electronic device according to one embodiment.

[0037] FIG. 12 is a diagram for explaining an operation of generating multiple layers based on conversation information in an electronic device according to one embodiment.

[0038] FIG. 13A is a diagram for explaining an operation of generating a sub-layer based on color information of an image extracted based on conversation information in an electronic device according to one embodiment.

[0039] FIG. 13b is a diagram for explaining an operation of placing objects in a sub-layer based on objects in an image extracted based on conversation information in an electronic device according to one embodiment.

[0040] FIG. 14 is a drawing for explaining an operation of displaying a generated image in full screen of an electronic device according to one embodiment.

[0041] FIG. 15 is a drawing for explaining an operation of displaying a generated image in full screen in an electronic device according to one embodiment.

[0042] FIG. 16 is a drawing for explaining an operation of modifying a generated image of an electronic device according to one embodiment.

[0043] FIG. 17 is a drawing for explaining an operation of generating an image based on an image selected by a user input in an electronic device according to one embodiment.

[0044] FIG. 18 is a drawing for explaining an operation of modifying a generated image based on an image selected by a user input of an electronic device according to one embodiment.

[0045] FIG. 19A is a diagram for explaining an operation of generating an image based on an image captured or stored through a camera application of an electronic device according to one embodiment.

[0046] FIG. 19b is a diagram for explaining an operation of generating an image based on an image captured or stored through a camera application of an electronic device according to one embodiment.

[0047] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to one embodiment of the present disclosure. Referring to FIG. 1 , in the network environment (100), the electronic device (101) may communicate with the electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (104) or the server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0048] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0049] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0050] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0051] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0052] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0053] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0054] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0055] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0056] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0057] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0058] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0059] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0060] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0061] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0062] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0063] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0064] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0065] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0066] In one embodiment, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0067] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0068] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0069] FIG. 2 is a flowchart illustrating an image generation operation of an electronic device according to one embodiment.

[0070] Referring to FIG. 2, in operation 210, an electronic device (e.g., the electronic device (101) of FIG. 1 or the processor (120) of FIG. 1) may display an execution screen of a message application.

[0071] In one embodiment, the messaging application may be already stored in the memory of the electronic device when the electronic device is shipped from the factory, or may be downloaded and installed by user input.

[0072] According to one embodiment, the electronic device may display a launch screen of a messaging application based on a user input (e.g., a touch) to launch the messaging application. According to one embodiment, the launch screen of the messaging application may include a chat window containing a conversation with a specific party.

[0073] According to one embodiment, in operation 220, the electronic device may extract at least one keyword text based on conversation information with a history displayed through the execution screen. According to one embodiment, the electronic device may extract at least one keyword text not only from conversation information currently displayed on a display (e.g., display module (160) of FIG. 1), but also from conversation information that was previously displayed but is no longer displayed on the display as new conversation information is input. For example, the electronic device may extract at least one keyword text from conversation information from a set time (e.g., 30 minutes, 1 hour) ago to the present.

[0074] In one embodiment, the electronic device may extract keyword texts from various categories. For example, the categories may include at least one of events (e.g., travel, birthday, vacation, business trip, golf), people (e.g., person names), dates / times (e.g., weekdays, holidays), emotions (e.g., joy, anger, sorrow, pleasure), places (e.g., LA, Hawaii), text related to images used by the user, or text related to objects (e.g., cake, sunglasses).

[0075] In one embodiment, the electronic device can extract the context of a conversation from a combination of texts by category.

[0076] In one embodiment, in operation 230, the electronic device can identify an image corresponding to at least one keyword text among a plurality of images stored in a memory based on receiving a user input for generating an image displayable on an execution screen.

[0077] According to one embodiment, images that can be displayed on the execution screen of the message application may include sticker images, emoticons, emojis, still images, and moving images.

[0078] According to one embodiment, when an electronic device selects a text window for entering text on a running screen, the electronic device may display a soft keyboard for entering text. According to one embodiment, a region of the soft keyboard may include a UI for generating an image that can be displayed on the running screen. According to one embodiment, when a user input for selecting the UI for generating an image is received, the electronic device may identify an image corresponding to at least one keyword text among a plurality of images stored in the memory.

[0079] For example, an electronic device may display the UI for generating an image in an activated state if the extracted keyword text can generate an image. If the extracted keyword text has not yet been sufficient to generate an image, the electronic device may display the UI for generating an image in an inactive state.

[0080] In one embodiment, an electronic device may extract a relevant image from among a plurality of images stored in memory based on at least one keyword text. For example, the electronic device may extract one image from among the plurality of images by comparing at least one keyword with information about people, objects, animals, location information, date information, and / or tag information contained in each image stored in the plurality of images.

[0081] According to one embodiment, the electronic device can identify an image selected by a user input from among a plurality of images stored in the memory as an image corresponding to at least one keyword text. For example, when a UI for generating an image is selected, the electronic device can display a UI for selecting a photo. According to one embodiment, when the UI for selecting a photo is selected, the electronic device can display a plurality of images stored in the memory and identify an image selected by a user input from among the plurality of images as an image corresponding to at least one keyword text (or an image to be used for image generation). According to one embodiment, an operation of selecting an image to be used for image generation by a user input will be described in more detail below with reference to FIG. 17.

[0082] According to one embodiment, the electronic device can identify an image captured through a camera application as an image corresponding to at least one keyword text. For example, when the electronic device receives a user input, such as capturing an image through the camera application and selecting a UI for generating an image displayable in a message application using the captured image, the electronic device can identify the captured image as an image corresponding to at least one keyword text. According to one embodiment, the operation of generating an image using the image captured through the camera application will be described in more detail below with reference to FIGS. 19A and 19B.

[0083] According to an embodiment, in operation 240, the electronic device may acquire at least one first layer based on the identified image. According to an embodiment, the electronic device may generate a set number of first layers, or determine the number of at least one first layer based on at least one of conversation information, at least one keyword text, or information related to the identified image, and generate the determined number of first layers. According to an embodiment, the operation of generating one first layer, which is the set number, will be described in more detail below with reference to FIG. 4. According to an embodiment, the operation of determining the number of at least one first layer and generating the determined number of at least one first layer will be described in more detail below with reference to FIG. 5.

[0084] In one embodiment, the electronic device may generate at least one first layer using generative AI. For example, when generating a set number of first layers, the electronic device may generate one first layer containing an object included in the identified image. In one embodiment, the first layer may be a main layer containing the main object of the image to be generated.

[0085] In one embodiment, when determining the number of at least one first layer and generating the determined number of at least one first layer, the electronic device may generate at least one first layer including a first layer including an object included in the identified image and / or a first layer including a background included in the identified image. In one embodiment, the first layer including the object may be a main layer, and the first layer including the background may be a background layer.

[0086] According to one embodiment, the electronic device may determine whether to use at least one of an object included in the identified image or a background included in the identified image based on at least one of conversation information, at least one keyword text, and information related to the identified image. According to one embodiment, the information related to the image may include image analysis information and / or meta information of the identified image. The image analysis information may include vision information such as objects included in the image (e.g., text in the image, people, clothing, weather, environment). The meta information may include information on the location where the image was taken (e.g., GPS information), date information, and / or tag information (e.g., text in the image or tags added by a user).

[0087] In one embodiment, the electronic device may generate a main layer including an object based on a determination that the object in the identified image is to be used. In one embodiment, the electronic device may generate a background layer including a background based on a determination that the background in the identified image is to be used.

[0088] In one embodiment, the electronic device may generate a prompt to be input to the generative AI based on at least one of conversation information, at least one keyword text, or information related to the identified image. In one embodiment, the prompt may be generated by considering whether the image (e.g., a sticker image) includes a motion effect (e.g., an animation effect).

[0089] In one embodiment, the electronic device may generate a prompt related to an effect to be applied to a main object included in the main layer based on at least one of conversation information, at least one keyword text, or information related to the identified image.

[0090] For example, the electronic device may generate a prompt (e.g., "Friends 'smiling'") that includes the text "friends" as an object related to the identified image, and the text "smiling" as an effect related to the effect.

[0091] In one embodiment, the electronic device may generate a prompt (e.g., "'swaying' palm tree") that includes text "palm tree" related to a background contained in the identified image and text "swaying" related to an effect.

[0092] According to one embodiment, the electronic device may use the generated prompt as input data of the generative AI and obtain at least one first layer as output data of the generative AI.

[0093] According to one embodiment, the operation of generating a layer based on at least one of conversation information, at least one keyword text, or information related to a confirmed image is described in more detail below with reference to FIG. 9 . According to one embodiment, the prompts input to the generative AI to generate each layer are described in more detail below with reference to FIGS. 10 , 11 , and 12 .

[0094] In one embodiment, in operation 250, the electronic device may obtain at least one second layer based on at least one of conversation information, keyword text, or information related to the identified image.

[0095] According to one embodiment, the electronic device may generate a set number of second layers, or determine the number of at least one second layer based on at least one of conversation information, at least one keyword text, or information related to the identified image, and generate the determined number of second layers. According to one embodiment, the operation of generating two first layers, which is the set number, will be described in more detail below with reference to FIG. 4. According to one embodiment, the operation of determining the number of at least one second layer and generating the determined number of at least one second layer will be described in more detail below with reference to FIG. 5.

[0096] According to one embodiment, the electronic device may generate at least one second layer using generative AI. For example, when generating a set number of second layers, the electronic device may generate two second layers based on at least one of conversation information, at least one keyword text, or information related to the identified image. According to one embodiment, the two second layers may include a sub-layer including a sub-object of the image to be generated and a background layer including a background of the image to be generated. According to one embodiment, the sub-object may be related to an effect of the image to be generated in relation to a conversation context based on the conversation information and at least one keyword.

[0097] In one embodiment, when determining the number of at least one second layer and generating the determined number of at least one second layer, the electronic device may generate one second layer including a sub-layer or a background layer, or may generate two or more sub-layers and / or two or more background layers.

[0098] In one embodiment, information related to an image may include image analysis information and / or metadata of the identified image. The image analysis information may include vision information such as objects included in the image (e.g., text, people, clothing, weather, and environment within the image). The metadata may include information about the location where the image was captured (e.g., GPS information), date information, and / or tag information (e.g., text within the image or tags added by the user).

[0099] According to one embodiment, if the electronic device cannot obtain at least one of the image analysis information of the identified image or the metadata of the identified image, the number of second layers may be determined to be one. According to one embodiment, if the image analysis information of the identified image is two or more and the metadata is two or more, the electronic device may determine the number of second layers to be two or more.

[0100] In one embodiment, the electronic device may generate a prompt to be input to the generative AI based on at least one of conversation information, at least one keyword text, or information related to the identified image. In one embodiment, the prompt may be generated by considering whether the image (e.g., a sticker image) includes a motion effect (e.g., an animation effect).

[0101] According to one embodiment, the electronic device may generate a prompt related to an effect to be applied to a sub-object included in a sub-layer and a prompt related to an effect to be applied to a background included in a background layer based on at least one of conversation information, at least one keyword text, or information related to an identified image.

[0102] For example, an electronic device may generate a prompt (e.g., "Airplane 'flying'") that includes the text 'airplane' as a sub-object and the text 'flying' as an effect.

[0103] In one embodiment, the electronic device may generate a prompt (e.g., "'swaying' palm tree") that includes the text 'palm tree' as a background-related text and 'swaying' as an effect-related text.

[0104] According to one embodiment, the electronic device can obtain at least one second layer as output data of the generative AI by using the generated prompt as input data of the generative AI.

[0105] According to one embodiment, the operation of generating a layer based on at least one of conversation information, at least one keyword text, or information related to a confirmed image is described in more detail below with reference to FIG. 9 . According to one embodiment, the prompts input to the generative AI to generate each layer are described in more detail below with reference to FIGS. 10 , 11 , and 12 .

[0106] According to one embodiment, the electronic device may determine the color of an object included in at least one second layer based on color information of an image identified based on at least one extracted keyword. According to one embodiment, the electronic device may determine the location of a sub-object based on the location of a main object included in an image identified based on at least one extracted keyword. According to one embodiment, an operation of identifying the color of the second layer and the location of a sub-object based on an image identified based on at least one keyword will be described in more detail below with reference to FIGS. 13A and 13B.

[0107] In one embodiment, in operation 260, the electronic device can generate an image by overlapping at least one first layer and at least one second layer.

[0108] According to one embodiment, the electronic device can generate an image by overlaying a main layer on a background layer and overlaying a sub-layer on the main layer.

[0109] According to one embodiment, the electronic device may display a layer used as a background layer of the generated image among at least one first layer and at least one second layer on the entire area of ​​the execution screen based on a user input for displaying the generated image. According to one embodiment, the electronic device may also display objects of sub-layers as well as the background layer of the generated image on the entire area of ​​the execution screen. According to one embodiment, an operation of displaying some layers of the generated image on the entire screen will be described in more detail with reference to FIGS. 14 and 15 below.

[0110] In one embodiment, the electronic device may display the generated image as a preview in the soft keyboard area. In another embodiment, the electronic device may modify the generated image based on user input for modifying the generated image. For example, the electronic device may change the main object or change the effects applied to each layer based on the user input. In another embodiment, the operation of modifying the generated image based on user input will be described in more detail with reference to FIGS. 16 and 18 below.

[0111] In this way, by generating images based on the content of conversations through messaging applications, personal emotions and verbal skinship can be richly expressed, and personalized images that fit the context of the conversation can be provided through image creation centered on user information.

[0112] FIG. 3A is a diagram illustrating an operation of generating an image based on conversation information in an electronic device according to one embodiment.

[0113] Referring to FIG. 3A, an electronic device (e.g., the electronic device (101) of FIG. 1 or the processor (120) of FIG. 1) may display an execution screen (310) of a message application. For example, the electronic device may summarize a conversation and extract at least one keyword text based on conversation content currently being displayed or previously displayed on the execution screen (310) of the message application. For example, if a travel-related conversation is conducted with friends who are conversation partners through the execution screen (310) of the message application, the electronic device may summarize the conversation content based on information mentioned in the conversation content and extract major keywords related to travel.

[0114] According to one embodiment, if the electronic device is capable of generating an image based on summarized conversation content and main keywords, the electronic device may display a UI for executing an AI function in an activated state within the soft keyboard area (320). For example, when a user input for selecting a text window for text input is received, the electronic device may display the soft keyboard area (320). For example, the soft keyboard area (320) may include a plurality of keys for text input and UIs for executing functions. For example, the electronic device may display the UI for executing the AI ​​function in an activated state that flashes.

[0115] According to one embodiment, when a user input (321) for selecting a UI for executing an AI function is received, the electronic device may display a list (330) related to the AI ​​function. For example, the electronic device may display a list (330) related to the AI ​​function instead of multiple keys for text input. For example, the list (330) related to the AI ​​function may include 'generative stickers' that generate images using AI, 'conversational translation' that performs translation using AI, 'sentence styles' that correct sentences using AI, and / or 'spelling and grammar' that corrects spelling errors using AI.

[0116] According to one embodiment, when a user input (331) is received to select a 'generative sticker' from a list (330) related to the AI ​​function, the electronic device may perform the image generation operation illustrated in FIG. 3b.

[0117] According to one embodiment, in FIG. 3a, the UI for executing the AI ​​function is illustrated and described as selecting a function for generating an image, but this is not limited thereto, and the UI for generating an image may also be displayed in the soft keyword area (320).

[0118] FIG. 3b is a diagram for explaining an operation of generating an image based on conversation information in an electronic device according to one embodiment.

[0119] Referring to FIG. 3B, the electronic device may display a screen (340) for generating an image based on extracted keyword text instead of a list related to the AI ​​function (e.g., the list (330) related to the AI ​​function of FIG. 3A). According to one embodiment, the screen (340) for generating an image may include a UI for providing at least one extracted keyword text and generating an image based on the provided at least one keyword text.

[0120] According to one embodiment, when a user input (341) for selecting a UI for generating an image is received, the electronic device may display a preview (350) of the generated image. According to one embodiment, the electronic device may extract a related image from among a plurality of images stored in a memory (e.g., a plurality of images stored in a photo album application) based on the extracted keyword text, and extract derived information keyword text from the extracted image. For example, the derived information keyword text may include image analysis information obtained by analyzing the image and meta information stored for the image. For example, the image analysis information may include information about a person included in the image (e.g., a person's name), and the meta information may include location information.

[0121] According to one embodiment, when a user input (351) is received for selecting a UI for transmitting a preview (350) of a generated image, the electronic device may display the generated image (360) in a portion of a running screen (e.g., a chat window) of a message application. According to one embodiment, the portion of the screen may be a designated area for displaying an image within the running screen.

[0122] According to one embodiment, the electronic device may display a UI (352) for modifying the generated image (e.g., regeneration) along with a preview (350) of the generated image. According to one embodiment, an image modification operation when the UI (352) for modifying the generated image is selected will be described in more detail with reference to FIGS. 16 and 18 below.

[0123] FIG. 4 is a flowchart illustrating an operation of generating an image by creating a set number of layers based on conversation information in an electronic device according to one embodiment. For example, FIG. 4 illustrates an operation of generating an image using three layers, which may include a main layer, a sub-layer, and a background layer.

[0124] Referring to FIG. 4, in operation 410, an electronic device (e.g., the electronic device (101) of FIG. 1 or the processor (120) of FIG. 1) may obtain conversation information. For example, the electronic device may obtain conversation information that is currently being displayed on the execution screen of a message application or that has a history of being displayed. According to one embodiment, the electronic device may obtain conversation information from a memory (e.g., the memory (130) of FIG. 1) or from a server associated with the message application.

[0125] According to one embodiment, in operation 420, the electronic device may summarize the conversation information and extract main keyword text based on the acquired conversation information, and may suggest image generation. According to one embodiment, the image generation suggestion may include an operation of activating and displaying a UI for image generation, suggesting image generation based on the extracted keyword text. According to one embodiment, the operation of summarizing the conversation information and extracting main keyword text is the same as operation 220 of FIG. 2, and therefore, a duplicate description will be omitted.

[0126] According to one embodiment, in operation 430, the electronic device may extract an image based on the extracted keyword text. According to one embodiment, the electronic device may extract an image related to the extracted keyword text from among a plurality of images stored in the memory. According to one embodiment, the image extraction operation is identical to operation 230 of FIG. 2, and thus, a duplicate description thereof will be omitted.

[0127] In one embodiment, in operation 440, the electronic device may separate the background and object of the extracted image and apply background erasure. In one embodiment, the electronic device may separate the object and background of the image and obtain only the object included in the extracted image, or delete the separated background.

[0128] According to one embodiment, in operation 450, the electronic device may extract image-related information from the extracted image. According to one embodiment, the image-related information may include image analysis information and / or meta-information of the identified image. The image analysis information may include vision information such as objects included in the image (e.g., text in the image, people, clothing, weather, environment). The meta-information may include information on the location where the image was taken (e.g., GPS information), date information, and / or tag information (e.g., text in the image or tags added by the user).

[0129] According to one embodiment, in operation 460, the electronic device may classify the extracted information by the main layer, sub-layer, and background layer (BG layer) of the image to be generated. For example, the electronic device may classify the information to be used for each layer among conversation information, keywords, and image-related information. According to one embodiment, the operation of classifying the information to be used for each layer will be described in more detail below with reference to FIG. 9.

[0130] According to an embodiment, in operation 471, the electronic device may generate a main layer. According to an embodiment, in operation 472, the electronic device may generate a sub-layer. According to an embodiment, in operation 473, the electronic device may generate a background layer. According to an embodiment, the electronic device may generate the main layer, the sub-layer, and the background layer using generative AI. According to an embodiment, the operation of generating each layer is the same as operation 240 and / or operation 250 of FIG. 2, and thus, redundant descriptions are omitted.

[0131] According to one embodiment, in operation 480, the electronic device may overlap the three generated layers. According to one embodiment, the electronic device may generate an image by overlapping the three layers. According to one embodiment, the operation of overlapping the generated layers to generate an image is identical to operation 260 of FIG. 2, and thus, a duplicate description will be omitted.

[0132] FIG. 5 is a flowchart illustrating an operation of determining the number of layers based on conversation information and generating an image by creating the determined number of layers in an electronic device according to an embodiment. For example, FIG. 5 is a diagram illustrating an operation of generating an image that further includes an operation of determining the number of layers and the layers to which the extracted image will be applied based on extracted information.

[0133] Referring to FIG. 5, in operation 501, an electronic device (e.g., the electronic device (101) of FIG. 1 or the processor (120) of FIG. 1) may obtain conversation information. For example, the electronic device may obtain conversation information that is currently being displayed on the execution screen of a message application or that has a history of being displayed. According to one embodiment, the electronic device may obtain conversation information from a memory (e.g., the memory (130) of FIG. 1) or from a server associated with the message application.

[0134] According to one embodiment, in operation 502, the electronic device may summarize the conversation information and extract main keyword text based on the acquired conversation information, and may suggest image generation. According to one embodiment, the image generation suggestion may include an operation of activating and displaying a UI for image generation, suggesting image generation based on the extracted keyword text. According to one embodiment, the operation of summarizing the conversation information and extracting main keyword text is the same as operation 220 of FIG. 2, and therefore, a duplicate description will be omitted.

[0135] According to one embodiment, in operation 503, the electronic device may extract an image based on the extracted keyword text. According to one embodiment, the electronic device may extract an image related to the extracted keyword text from among a plurality of images stored in the memory. According to one embodiment, the image extraction operation is identical to operation 230 of FIG. 2, and thus, a duplicate description thereof will be omitted.

[0136] In one embodiment, in operation 504, the electronic device may separate the background and the object of the extracted image. In one embodiment, the electronic device may separate the object and the background contained in the extracted image to generate a main layer based on the object and / or a background layer based on the background.

[0137] According to one embodiment, in operation 505, the electronic device may extract image-related information from the extracted image. According to one embodiment, the image-related information may include image analysis information and / or meta-information of the identified image. The image analysis information may include vision information such as objects included in the image (e.g., text in the image, people, clothing, weather, environment). The meta-information may include information on the location where the image was taken (e.g., GPS information), date information, and / or tag information (e.g., text in the image or tags added by the user).

[0138] According to one embodiment, in operation 506, the electronic device may determine the number of layers of the image to be generated. According to one embodiment, the electronic device may determine the number of images to be generated based on whether conversation content, extracted keyword text, or image-related information is acquired. For example, based on whether a main layer and / or a background layer are generated based on the extracted image, it may be determined that one or two layers are generated based on the extracted image. According to one embodiment, based on the number of pieces of information acquired among image analysis information and image meta information included in the image-related information, it may be determined that one or more layers are generated. For example, if only one piece of information among image analysis information and image meta information is acquired, the electronic device may determine that only one layer among the sub layer and the background layer is generated.

[0139] In one embodiment, the electronic device may be configured to have two or more layers.

[0140] In one embodiment, in operation 507, the electronic device may determine a layer to which the extracted image is to be applied. In one embodiment, the electronic device may determine whether to use an object and / or background of the extracted image based on conversation content, extracted keyword text, or image-related information. In one embodiment, if the object of the extracted image is determined to be used, the electronic device may determine that the extracted image is to be applied to the main layer. In one embodiment, if the background of the extracted image is determined to be used, the electronic device may determine that the extracted image is to be applied to the background layer. In one embodiment, if the background and object of the extracted image are determined to be used, the electronic device may determine that the extracted image is to be applied to the main layer and the background layer.

[0141] According to one embodiment, in operation 508, the electronic device may classify the extracted information by at least two layers of the image to be generated. For example, the electronic device may classify the extracted information by at least two layers among a main layer, a sub layer, and a background layer (BG layer). For example, the electronic device may classify information to be used by each layer among conversation information, keywords, and image-related information. According to one embodiment, the operation of classifying information to be used by each layer will be described in more detail below with reference to FIG. 9.

[0142] In one embodiment, in operation 509, the electronic device may generate at least two layers of an image to be generated. In one embodiment, the electronic device may generate at least two layers of a main layer, a sub layer, or a background layer using generative AI. In one embodiment, the operation of generating each layer is the same as operation 240 and / or operation 250 of FIG. 2, and thus, a duplicate description is omitted.

[0143] According to one embodiment, in operation 510, the electronic device may overlap at least two generated layers. According to one embodiment, the electronic device may generate an image by overlapping at least two layers. According to one embodiment, the operation of overlapping the generated layers to generate an image is identical to operation 260 of FIG. 2, and thus, a duplicate description will be omitted.

[0144] FIG. 6 is a diagram for explaining an operation of generating an image based on generative AI in an electronic device according to one embodiment.

[0145] Referring to FIG. 6, an electronic device (e.g., the electronic device (101) of FIG. 1 or the processor (120) of FIG. 1) may input input text (601) into a large language model (LLM) (610). For example, the input text (601) may include conversation content. According to one embodiment, the electronic device may summarize conversation information and extract key keyword texts through the LLM (610).

[0146] According to one embodiment, the electronic device may extract related images from among a plurality of images stored in a native gallery application (620) using the extracted keyword text. According to one embodiment, the native gallery application (620) may be already stored in the memory of the electronic device (e.g., the memory (130) of FIG. 1) when the electronic device is shipped. According to one embodiment, the electronic device may separate objects and backgrounds of the extracted images.

[0147] According to one embodiment, the electronic device can obtain image-related information about the extracted image using a CNN model and LLM (630). For example, the electronic device can obtain image analysis information related to objects included in the image (e.g., text, people, clothing, weather, and environment within the image) through the CNN model. According to one embodiment, the electronic device can input the image analysis information into the LLM to obtain keyword text related to the image.

[0148] According to one embodiment, the electronic device can obtain meta information (631) of the extracted image. For example, the meta information (631) can include information about the location where the image was captured, date information, and / or related tag information.

[0149] According to one embodiment, the electronic device may input keyword text and meta information (631) based on image analysis information into the LLM (640), thereby generating words, phrases, and / or sentences for image generation based on the extracted image and image-related information. For example, the electronic device may generate words, phrases, and / or sentences related to people, moods, objects, places, dates, and weather to be used in image generation.

[0150] According to one embodiment, the electronic device may input words, phrases, and / or sentences generated by a sequence-to-sequence model (seq2seq model) (650) and generate a prompt including descriptive text for generating an effect (e.g., animation) as output data. For example, if the object is a friend, the electronic device may generate a prompt such as "'smiling' friends" including descriptive text such as "smiling," or if the object is an airplane, the electronic device may generate a prompt such as "'flying' airplane" including descriptive text such as "flying," or if the object is a palm tree, the electronic device may generate a prompt such as "'swaying' palm tree" including descriptive text such as "swaying." According to one embodiment, the electronic device may generate a prompt corresponding to each of the main layer, the sub-layer, and / or the background layer.

[0151] In one embodiment, the electronic device may generate each layer including an effect through an image generation model (e.g., a diffusion depth estimation model) (660). In one embodiment, the electronic device may first generate a main layer to match the color tone of each layer, and then generate sub-layers and background layers to match the color of the main layer.

[0152] According to one embodiment, the electronic device may output (602) an image generated by overlapping each generated layer. According to one embodiment, the electronic device may encode the generated image and transmit it via a messaging application.

[0153] FIG. 7 is a flowchart illustrating a system for generating an image based on conversation information, according to an embodiment of the present disclosure. For example, FIG. 7 illustrates an application 3 installed by user input. rd This is a drawing for explaining the operation of creating an image through a party service (700).

[0154] Referring to FIG. 7, the operation of generating an image based on the conversation content may include an operation of extracting a conversation information summary and main keyword text (710), an operation of extracting related images within a gallery app through the extracted keywords (720), an operation of extracting image-related information from the extracted images (730), an operation of classifying the extracted information by layer of the image to be generated (740), an operation of generating a layer (750), and an operation of generating an image by overlapping each layer (760).

[0155] According to one embodiment, the operation (710) of extracting conversation information summary and main keyword text is performed in operation 711, 3 rd Party service (700) is 3 rd An action may include executing a party message application. According to one embodiment, in operation 712, an electronic device (e.g., electronic device (101) of FIG. 1 or processor (120) of FIG. 1) may: rd Once the execution of the party message application is confirmed, the AI ​​within the terminal can be executed.

[0156] In one embodiment, in operation 713, 3 rdThe party service (700) can check whether the AI ​​running on the electronic device (101) can be called up. According to one embodiment, 3 rd If the party service (700) cannot call the AI ​​running on the electronic device (101) (action 713 - No)), it proceeds to action 780, 3 rd A preset image previously provided within the party service (700) can be transmitted. According to one embodiment, 3 rd If the party service (700) can call the AI ​​running on the electronic device (101) (operation 713 - yes), in operations 714 and 715, the AI ​​API can be synchronized with the server (108) (e.g., the server (108) of FIG. 1 or the electronic device (101). According to one embodiment, if the keyword extraction and image generation operations are performed on the server (108), 3 rd The party service (700) can perform synchronization with the server (108). In one embodiment, when the keyword extraction and image generation operations are performed on-device in the electronic device (101), 3 rd The party service (700) can perform synchronization with the electronic device (101). In the following, the keyword extraction and image generation operations are described as being performed on the server (108), but are not limited thereto and can also be performed on the electronic device (101).

[0157] For example, 3 rd The party service (700) transmits the message conversation content to the server (108), and the server (108) sends the generated image (e.g., sticker image) to 3 rd Synchronization can be performed by transmitting to a party service (700).

[0158] In one embodiment, in operation 716, the electronic device (101) may compose a message conversation within the terminal. For example, the electronic device (101) may compose a message conversation based on user input via a soft keyboard.

[0159] In one embodiment, in operation 717, 3 rd The party service (700) can generate message conversation content based on the message conversation written in the electronic device (101). For example, the electronic device may generate 3 rd The party service (700) can generate message conversation content by displaying a message conversation written in an electronic device (101) on the execution screen.

[0160] In one embodiment, in operation 718, the server (108) 3 rd Based on the message conversation content received from the party service (700), LLM context analysis and main keywords can be extracted.

[0161] According to one embodiment, the operation (720) of extracting related images within the gallery app through extracted keywords may include, in operation 721, an operation of the electronic device (101) extracting images within the gallery application based on keyword information received from the server (108). For example, the electronic device (101) may compare the extracted keywords with information about people, objects, and animals, location information, date information, and / or tag information contained in images stored for each of the plurality of images in the gallery application, and extract one image from among the plurality of images.

[0162] According to one embodiment, the operation (730) of extracting image-related information from the extracted image may include, in operation 731, an operation of the server (108) extracting image analysis information and meta information from the extracted image. According to one embodiment, the image-related information may include image analysis information and / or meta information of the identified image. The image analysis information may include vision information such as objects included in the image (e.g., text in the image, people, clothing, weather, environment). The meta information may include information on a location where the image was taken (e.g., GPS information), date information, and / or tag information (e.g., text in the image or tags added by a user).

[0163] According to one embodiment, the operation (740) of classifying the extracted information by layer of the image to be generated may include, in operation 741, the operation of the server (108) classifying the extracted information by layer of the image to be generated. For example, the server (108) may classify information to be used by the main layer, sub layer, and background layer (BG layer) of the image to be generated among conversation information, keywords, and image-related information.

[0164] In one embodiment, the layer creation operation (750) may include, in operation 751, an operation in which the server (108) generates images and effects for each layer of the image. For example, the server (108) may generate images and effects for each of the main layer, sub-layer, and background layer. In one embodiment, the server (108) may generate the images and effects for each layer using generative AI.

[0165] According to one embodiment, in operation 752, the electronic device (101) may receive images and effects of each layer generated from the server (108) and provide recommended images and effects for each layer. For example, the electronic device (101) may display images and effects generated for each layer to allow the user to check the images and effects for each layer before generating the images. According to one embodiment, when a user input for changing the images and / or effects for each layer is received, the electronic device (101) may transmit change information to the server (108), so that the server (108) may change the images and / or effects for each layer.

[0166] According to one embodiment, the operation (760) of generating an image by overlapping each layer may include, in operation 761, the operation of the server (108) generating an image by overlapping each layer. For example, the server (108) may generate an image by overlapping a background layer, a main layer, and a sub layer from below.

[0167] According to one embodiment, the electronic device (101) may receive the generated image and display it as a preview. According to one embodiment, in operation 762, the electronic device (101) may receive a user input for finally selecting the image displayed as a preview. For example, the electronic device (101) may select the image displayed as a preview by 3 rd User input can be received for display on the execution screen of the party service (700).

[0168] In one embodiment, in operation 770, 3 rd The party service (700) can transmit the generated image. For example, 3 rd The party service (700) can transmit the generated image to the conversation partner by displaying the generated image on the execution screen.

[0169] FIG. 8 is a diagram for explaining an operation of generating an image using multiple layers generated based on conversation information in an electronic device according to one embodiment.

[0170] Referring to FIG. 8, an electronic device (e.g., an electronic device (101) of FIG. 1 or a processor (120) of FIG. 1) can generate an image (840) (e.g., a sticker image, an emoticon, an emoji, a still image, a moving image) that can be displayed on the execution screen of a message application through overlapping of a main layer (810), a sub layer (820), and a background layer (830).

[0171] According to one embodiment, the electronic device may obtain a main layer (810) as output data by using an image extracted from a conversation, vision information of the extracted image, and the extracted keyword text as input data of a generative AI. According to one embodiment, the main layer (810) may be obtained by regenerating a portion of an object of the extracted image. For example, the electronic device may generate a prompt, such as “Make the input image have an excited expression,” based on the extracted keywords “travel” and “excited,” and input the prompt to the generative AI to regenerate a portion of the extracted image. For example, the main layer (810) may include a person object (811) regenerated with an excited expression.

[0172] According to one embodiment, the electronic device may obtain a sub-layer (820) as output data by using the vision information of the image and the extracted keywords as input data of the generative AI based on the keyword text extracted from the conversation content. According to one embodiment, the sub-layer (820) may be obtained by newly generating a sub-object (821). For example, the electronic device may generate a prompt such as “make me a pair of sparkly sunglasses” based on the vision information of the image, such as “women” and “people,” and input the prompt into the generative AI to generate the sub-object (821). For example, the sub-layer (820) may include a sparkly sunglasses object (821).

[0173] According to one embodiment, the electronic device may use the extracted image's meta information (e.g., location, date, tag) and the extracted keywords as input data of the generative AI based on the keyword text extracted from the conversation content to obtain a background layer (830) as output data. According to one embodiment, the background layer (830) may be obtained by newly generating a background object (831). For example, the electronic device may generate a prompt, such as "Make a swaying palm tree," based on the location information (e.g., Hawaii), which is the meta information of the image, and input the prompt into the generative AI to generate the background object (831). For example, the background layer (830) may include a swaying palm tree object (831).

[0174] According to one embodiment, the electronic device may generate a main layer (810), a sub layer (820), and a background layer (830) in that order, and may generate the sub layer (820) and the background layer (830) based on the style (e.g., color) of the main layer (810).

[0175] FIG. 9 is a flowchart illustrating an operation of generating multiple layers based on conversation information in an electronic device according to one embodiment.

[0176] Referring to FIG. 9, in operation 901, an electronic device (e.g., the electronic device (101) of FIG. 1 or the processor (120) of FIG. 1) may extract keywords from the conversation content. According to one embodiment, the operation of extracting keywords is identical to operation 220 of FIG. 2, and thus, a duplicate description is omitted.

[0177] According to one embodiment, in operation 902, the electronic device may extract an image based on a conversation keyword from among a plurality of images stored in a memory (e.g., memory (130) of FIG. 1). According to one embodiment, the operation of extracting the image is identical to operation 230 of FIG. 2, and thus, a duplicate description will be omitted.

[0178] In one embodiment, in operation 903, the electronic device may derive keywords from image vision information of the extracted image. For example, the electronic device may derive keywords related to text, people, clothing, weather, and / or environment within the image.

[0179] In one embodiment, in operation 904, the electronic device may derive keywords from the metadata of the extracted image. For example, the electronic device may derive keywords related to location information, date information, and / or tag information of the image.

[0180] According to one embodiment, in operation 905, the electronic device may use the image extracted in operation 902, the keyword extracted in operation 903, and / or the keyword extracted in operation 904 as information for generating a main layer. According to one embodiment, the electronic device may prioritize conversation information, vision information, and meta information in that order when classifying information to be used in the main layer. For example, in order to generate an image that matches the conversation context, the electronic device may preferentially use information obtained through conversation content for generating a main layer, and if there is no additional information to be used from the conversation information, the electronic device may use information extracted from image vision information and information extracted from image meta information for generating a layer.

[0181] According to one embodiment, in operation 906, the electronic device may derive keywords related to the sub-object from the conversation content keyword information. According to one embodiment, after extracting keywords through operation 903 and extracting keywords through operation 904, the electronic device may additionally derive keywords related to the sub-object from the conversation content keyword information.

[0182] According to one embodiment, in operation 907, the electronic device may use the image extracted in operation 903, the keyword extracted in operation 904, and / or the keyword extracted in operation 906 as information for generating a sublayer. According to one embodiment, the electronic device may prioritize vision information, conversation information, and meta information in that order when classifying information to be used in the sublayer. For example, the electronic device may use vision information of the extracted image as a priority for generating a sublayer to generate a sub-object that is not incongruous with the main layer.

[0183] According to one embodiment, in operation 908, the electronic device may derive background and / or location-related keywords from the conversation content keyword information. According to one embodiment, after extracting keywords through operation 903 and extracting keywords through operation 904, the electronic device may additionally derive background and / or location-related keywords from the conversation content keyword information.

[0184] According to one embodiment, in operation 909, the electronic device may use the image extracted in operation 903, the keyword extracted in operation 904, and / or the keyword extracted in operation 908 as information for creating a background layer. According to one embodiment, the electronic device may prioritize meta information, dialogue information, and vision information in that order when classifying information to be used in the background layer. For example, the electronic device may create a background object that is not incongruous with the main layer, but may use meta information as a priority for creating the background layer to provide a colorful image.

[0185] FIG. 10 is a diagram for explaining an operation of generating multiple layers based on conversation information in an electronic device according to one embodiment.

[0186] Referring to FIG. 10, an electronic device (e.g., the electronic device (101) of FIG. 1 or the processor (120) of FIG. 1) may extract keyword text based on conversation information that is being displayed or has been displayed on the execution screen (1010) of a message application. According to one embodiment, information that may be obtained based on the keyword text may include an event (e.g., a trip, a birthday, an anniversary, a vacation), a date, a time, an emotion (e.g., joy, anger, sorrow, pleasure), and / or a location.

[0187] In one embodiment, the electronic device may acquire relevant images from among a plurality of stored images based on extracted keyword text. For example, the electronic device may acquire relevant images based on keyword texts included in the conversation, such as "travel," "overseas," "person's name (e.g., Minju, Saebyeok)," and "exciting."

[0188] According to one embodiment, the electronic device may obtain image-related information from the extracted image. For example, the electronic device may obtain image vision information (e.g., people, objects, animals) from the image. According to one embodiment, the electronic device may obtain meta-information (e.g., location information, date information, tag information) of the extracted image. For example, the location information may include the detailed location where the image was captured (e.g., Paris) and landmark information (e.g., the Eiffel Tower). According to one embodiment, the date information may include the date the image was captured (e.g., an anniversary, an appointment date). According to one embodiment, the tag information may include text in the image and / or tags added by the user.

[0189] According to one embodiment, the electronic device may generate a prompt (1020) for generating a main layer (1021) based on the conversation content. For example, the prompt (1020) for generating the main layer (1021) may be "Regenerate with images / effects of exciting friends," which includes keywords such as "travel" and "exciting" included in the conversation content. According to one embodiment, the electronic device may input the prompt (1020) for generating the main layer (1021) into a generative AI and obtain the main layer (1021) as output data. According to one embodiment, the main layer (1021) may include an exciting friends object (1022).

[0190] According to one embodiment, the electronic device may generate a prompt (1030) for generating a sub-layer (1031) based on the conversation content, extracted keywords, and / or image vision information. For example, the prompt (1030) for generating the sub-layer (1031) may be “Generate an object image / animation suitable for person, woman, and travel,” which includes “travel,” a keyword included in the conversation content, and “person,” and “woman,” keywords included in the vision information. According to one embodiment, the electronic device may input the prompt (1030) for generating the sub-layer (1031) into a generative AI, and obtain the sub-layer (1031) as output data. According to one embodiment, the sub-layer (1031) may include a sunglasses object (1032).

[0191] According to one embodiment, the electronic device may generate a prompt (1040) for generating a background layer (1041) based on image metadata. For example, the prompt (1040) for generating the background layer (1041) may be 'Generate a background image / animation related to Hawaii', which includes 'Hawaii' as a keyword related to location information included in the metadata. According to one embodiment, the electronic device may input the prompt (1040) for generating the background layer (1041) into a generative AI and obtain the background layer (1041) as output data. According to one embodiment, the background layer (1041) may include a swaying palm tree object (1042).

[0192] According to one embodiment, the electronic device may generate a final image (1050) by overlapping a main layer (1021), a sub-layer (1031), and a background layer (1041). For example, the electronic device may generate a final image (1050) by overlapping an exciting friends object (1022) included in the main layer (1021), a sunglasses object (1032) included in the sub-layer (1031), and a swaying palm tree object (1042) included in the background layer (1041).

[0193] FIG. 11 is a diagram for explaining an operation of generating multiple layers based on conversation information in an electronic device according to one embodiment.

[0194] Referring to FIG. 11, an electronic device (e.g., the electronic device (101) of FIG. 1 or the processor (120) of FIG. 1) may extract keyword text based on conversation information that is being displayed or has been displayed on the execution screen (1110) of a message application. According to one embodiment, information that may be obtained based on the keyword text may include an event (e.g., a trip, a birthday, an anniversary, a vacation), a date, a time, an emotion (e.g., joy, anger, sorrow, pleasure), and / or a location.

[0195] In one embodiment, the electronic device may acquire a relevant image from among a plurality of stored images based on extracted keyword text. For example, the electronic device may acquire a relevant image (e.g., an image of a birthday cake) based on keyword texts included in the conversation, such as "birthday" and "happiness."

[0196] According to one embodiment, the electronic device may obtain image-related information from the extracted image. For example, the electronic device may obtain image vision information (e.g., people, objects, animals) from the image. According to one embodiment, the electronic device may obtain meta-information (e.g., location information, date information, tag information) of the extracted image. For example, the location information may include the detailed location where the image was captured (e.g., Paris) and landmark information (e.g., the Eiffel Tower). According to one embodiment, the date information may include the date the image was captured (e.g., an anniversary, an appointment date). According to one embodiment, the tag information may include text in the image and / or tags added by the user.

[0197] According to one embodiment, the electronic device may generate a prompt (1120) for generating a main layer (1121) based on the conversation content. For example, the prompt (1120) for generating the main layer (1121) may be "Regenerate it with an image / effect of a birthday cake," which includes the keyword "birthday" included in the conversation content. According to one embodiment, the electronic device may input the prompt (1120) for generating the main layer (1121) into a generative AI and obtain the main layer (1121) as output data. According to one embodiment, the main layer (1121) may include a birthday cake object (1122) with flickering candles.

[0198] According to one embodiment, the electronic device may generate a prompt (1130) for generating a sub-layer (1131) based on the conversation content, extracted keywords, and / or image vision information. For example, the prompt (1130) for generating the sub-layer (1131) may be “Generate an object image / animation suitable for cake, happy, and birthday”, which includes keywords “birthday” and “happy” included in the conversation content and “cake” included in the vision information. According to one embodiment, the electronic device may input the prompt (1130) for generating the sub-layer (1131) into a generative AI and obtain the sub-layer (1131) as output data. According to one embodiment, the sub-layer (1131) may include a rising balloon object (1132).

[0199] In one implementation, if there is no information for creating a sub-object in the conversation content, extracted keywords, and vision information, the electronic device may create a sub-layer by generating a text phrase (e.g., Happy 20th Birthday!) that includes the extracted keywords (e.g., birthday).

[0200] According to one embodiment, the electronic device may generate a prompt (1140) for generating a background layer (1141) based on image vision information and conversation content. For example, the prompt (1140) for generating the background layer (1141) may be “Generate a background image / animation related to ‘happy, birthday’”, which includes keywords ‘cake’ and ‘balloon’ included in the image vision information, and keywords ‘birthday’ and ‘happy’ included in the conversation content. According to one embodiment, the electronic device may input the prompt (1140) for generating the background layer (1141) into a generative AI, and obtain the background layer (1141) as output data. According to one embodiment, the background layer (1141) may include a scattering pollen object (1142).

[0201] According to one embodiment, the electronic device may generate a final image (1150) by overlapping a main layer (1121), a sub-layer (1131), and a background layer (1141). For example, the electronic device may generate a final image (1150) by overlapping a birthday cake object (1122) with flickering candles included in the main layer (1121), a rising balloon object (1132) included in the sub-layer (1131), and a scattering pollen object (1142) included in the background layer (1141).

[0202] According to one embodiment of the present disclosure, even if the conversation content is the same, different images may be generated if the images stored on the electronic device are different. For example, although each user in a conversation group shares the same conversation content, the generated images may differ because each user's device stores different images.

[0203] FIG. 12 is a diagram for explaining an operation of generating multiple layers based on conversation information in an electronic device according to one embodiment.

[0204] Referring to FIG. 12, an electronic device (e.g., the electronic device (101) of FIG. 1 or the processor (120) of FIG. 1) may extract keyword text based on conversation information that is being displayed or has been displayed on the execution screen (1210) of a message application. According to one embodiment, information that may be obtained based on the keyword text may include an event (e.g., a trip, a birthday, an anniversary, a vacation), a date, a time, an emotion (e.g., joy, anger, sorrow, pleasure), and / or a location.

[0205] In one embodiment, the electronic device may acquire a relevant image from among a plurality of stored images based on extracted keyword text. For example, the electronic device may acquire a relevant image based on keyword texts included in the conversation, such as "wedding," "anniversary," "happy," and "March 21st."

[0206] According to one embodiment, the electronic device may obtain image-related information from the extracted image. For example, the electronic device may obtain image vision information (e.g., people, objects, animals) from the image. According to one embodiment, the electronic device may obtain meta-information (e.g., location information, date information, tag information) of the extracted image. For example, the location information may include the detailed location where the image was captured (e.g., Paris) and landmark information (e.g., the Eiffel Tower). According to one embodiment, the date information may include the date the image was captured (e.g., an anniversary, an appointment date). According to one embodiment, the tag information may include text in the image and / or tags added by the user.

[0207] According to one embodiment, the electronic device may generate a prompt (1220) for generating a main layer (1221) based on the conversation content. For example, the prompt (1220) for generating the main layer (1221) may be "Regenerate the image / effect with 'happy, marriage'", which includes keywords 'marriage' and 'happy' included in the conversation content. According to one embodiment, the electronic device may input the prompt (1220) for generating the main layer (1221) into a generative AI and obtain the main layer (1221) as output data. According to one embodiment, the main layer (1221) may include a couple object (1222) facing each other.

[0208] According to one embodiment, the electronic device may generate a prompt (1230) for generating a sub-layer (1231) based on the conversation content, extracted keywords, and / or image vision information. For example, the prompt (1230) for generating the sub-layer (1231) may be “Generate an object image / animation suitable for marriage, groom, bride, and wedding,” which includes keywords “marriage,” “wedding,” and “anniversary” included in the conversation content and keywords “couple,” “love,” “groom,” and “bride” included in the vision information. According to one embodiment, the electronic device may input the prompt (1230) for generating the sub-layer (1231) into a generative AI and obtain the sub-layer (1231) as output data. According to one embodiment, the sub-layer (1231) may include a wreath object (1232) of a blooming flower.

[0209] According to one embodiment, the electronic device may generate a prompt (1240) for generating a background layer (1241) based on image vision information and image meta information. For example, the prompt (1240) for generating the background layer (1241) may be "Generate a background image / animation related to a wedding hall," which includes keywords related to the image vision information, such as "couple," "love," "groom," and "bride," and "wedding hall," which is a keyword related to location information included in the meta information. According to one embodiment, the electronic device may input the prompt (1240) for generating the background layer (1241) into a generative AI and obtain the background layer (1241) as output data. According to one embodiment, the background layer (1241) may include a vine object (1242) with a flower blooming.

[0210] According to one embodiment, the electronic device may generate a final image (1250) by overlapping a main layer (1221), a sub-layer (1231), and a background layer (1241). For example, the electronic device may generate a final image (1250) by overlapping a facing couple object (1222) included in the main layer (1221), a blooming wreath object (1232) included in the sub-layer (1231), and a blooming vine object (1242) included in the background layer (1241).

[0211] FIG. 13A is a diagram for explaining an operation of generating a sub-layer based on color information of an image extracted based on conversation information in an electronic device according to one embodiment.

[0212] Referring to FIG. 13A, an electronic device (e.g., the electronic device (101) of FIG. 1 or the processor (120) of FIG. 1) can identify primary color information (1311) of an object (1310) included in a main layer. For example, the color information (1311) may include color and brightness. According to one embodiment, the object (1310) included in the main layer may be an object included in an image extracted based on keywords extracted from conversation content.

[0213] According to one embodiment, the electronic device may extract primary color information (1311) of an object (1310) included in a main layer and generate a prompt (1320) for generating a sub-layer (1330) with colors and brightness within a similar color group and brightness group. For example, the prompt (1320) for generating the sub-layer (1330) may be, "Generate a pair of shiny sunglasses with a brown color and mid tone."

[0214] In one embodiment, the electronic device may input a prompt (1320) to generate a sublayer (1330) into the generative AI and obtain the sublayer (1330) as output data. In one embodiment, the sublayer (1330) may include a shiny brown sunglasses object (1331).

[0215] FIG. 13b is a diagram for explaining an operation of placing objects in a sub-layer based on objects in an image extracted based on conversation information in an electronic device according to one embodiment.

[0216] Referring to FIG. 13b, the electronic device may place a sub-object in an area (1340) of the main layer where deformation has not occurred due to the generative AI. For example, if the face of the main object of the main layer has been regenerated with an excited expression through the generative AI, the electronic device may place a sub-object in an area surrounding the main object where deformation has not occurred.

[0217] In one embodiment, if the sub-object is related to a person, the electronic device may be placed in a peripheral area (1341) of the main object that is the person. For example, if the sub-object is a sunglasses object that is mounted on the face of a person, the electronic device may be placed in a peripheral area (1341) of the face of the person object.

[0218] In this way, the electronic device can generate a final image (1350) in which a sub-object (1331) that is sunglasses is placed on the eyes or head of the person.

[0219] FIG. 14 is a drawing for explaining an operation of displaying a generated image in full screen of an electronic device according to one embodiment.

[0220] Referring to FIG. 14, when an electronic device (e.g., the electronic device (101) of FIG. 1 or the processor (120) of FIG. 1) generates an image through a message application, which is a 3rd party service downloaded and installed by user input, the electronic device can display the generated image (1412) in a designated area (1411) within the execution screen (1410) of the 3rd party service.

[0221] According to one embodiment, when an image is created through a message application, which is a Native service already stored when the electronic device is shipped, the main layer (1422) of the created image may be displayed in a designated area, and some layers (1421) (e.g., a sub-layer or a background layer) may be displayed through the entire area (1423) within the execution screen (1420) of the Native service. For example, when the electronic device receives a user input for displaying the created image or a user input for selecting the displayed image while the created image is displayed in the designated area, the cake object included in the main layer (1422) may be maintained in the designated area, and the rising balloon object of the sub-layer (1421) may be displayed through the entire area (1423) within the execution screen (1420) of the Native service.

[0222] FIG. 15 is a drawing for explaining an operation of displaying a generated image in full screen in an electronic device according to one embodiment.

[0223] Referring to FIG. 15, an electronic device (e.g., the electronic device (101) of FIG. 1 or the processor (120) of FIG. 1) may display a preview image (1510) of a generated image instead of a soft keyboard in a soft keyboard area, and when a user input (1511) for displaying the generated image is received, may display a screen for selecting which layer among a sub-layer or a background layer among the generated images to display in full screen. According to one embodiment, the electronic device may obtain attribute information for displaying the sub-layer or the background layer in full screen. For example, the attribute information may include an aspect ratio of the full screen (e.g., 16:9), a position of a layer (e.g., front, back), and a display time (e.g., 3 seconds).

[0224] According to one embodiment, the electronic device may display a preview image (1520) of a sub-layer, and when a UI (1521) for displaying the sub-layer in full screen is selected, display a main layer (1530) including a birthday cake object in a designated area of ​​the execution screen of the message application, and display a sub-layer (1531) including a balloon object floating in the entire area of ​​the execution screen of the message application.

[0225] In one embodiment, the electronic device may display the sublayer (1531) in full screen using a prompt such as 'show the image and effect information of the sublayer, screen ratio 16:9, position in front of the main layer, and display in full screen for 3 seconds'.

[0226] According to one embodiment, the electronic device may display a preview image (1540) of a background layer, and when a UI (1541) for displaying the background layer in full screen is selected, display a main layer (1530) including a birthday cake object in a designated area of ​​the execution screen of the message application, and display a background layer (1551) including a flower pollen object scattered in the entire area of ​​the execution screen of the message application.

[0227] In one embodiment, the electronic device may display the background layer (1551) in full screen using a prompt such as 'image and effect information of the background layer, aspect ratio of 16:9, positioned behind the main layer, and displayed in full screen for 3 seconds'.

[0228] FIG. 16 is a drawing for explaining an operation of modifying a generated image of an electronic device according to one embodiment.

[0229] Referring to FIG. 16, an electronic device (e.g., electronic device (101) of FIG. 1 or processor (120) of FIG. 1) may provide a preview (1610) of a generated image. For example, the electronic device may display a preview (1610) of the generated image instead of multiple keys in a soft keyboard area.

[0230] According to one embodiment, when an electronic device receives a user input (1611) for regenerating a generated image, it may display a screen for modifying the generated image. According to one embodiment, the electronic device may display objects (1620) with different effects applied to each layer (main, sub background) through the screen for modifying the generated image. According to one embodiment, when a user input for selecting an object with different effects applied through a swipe input (1621) is received, the electronic device may modify the generated image by reflecting the selected object. According to one embodiment, a preview of the modified image may be included in an area of ​​the screen for modifying the generated image, and the electronic device may provide the modified image in real time as an object for each layer is selected.

[0231] FIG. 17 is a drawing for explaining an operation of generating an image based on an image selected by a user input in an electronic device according to one embodiment.

[0232] Referring to FIG. 17, an electronic device (e.g., the electronic device (101) of FIG. 1 or the processor (120) of FIG. 1) may display a screen that provides keywords extracted based on conversation content and suggests image generation. According to one embodiment, when a user input (1710) for selecting an image (e.g., a photo) is received through the screen suggesting image generation, the electronic device may display a plurality of images (1720) stored in a memory (e.g., the memory (130) of FIG. 1).

[0233] In one embodiment, when a user input is received to select one image (1721) from among multiple images (1720), the electronic device may redisplay a screen suggesting image generation.

[0234] In one embodiment, when a user input (1730) for generating an image is received through a screen suggesting image generation, the electronic device may display a preview (1740) of the generated image. In one embodiment, the generated image may be a main layer, a sub-layer, and a background layer overlapping each other based on a selected image (1721) and the conversation content.

[0235] According to one embodiment, when a user input (1741) for transmitting a generated image through the execution screen of a message application is received, the electronic device can display the generated image (1750) in an area of ​​the execution screen of the message application.

[0236] In this way, personalized images can be created by selecting an image to be applied to the main layer through user input.

[0237] FIG. 18 is a drawing for explaining an operation of modifying a generated image based on an image selected by a user input of an electronic device according to one embodiment.

[0238] Referring to FIG. 18, an electronic device (e.g., the electronic device (101) of FIG. 1 or the processor (120) of FIG. 1) may display a preview (1810) of a generated image. According to one embodiment, the generated image may be generated as an image extracted from among a plurality of images stored based on the conversation content.

[0239] According to one embodiment, when a user input (1811) for modifying a generated image is received, the electronic device may display a screen for modifying the generated image. According to one embodiment, the electronic device may display objects with different effects applied to each layer (main, sub-background) through the screen for modifying the generated image.

[0240] According to one embodiment, when the electronic device receives a user input (1820) for selecting an image to be applied to the main layer on a screen for modifying the main layer among the generated images, the electronic device may display a plurality of images (1830) stored in a memory (e.g., the memory (130) of FIG. 1).

[0241] In one embodiment, when a user input is received to select one image (1831) from among multiple images (1830), the electronic device may display a preview (1840) of an image generated based on the selected image (1831). In one embodiment, the generated image may be a main layer, a sub layer, and a background layer overlapping each other based on the selected image (1831) and the conversation content.

[0242] According to one embodiment, when a user input (1841) is received to transmit a generated image (1840) through the execution screen of a message application, the electronic device may display the generated image (1850) in an area of ​​the execution screen of the message application.

[0243] FIG. 19A is a diagram for explaining an operation of generating an image based on an image captured or stored through a camera application of an electronic device according to one embodiment.

[0244] Referring to FIG. 19A, an electronic device (e.g., the electronic device (101) of FIG. 1 or the processor (120) of FIG. 1) may acquire an image based on a user input (1911) for capturing a preview screen (1910) displayed through an execution screen of a camera application. According to one embodiment, the electronic device may display the captured image (1930) and receive a user input (1930) for generating a sticker image using a generative AI based on the captured image (1931).

[0245] According to one embodiment, when an electronic device selects an image through the execution screen of a gallery application in which multiple images are stored, the electronic device may enlarge and display the selected image (1920) and display multiple function icons related to the selected image (1920). According to one embodiment, when a user input (1921) for selecting an icon for editing the selected image (1920) among the multiple function icons is received, the electronic device may display the selected image (1930) and receive a user input (1931) for generating a sticker image using a generative AI based on the selected image (1920). According to one embodiment, an operation after receiving a user input (1931) for generating a sticker image using a generative AI will be described in more detail below with reference to FIG. 19b.

[0246] FIG. 19b is a diagram for explaining an operation of generating an image based on an image captured or stored through a camera application of an electronic device according to one embodiment.

[0247] Referring to FIG. 19B, the electronic device may display a preview (1940) of a sticker image generated based on a captured or selected image. According to an embodiment, the electronic device may display a list (1941) for modifying the effect of each layer of the generated sticker image. According to an embodiment, the list (1941) displays multiple images with different effects applied to objects in each layer, and a desired image to be applied among the multiple images may be selected through a swipe motion. According to an embodiment, when the effect is changed for each layer, the changed effect may be reflected in real time in the preview (1940) of the generated sticker image.

[0248] According to one embodiment, when a user input (1942) for generating a sticker image with the selected effect applied is received after selection of an effect for each layer is completed, the electronic device may display a preview (1950) of the final generated image. According to one embodiment, when a user input (1951) for terminating the image generation operation is received, the electronic device may store the generated image in a storage as a sticker image (1960) related to the captured or selected image.

[0249] According to one embodiment, when a user input (1971) for confirming a sticker image is received through the execution screen (1970) of the message application, the electronic device may display a sticker image (1972) stored in the storage. According to one embodiment, the electronic device may transmit a sticker image (1972) created through the camera application or the gallery application to the conversation partner through the execution screen (1970) of the message application.

[0250] According to one embodiment, an electronic device (e.g., electronic device (101) of FIG. 1) may include a memory (e.g., memory (130) of FIG. 1), a display (e.g., display (160) of FIG. 1), and a processor (e.g., processor (120) of FIG. 1).

[0251] According to one embodiment, the memory may store instructions that, when executed by the processor, cause the electronic device to perform the following operations.

[0252] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to display an execution screen of a message application.

[0253] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to extract at least one keyword text based on conversation information displayed through the execution screen.

[0254] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to identify an image corresponding to at least one keyword text among a plurality of images stored in the memory based on a user input received for generating an image displayable on the execution screen.

[0255] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to obtain at least one first layer based on the identified image.

[0256] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to obtain at least one second layer based on at least one of the conversation information, the at least one keyword text, or information related to the identified image.

[0257] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to generate the image by overlapping the at least one first layer and the at least one second layer.

[0258] According to one embodiment, the information related to the identified image may include at least one of image analysis information of the identified image or meta information of the identified image.

[0259] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to determine the number of the at least one second layer based on whether at least one of the image analysis information or the meta information is acquired.

[0260] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to determine whether to use at least one of an object included in the identified image or a background included in the identified image based on at least one of the conversation information, the at least one keyword text, image analysis information of the identified image, or meta information of the identified image.

[0261] In one embodiment, the instructions, when executed by the processor, may cause the electronic device to generate a main layer including the object based on a determination to use the object.

[0262] In one embodiment, the instructions, when executed by the processor, may cause the electronic device to generate a background layer including the background based on a determination to use the background.

[0263] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to generate at least one of the at least one first layer or the at least one second layer using generative AI.

[0264] In one embodiment, the image may be a sticker image.

[0265] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to generate a prompt to be input to the generative AI based on at least one of the conversation information, the at least one keyword text, image analysis information of the identified image, or meta information of the identified image.

[0266] In one embodiment, the prompt may be generated taking into account whether the sticker image includes a motion effect.

[0267] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to obtain at least one of the at least one first layer or the at least one second layer as output data of the generative AI, using the prompt as input data of the generative AI.

[0268] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to identify an image selected by a user input from among the plurality of images stored in the memory as the image corresponding to the at least one keyword text.

[0269] In one embodiment, the instructions, when executed by the processor, may cause the electronic device to capture an image via a camera application.

[0270] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to identify the captured image as the image corresponding to the at least one keyword text.

[0271] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to determine a color of an object included in the at least one second layer based on color information of the identified image.

[0272] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to display, based on a user input for displaying the generated image, a layer used as a background layer of the generated image among the at least one first layer and the at least one second layer, on the entire area of ​​the execution screen.

[0273] In one embodiment, the instructions, when executed by the processor, may cause the electronic device to display the generated image.

[0274] In one embodiment, the instructions, when executed by the processor, may cause the electronic device to modify the generated image based on user input.

[0275] According to one embodiment, a method of controlling an electronic device may include an action of displaying an execution screen of a message application.

[0276] According to one embodiment, a method for controlling an electronic device may include an operation of extracting at least one keyword text based on conversation information having a history displayed through the execution screen.

[0277] According to one embodiment, a method for controlling an electronic device may include an operation of identifying an image corresponding to at least one keyword text among a plurality of images stored in a memory of the electronic device based on receiving a user input for generating an image displayable on the execution screen.

[0278] According to one embodiment, a method of controlling an electronic device may include an operation of obtaining at least one first layer based on the identified image.

[0279] According to one embodiment, a method for controlling an electronic device may include an operation of obtaining at least one second layer based on at least one of the conversation information, the at least one keyword text, or information related to the identified image.

[0280] According to one embodiment, a method of controlling an electronic device may include an operation of generating the image by overlapping the at least one first layer and the at least one second layer.

[0281] According to one embodiment, the information related to the identified image may include at least one of image analysis information of the identified image or meta information of the identified image.

[0282] According to one embodiment, the operation of obtaining the second layer may further include an operation of determining the number of the at least one second layer based on whether at least one of the image analysis information or the meta information is obtained.

[0283] According to one embodiment, the operation of obtaining the at least one first layer may determine whether to use at least one of an object included in the identified image or a background included in the identified image based on at least one of the conversation information, the at least one keyword text, image analysis information of the identified image, or meta information of the identified image.

[0284] In one embodiment, the act of obtaining at least one first layer may generate a main layer including the object based on a determination that the object is to be used.

[0285] In one embodiment, the operation of obtaining at least one first layer may generate a background layer including the background based on a determination that the background is to be used.

[0286] In one embodiment, at least one of the operations of obtaining the at least one first layer or the operations of obtaining the at least one second layer may generate at least one of the at least one first layer or the at least one second layer using generative AI.

[0287] In one embodiment, the image may be a sticker image.

[0288] In one embodiment, at least one of the operations of obtaining the at least one first layer or the operations of obtaining the at least one second layer may generate a prompt to be input to the generative AI based on at least one of the conversation information, the at least one keyword text, image analysis information of the identified image, or meta information of the identified image.

[0289] In one embodiment, the prompt may be generated taking into account whether the sticker image includes a motion effect.

[0290] In one embodiment, at least one of the operations of obtaining the at least one first layer or the operations of obtaining the at least one second layer may be performed by using the prompt as input data of the generative AI to obtain at least one of the at least one first layer or the at least one second layer as output data of the generative AI.

[0291] According to one embodiment, the operation of confirming the image may confirm an image selected by a user input from among the plurality of images stored in the memory as the image corresponding to the at least one keyword text.

[0292] In one embodiment, the action of checking the image may include capturing the image via a camera application.

[0293] According to one embodiment, the operation of verifying the image may verify the captured image as an image corresponding to the at least one keyword text.

[0294] According to one embodiment, the operation of obtaining the at least one second layer may determine a color of an object included in the at least one second layer based on color information of the identified image.

[0295] According to one embodiment, the control method of the electronic device may further include an operation of displaying a layer used as a background layer of the generated image among the at least one first layer and the at least one second layer, in the entire area of ​​the execution screen, based on a user input for displaying the generated image.

[0296] According to one embodiment, the control method of the electronic device may further include an operation of displaying the generated image.

[0297] According to one embodiment, the control method of the electronic device may further include an operation of modifying the generated image based on a user input.

[0298] According to one embodiment, a non-transitory computer-readable recording medium storing one or more programs may include instructions that cause an electronic device to display an execution screen of a message application.

[0299] According to one embodiment, the one or more programs may include instructions that cause the electronic device to extract at least one keyword text based on conversation information displayed through the execution screen.

[0300] According to one embodiment, the one or more programs may include instructions that cause the electronic device to identify an image corresponding to the at least one keyword text among a plurality of images stored in the memory based on receiving a user input for generating an image displayable on the execution screen.

[0301] In one embodiment, the one or more programs may include instructions that cause the electronic device to obtain at least one first layer based on the identified image.

[0302] According to one embodiment, the one or more programs may include instructions that cause the electronic device to obtain at least one second layer based on at least one of the conversation information, the at least one keyword text, or information related to the identified image.

[0303] In one embodiment, the one or more programs may include instructions that cause the electronic device to generate the image by overlapping the at least one first layer and the at least one second layer.

[0304] According to one embodiment, the information related to the identified image may include at least one of image analysis information of the identified image or meta information of the identified image.

[0305] According to one embodiment, the one or more programs may include instructions that, when executed by the processor, cause the electronic device to determine the number of the at least one second layer based on whether at least one of the image analysis information or the meta information is acquired.

[0306] According to one embodiment, the one or more programs may include instructions that, when executed by the processor, cause the electronic device to determine whether to use at least one of an object included in the identified image or a background included in the identified image based on at least one of the conversation information, the at least one keyword text, image analysis information of the identified image, or meta information of the identified image.

[0307] In one embodiment, the one or more programs may include instructions that, when executed by the processor, cause the electronic device to create a main layer including the object based on a determination that the electronic device uses the object.

[0308] In one embodiment, the one or more programs may include instructions that, when executed by the processor, cause the electronic device to generate a background layer including the background based on a determination to use the background.

[0309] According to one embodiment, the one or more programs may include instructions that, when executed by the processor, cause the electronic device to generate at least one of the at least one first layer or the at least one second layer using generative AI.

[0310] In one embodiment, the image may be a sticker image.

[0311] According to one embodiment, the one or more programs may include instructions that, when executed by the processor, cause the electronic device to generate a prompt to be input to the generative AI based on at least one of the conversation information, the at least one keyword text, image analysis information of the identified image, or meta information of the identified image.

[0312] In one embodiment, the prompt may be generated taking into account whether the sticker image includes a motion effect.

[0313] According to one embodiment, the one or more programs may include instructions that, when executed by the processor, cause the electronic device to obtain at least one of the at least one first layer or the at least one second layer as output data of the generative AI, using the prompt as input data of the generative AI.

[0314] According to one embodiment, the one or more programs may include instructions that, when executed by the processor, cause the electronic device to identify an image selected by a user input from among the plurality of images stored in the memory as the image corresponding to the at least one keyword text.

[0315] According to one embodiment, the one or more programs may include instructions that, when executed by the processor, cause the electronic device to capture an image via a camera application.

[0316] According to one embodiment, the one or more programs may include instructions that, when executed by the processor, cause the electronic device to identify the captured image as the image corresponding to the at least one keyword text.

[0317] According to one embodiment, the one or more programs may include instructions that, when executed by the processor, cause the electronic device to determine a color of an object included in the at least one second layer based on color information of the identified image.

[0318] According to one embodiment, the one or more programs may include instructions that, when executed by the processor, cause the electronic device to display, based on a user input for displaying the generated image, a layer used as a background layer of the generated image among the at least one first layer and the at least one second layer, on the entire area of ​​the execution screen.

[0319] According to one embodiment, the one or more programs may include instructions that, when executed by the processor, cause the electronic device to display the generated image.

[0320] According to one embodiment, the one or more programs may include instructions that, when executed by the processor, cause the electronic device to modify the generated image based on user input.

[0321] Electronic devices according to the embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments disclosed in this document are not limited to the aforementioned devices.

[0322] The embodiments of this document and the terminology used herein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0323] The term "module" used in the embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0324] One embodiment of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0325] According to one embodiment, the method according to one embodiment disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0326] According to one embodiment, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to one embodiment, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to one embodiment, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In electronic devices, Memory that stores one or more computer programs; display; and one or more processors communicatively connected to the memory and the display; The one or more computer programs, when individually or collectively executed by the one or more processors, cause the electronic device to: Display the launch screen of the messaging application, Based on the conversation information with the history displayed through the above execution screen, extract at least one keyword text, Based on receiving a user input for generating an image that can be displayed on the execution screen, an image corresponding to at least one keyword text among a plurality of images stored in the memory is identified, Obtain at least one first layer based on the above-mentioned confirmed image, Obtaining at least one second layer based on at least one of the above conversation information, the at least one keyword text, or the information related to the identified image, An electronic device storing computer-executable instructions that operate to generate the image by overlapping at least one first layer and at least one second layer.

2. In paragraph 1, Information related to the above confirmed image is: Contains at least one of image analysis information of the above-mentioned confirmed image or meta information of the above-mentioned confirmed image, The one or more computer programs, when executed individually or collectively by the one or more processors, cause the electronic device to: An electronic device that determines the number of at least one second layer based on whether at least one of the image analysis information or the meta information is acquired.

3. In paragraph 1 or 2, The one or more computer programs, when executed individually or collectively by the one or more processors, cause the electronic device to: Based on at least one of the above conversation information, the at least one keyword text, the image analysis information of the identified image, or the meta information of the identified image, determine whether to use at least one of the objects included in the identified image or the background included in the identified image; Based on the decision to use the above object, create a main layer including the above object, An electronic device that generates a background layer including the background based on a determination to use the background.

4. In any one of paragraphs 1 to 3, The one or more computer programs, when executed individually or collectively by the one or more processors, cause the electronic device to: An electronic device that generates at least one of the at least one first layer or the at least one second layer using generative AI.

5. In paragraph 4, The image above is a sticker image, The one or more computer programs, when executed individually or collectively by the one or more processors, cause the electronic device to: Generate a prompt to be input to the generative AI based on at least one of the above conversation information, the at least one keyword text, the image analysis information of the image, or the meta information of the image, By using the above prompt as input data of the generative AI, at least one of the at least one first layer or the at least one second layer is obtained as output data of the generative AI, The above prompt is, An electronic device generated by taking into account whether or not to include a motion effect in the above sticker image.

6. In any one of paragraphs 1 to 5, The one or more computer programs, when executed individually or collectively by the one or more processors, cause the electronic device to: An electronic device that identifies an image selected by user input from among the plurality of images stored in the memory as the image corresponding to at least one keyword text.

7. In any one of paragraphs 1 to 5, The one or more computer programs, when executed individually or collectively by the one or more processors, cause the electronic device to: Capture images through the camera application, An electronic device that identifies the captured image as an image corresponding to at least one keyword text.

8. In any one of paragraphs 1 to 7, The one or more computer programs, when executed individually or collectively by the one or more processors, cause the electronic device to: An electronic device that determines the color of an object included in at least one second layer based on color information of the above-determined image.

9. In any one of paragraphs 1 to 8, The one or more computer programs, when executed individually or collectively by the one or more processors, cause the electronic device to: An electronic device that displays a layer used as a background layer of the generated image among the at least one first layer and the at least one second layer in the entire area of ​​the execution screen based on a user input for displaying the generated image.

10. In any one of paragraphs 1 to 9, The one or more computer programs, when executed individually or collectively by the one or more processors, cause the electronic device to: Display the image generated above, An electronic device that modifies the generated image based on user input.

11. In a method for controlling an electronic device, An action that displays the launch screen of the messaging application; An action of extracting at least one keyword text based on conversation information with a history displayed through the above execution screen; An operation of identifying an image corresponding to at least one keyword text among a plurality of images stored in a memory of the electronic device based on receiving a user input for generating an image displayable on the execution screen; An operation of obtaining at least one first layer based on the above-determined image; An operation of obtaining at least one second layer based on at least one of the above conversation information, the at least one keyword text, or information related to the identified image; and A control method of an electronic device, comprising: an operation of generating the image by overlapping at least one first layer and at least one second layer.

12. In paragraph 11, Information related to the above confirmed image is: Contains at least one of image analysis information of the above-mentioned confirmed image or meta information of the above-mentioned confirmed image, The operation of obtaining the above second layer is: A control method of an electronic device further comprising: an operation of determining the number of at least one second layer based on whether at least one of the image analysis information or the meta information is acquired; 13. In paragraph 11 or 12, The operation of obtaining at least one first layer, Based on at least one of the above conversation information, the at least one keyword text, the image analysis information of the identified image, or the meta information of the identified image, determine whether to use at least one of the objects included in the identified image or the background included in the identified image; Based on the decision to use the above object, create a main layer including the above object, A method of controlling an electronic device for generating a background layer including the background based on a determination to use the background.

14. In any one of paragraphs 11 to 13, At least one of the operations of obtaining at least one first layer or the operations of obtaining at least one second layer, Generating at least one of the at least one first layer or the at least one second layer using generative AI, The image above is a sticker image, At least one of the operations of obtaining at least one first layer or the operations of obtaining at least one second layer, Generate a prompt to be input to the generative AI based on at least one of the above conversation information, the at least one keyword text, the image analysis information of the identified image, or the meta information of the identified image, Using the above prompt as input data of the generative AI, at least one of the at least one first layer or the at least one second layer is obtained as output data of the generative AI, The above prompt is, A control method of an electronic device, wherein the control method is generated by taking into account whether or not to include a motion effect in the above sticker image.

15. A non-transitory computer-readable recording medium storing one or more programs, wherein the one or more programs are: Display the launch screen of the messaging application, Based on the conversation information displayed through the above execution screen, extract at least one keyword text, Based on receiving a user input for generating an image that can be displayed on the execution screen, an image corresponding to at least one keyword text is identified among a plurality of images stored in the memory of the electronic device, Obtain at least one first layer based on the above-mentioned confirmed image, Obtaining at least one second layer based on at least one of the above conversation information, the at least one keyword text, or the information related to the identified image, A recording medium comprising instructions for generating the image by overlapping at least one first layer and at least one second layer.

Images