Electronic device for applying effect to image, operating method thereof, and recording medium

Abstract

This electronic device may comprise a display, a memory and at least one processor. The memory can store instructions that, when executed by the at least one processor, cause the electronic device to: provide a first image to a first artificial intelligence model so as to acquire a first text indicating information related to the first image when a first input for editing the first image stored in the memory is identified; display, through the display, a plurality of first guides corresponding to a plurality of first effects that can be applied to the first image; identify, on the basis of a second input for selecting at least one guide from among the plurality of first guides, a second text indicating at least one effect corresponding to the at least one guide; and provide the first image, the first text and the second text to a second artificial intelligence model so as to acquire a second image in which the at least one effect is applied to the first image. Other various embodiments are possible.

Description

Electronic device for applying effects to images, method of operation thereof, and recording medium

[0001] Embodiments of the present disclosure relate to an electronic device for applying effects to an image, a method of operating the same, and a recording medium.

[0002] Thanks to remarkable advancements in information and communication technology and semiconductor technology, the proliferation and use of various electronic devices is rapidly increasing. Electronic devices are being developed to enable users to carry and communicate with one another. An electronic device can refer to any device that performs a specific function based on its embedded software, such as a mobile communication terminal, tablet PC, audio / video device, desktop / laptop computer, or in-car navigation system.

[0003] Recently, users have become increasingly interested in acquiring high-quality images, beyond simply capturing them using electronic devices. Electronic devices offer image editing capabilities. Using image editing applications, electronic devices can provide users with an environment where they can edit images stored on their devices.

[0004] According to one embodiment, an electronic device may include a display, memory, and at least one processor.

[0005] According to one embodiment, the memory may store instructions that, when individually or collectively executed by the at least one processor, cause the electronic device to, when a first input for editing a first image stored in the memory is identified, provide the first image to a first artificial intelligence model to obtain a first text representing information related to the first image.

[0006] According to one embodiment, the memory may store instructions that, when individually or collectively executed by the at least one processor, cause the electronic device to display, through the display, a plurality of first guides corresponding to a plurality of first effects applicable to the first image.

[0007] According to one embodiment, the memory may store instructions that, when individually or collectively executed by the at least one processor, cause the electronic device to, based on a second input selecting at least one guide from the plurality of first guides, identify a second text indicating at least one effect corresponding to the at least one guide.

[0008] According to one embodiment, the memory may store instructions that, when individually or collectively executed by the at least one processor, cause the electronic device to provide the first image, the first text, and the second text to a second artificial intelligence model to obtain a second image having the at least one effect applied to the first image.

[0009] According to one embodiment, a method of operating an electronic device may include, when a first input for editing a first image stored in a memory of the electronic device is confirmed, providing the first image to a first artificial intelligence model to obtain a first text representing information related to the first image.

[0010] According to one embodiment, a method of operating an electronic device may include displaying, through a display of the electronic device, a plurality of first guides corresponding to a plurality of first effects that can be applied to the first image.

[0011] According to one embodiment, a method of operating an electronic device may include an operation of identifying a second text indicating at least one effect corresponding to at least one guide based on a second input selecting at least one guide from among the plurality of first guides.

[0012] According to one embodiment, a method of operating an electronic device may include providing the first image, the first text, and the second text to a second artificial intelligence model to obtain a second image having the at least one effect applied to the first image.

[0013] According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor of an electronic device, cause the electronic device to perform at least one operation, wherein the at least one operation may include, when a first input for editing a first image stored in a memory of the electronic device is confirmed, providing the first image to a first artificial intelligence model to obtain a first text representing information related to the first image.

[0014] According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor of an electronic device, cause the electronic device to perform at least one operation, wherein the at least one operation may include displaying, through a display of the electronic device, a plurality of first guides corresponding to a plurality of first effects that can be applied to the first image.

[0015] According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor of an electronic device, cause the electronic device to perform at least one operation, wherein the at least one operation may include an operation of identifying a second text indicating at least one effect corresponding to at least one guide based on a second input selecting at least one guide from among the plurality of first guides.

[0016] According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor of an electronic device, cause the electronic device to perform at least one operation, wherein the at least one operation may include providing the first image, the first text, and the second text to a second artificial intelligence model to obtain a second image having the at least one effect applied to the first image.

[0017] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0018] FIG. 2 is a schematic block diagram of an electronic device according to one embodiment.

[0019] FIG. 3A is a schematic block diagram of an image generation module according to one embodiment.

[0020] FIG. 3b is a diagram illustrating an operation of an electronic device applying an effect to a first image according to one embodiment.

[0021] FIG. 4A is a flowchart illustrating an operation of an electronic device editing an image according to one embodiment.

[0022] FIG. 4b is a flowchart illustrating an operation of an electronic device applying an effect to an image using an artificial intelligence model, according to one embodiment.

[0023] FIG. 5 is a diagram for explaining a template and a first text input to a first artificial intelligence model according to one embodiment.

[0024] FIG. 6 is a drawing for explaining a screen for editing an image by an electronic device according to one embodiment.

[0025] FIG. 7 is a diagram illustrating an operation of an electronic device obtaining text information for editing an image, according to one embodiment.

[0026] FIG. 8 is a diagram for explaining an operation of an electronic device to obtain an image with a bokeh effect applied, according to one embodiment.

[0027] FIG. 9A is a diagram illustrating an operation of an electronic device obtaining text information for editing an image, according to one embodiment.

[0028] FIG. 9b is a diagram illustrating an operation of an electronic device to obtain an image with a deblur effect applied, according to one embodiment.

[0029] FIG. 10A is a diagram illustrating an operation of an electronic device obtaining text information for editing an image, according to one embodiment.

[0030] FIG. 10b is a diagram for explaining an operation of an electronic device to obtain an image with an HDR effect applied, according to one embodiment.

[0031] FIG. 11A is a diagram illustrating an operation of an electronic device obtaining text information for editing an image, according to one embodiment.

[0032] FIG. 11b is a diagram illustrating an operation of an electronic device to obtain an image with a black and white effect applied, according to one embodiment.

[0033] FIG. 12 is a flowchart illustrating an operation of an electronic device editing a sketch according to one embodiment.

[0034] FIG. 13A is a diagram illustrating an operation of an electronic device to obtain text information for editing a sketch, according to one embodiment.

[0035] FIG. 13b is a diagram illustrating an operation of an electronic device to obtain an image with a pop art effect applied, according to one embodiment.

[0036] FIG. 14A is a diagram illustrating an operation of an electronic device to obtain text information for editing a third image, according to one embodiment.

[0037] FIG. 14b is a diagram illustrating an operation of an electronic device to obtain an image with a sketch effect applied to a third image, according to one embodiment.

[0038] FIG. 15 is a flowchart illustrating an operation of an electronic device editing a sketch according to one embodiment.

[0039] FIG. 16 is a diagram illustrating an example of a text command that indicates an effect corresponding to a guide provided to an artificial intelligence model, according to one embodiment.

[0040] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with the electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (104) or the server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0041] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0042] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, in the electronic device (101) itself where artificial intelligence is performed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0043] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0044] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0045] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0046] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0047] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0048] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0049] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0050] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0051] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0052] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0053] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0054] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0055] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0056] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0057] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0058] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas by, for example, the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0059] In one embodiment, the antenna module (197) may generate a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

[0060] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0061] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0062] FIG. 2 is a schematic block diagram of an electronic device according to one embodiment.

[0063] Referring to FIG. 2, according to one embodiment, an electronic device (201) (e.g., the electronic device (101) of FIG. 1) may include a memory (210) (e.g., the memory (130) of FIG. 1), a processor (220) (e.g., the processor (120) of FIG. 1), and a display (260) (e.g., the display (160) of FIG. 1). According to one embodiment, the electronic device (201) may be implemented in the same or similar manner as the electronic device (101) of FIG. 1.

[0064] According to one embodiment, the processor (220) can control the overall operation of the electronic device (201). According to one embodiment, the processor (220) can be implemented identically or similarly to the processor (120) of FIG. 1.

[0065] According to one embodiment, the processor (220) may display a screen for editing an image stored in the memory (210) through the display (260). According to one embodiment, the image may include an image captured by the electronic device (201) or an image acquired from an external electronic device. According to one embodiment, the screen for editing the image may include an execution screen of an application for editing the image.

[0066] According to one embodiment, the processor (220) may display a screen for obtaining a sketch (or drawing) by the user's body or stylus pen through the display (260). For example, the sketch may be obtained based on a plurality of strokes based on a touch input. For example, a stroke may represent a continuous stroke from a point where a touch input is started to a point where the touch input is released. According to one embodiment, the screen for obtaining the sketch (or drawing) may include an execution screen of an application for obtaining the sketch (e.g., a memo-related application, a drawing-related application). According to one embodiment, the processor (220) may display a screen for editing the obtained sketch (or drawing) through the display (260). For example, the sketch may include at least one of a person, an animal, an object, a plant, or a natural environment.

[0067] According to one embodiment, the processor (220) can apply a specific effect to a corresponding image through a screen for editing the image.

[0068] According to one embodiment, the processor (220) may acquire a sketch based on touch input through a screen for acquiring a sketch (or drawing). According to one embodiment, the processor (220) may apply a specific effect to the sketch (or drawing) through a screen for editing the acquired sketch (or drawing).

[0069] According to one embodiment, a particular effect may include an effect that changes at least one of the shape, background, color, brightness, or composition of an object (e.g., a subject) of a corresponding image (or sketch).

[0070] In one embodiment, a specific effect may include an effect that changes the image quality of a corresponding image (or sketch). For example, the specific effect may include at least one of a high dynamic range (HDR) effect, a bokeh effect, a blur effect for the background, an effect that changes the image to black and white, and a deblur effect. However, this is merely an example, and the specific effect may include various effects.

[0071] In one embodiment, a specific effect may include a graphic effect that changes the style of an image (or sketch), the style of the image (or sketch), or the presentation technique of the image (or sketch). For example, the specific effect may include an illustration effect, a watercolor effect, a sketch effect, a pop art effect, and a 3D cartoon effect. However, this is only an example, and the specific effect may include various effects.

[0072] For example, an illustration effect may include an effect that changes an image into an illustration style. For example, a caricature effect may include an effect that changes an image into an image sketched with a pencil. For example, a watercolor effect may include an effect using a watercolor technique that changes an image into an image that looks like it was drawn with watercolor paints. For example, a 3D character effect may include an effect that changes at least one object (e.g., a person, an animal) included in an image into a three-dimensional character. For example, a pop art effect may include an effect using a pop art technique that changes the colors of an image into the colors of acrylic paint and changes at least one object included in the image into a three-dimensional shape.

[0073] According to one embodiment, the processor (220) may apply a specific effect to an image (or sketch) using a pre-trained artificial intelligence (AI) model stored in the memory (210) (e.g., the memory (130) of FIG. 1). The processor (220) may also apply a specific effect to an image (or sketch) using a pre-trained AI model stored in a separate server or a separate external electronic device. The processor (220) may input a prompt text command (or text prompt) for applying a specific effect to the image (or sketch) to the AI ​​model. For example, the text command may include text or a command that causes the AI ​​model to output an image (or sketch) with a specific effect applied to the image.

[0074] Previously, to obtain an image (or sketch) with specific effects applied to it using an AI model, users had to manually enter text commands. However, because the text commands entered into the AI ​​model had to be written according to predefined rules, users had difficulty entering them directly.

[0075] According to one embodiment, the electronic device (201) may pre-store text commands that can apply specific effects to an image (or sketch) using an AI model, and provide the stored text commands in the form of a guide that is easy for the user to understand. This allows the user to easily apply specific effects to an image (or sketch) by selecting the guide provided by the electronic device (201).

[0076] According to one embodiment, the processor (220) can confirm a user input for editing a first image stored in the memory (210) through a screen for editing an image. For example, the user input for editing the first image may represent a user input for selecting an image to be edited.

[0077] According to one embodiment, when a user input for editing a first image is confirmed, the processor (220) may provide the first image to an image generation module (300) stored in the memory (210) (e.g., the image generation module (300) of FIG. 3A) to obtain first text (or text command) indicating information related to the first image. According to one embodiment, the first text indicating information related to the first image may include text based on at least one of the resolution of the first image, at least one object (e.g., subject) included in the first image, background, color, brightness, or composition.

[0078] According to one embodiment, the processor (220) may further provide a template including a query together with the first image to the image generation module (300). According to one embodiment, the image generation module (300) may output a first text indicating information related to the first image based on the template. According to one embodiment, the image generation module (300) may output a result of analyzing the first image based on the query included in the template as a first text indicating information related to the first image. According to one embodiment, the first text may include an answer corresponding to the query. Through this, the electronic device (201) according to one embodiment may not acquire unnecessary information about the first image, thereby reducing resources for the first text.

[0079] According to one embodiment, the processor (220) may obtain a sketch based on a user's touch input through a screen for obtaining a sketch (or drawing). According to one embodiment, when a user input for editing the obtained sketch (or drawing) is confirmed, the processor (220) may input the sketch into the image generation module (300) to obtain third text (or text command) indicating information related to the sketch. According to one embodiment, the processor (220) may further provide a template including a query together with the sketch to the image generation module (300). According to one embodiment, the third text indicating information related to the sketch may include text based on at least one of the resolution of the sketch, at least one object (e.g., subject) included in the sketch, background, color, brightness, or composition.

[0080] According to one embodiment, the processor (220) may display a plurality of guides corresponding to a plurality of effects that can be applied to an image (or sketch) stored in the memory (210) via the display (260). According to one embodiment, the plurality of guides may include texts indicating the plurality of effects or designated images indicating the plurality of effects. For example, the designated images may represent designated images to which the plurality of effects have been applied.

[0081] According to one embodiment, the plurality of effects may include effects that change the image quality of the image (or sketch) and / or effects that change the style of the image (or sketch), the drawing style of the image (or sketch), or the expression technique of the image (or sketch). According to one embodiment, the plurality of effects may include a high dynamic range (HDR) effect, a bokeh effect, a blur effect for the background, an effect that changes to a black and white image, and a deblur effect. According to one embodiment, the plurality of effects may include an illustration effect, a watercolor effect, a sketch effect, a pop art effect, and a 3D cartoon effect. However, this is an example, and the plurality of effects may not be limited to the above examples. For example, the plurality of guides may include text indicating an HDR effect, text indicating a bokeh effect, text indicating a blur effect, text indicating a black and white effect, and text indicating a deblur effect. For example, multiple guides may include text representing illustration effects, watercolor effects, sketch effects, pop art effects, and 3D cartoon effects.

[0082] According to one embodiment, the processor (220) may identify a plurality of first effects based on information related to the first image (sketch) among a plurality of effects that can be applied to the image. According to one embodiment, the processor (220) may identify a plurality of first effects corresponding to information related to the first image (sketch) among the plurality of effects. According to one embodiment, the plurality of first effects may include pre-designated effects corresponding to at least one of the resolution of the first image (sketch), at least one object included in the first image (sketch), background, color, brightness, or composition among the plurality of effects. For example, when the resolution of the first image (sketch) is lower than the designated resolution, the processor (220) may identify an HDR effect among the plurality of effects. For example, when the object included in the first image (sketch) is a person, the processor (220) may identify a blur effect among the plurality of effects. For example, when the representative color of the first image (sketch) is identified as a specific color, the processor (220) may identify a black and white effect. The representative color may include a color corresponding to the average value, maximum value, or minimum value of pixels included in multiple areas of the first image (sketch). For example, if the first image is determined to be a shaken image, the processor (220) may check for a deblur effect.

[0083] According to one embodiment, the processor (220) may verify a user input for selecting at least one guide from among a plurality of guides. According to one embodiment, the processor (220) may verify second text indicating at least one effect corresponding to at least one guide selected by the user input. For example, text indicating an HDR effect may indicate “HDR image, High contrast”. For example, text indicating a bokeh effect may indicate “Out-focused image, DSLR, 100mm, macro”. For example, text indicating a blur effect may indicate “Blurred image, background”. For example, text indicating a black and white effect may indicate “Black and white image, entire”. For example, text indicating a deblur effect may indicate “De-blurred image, super clear, 4k”. However, this is an example, and texts indicating at least one effect may not be limited to the above examples.

[0084] Depending on the implementation, according to one embodiment, a text indicating an effect corresponding to a single guide may be implemented as a plurality of texts. For example, the plurality of texts indicating a caricature effect may include “a style sketched using a cross-hatching technique using a 6B pencil,” “a style sketched using a cross-hatching technique using a 3B pencil,” and “a style sketched using a cross-hatching technique using a 2B pencil.” According to one embodiment, the processor (220) may select one text (e.g., a second text) from among the plurality of texts based on the first image or the first text. For example, based on an attribute of an object included in the first image (e.g., a female person, a male person, a young child, a young person, an elderly person, etc.), a text assigned to an attribute of the object may be selected from among the plurality of texts.

[0085] According to one embodiment, the processor (220) may input a first image, a first text (text command), and a second text (text command) into the image generation module (300), thereby obtaining a second image having at least one effect applied to the first image. For example, the image generation module (300) may include at least one AI model.

[0086] According to one embodiment, the processor (220) inputs a first image, a first text, and a second text into the image generation module (300), thereby obtaining a plurality of images having at least one effect applied to the first image.

[0087] For example, the processor (220) may obtain a second text based on a user input that selects a first guide (e.g., a guide corresponding to a bokeh effect) from among a plurality of guides. The processor (220) may input a first image, a first text, and a second text into the image generation module (300), thereby obtaining a plurality of images in which a first effect (e.g., a bokeh effect) corresponding to the second text is applied to the first image. The number of the plurality of images (e.g., four) may be set by the user or may be automatically set by the processor (220). The plurality of images may differ from each other in at least one of shape, color, or composition.

[0088] For example, the processor (220) may obtain a second text based on a user input that selects a first guide (e.g., a guide corresponding to a bokeh effect) and a second guide (e.g., a guide corresponding to a pop art effect) from among a plurality of guides. The processor (220) may input the first image, the first text, and the second text into the image generation module (300), thereby obtaining a plurality of first images in which a first effect (e.g., a bokeh effect) corresponding to the first guide is applied to the first image, and a plurality of second images in which a second effect (e.g., a pop art effect) corresponding to the second guide is applied to the first image. The number of the plurality of first images may be four, and the number of the plurality of second images may also be four. Here, the number of the plurality of images may be differently designated for each effect corresponding to the guide.

[0089] According to one embodiment, the processor (220) may display a plurality of images on the display (260). According to one embodiment, when a user input for selecting at least one image among the plurality of images is confirmed, the processor (220) may store the at least one image in the memory (210). According to one embodiment, images other than at least one image among the plurality of images may be discarded.

[0090] According to one embodiment, the processor (220) may combine the first text and the second text. According to one embodiment, the processor (220) may obtain text information for editing the first image based on the combination of the first text and the second text. According to one embodiment, the processor (220) may obtain text information for editing the first image, such as “apply a certain effect to something included in the first image.” According to one embodiment, the processor (220) may obtain information about “what included in the first image” using the first text, and obtain information about “apply a certain effect” using the second text.

[0091] According to one embodiment, the processor (220) may input text information for editing a first image and the first image to an image generation module (300), thereby obtaining a second image having at least one effect applied to the first image.

[0092] According to one embodiment, the processor (220) can display a second image through the display (260).

[0093] According to one embodiment, the processor (220) may input text information for editing a first image and the first image to the image generation module (300), thereby obtaining a plurality of images to which at least one effect is applied to the first image.

[0094] Conventional electronic devices have been able to obtain user input for text corresponding to an effect to be applied to an image to improve image quality. Conventional electronic devices can obtain an image with the effect corresponding to the text applied by inputting the text corresponding to the effect to be applied to the image based on the user input into an artificial intelligence model. The text corresponding to the effect to be applied to the image may include information about the image and text containing the effect to be applied to the image. However, conventional electronic devices have the inconvenience of having to obtain text written by the user, and the text must be written specifically and accurately. If the text containing information about the image and the effect to be applied to the image is written inaccurately or vaguely, conventional electronic devices may obtain a distorted image or an image without the effect applied.

[0095] An electronic device (201) according to one embodiment may input an image to be edited into an artificial intelligence model to obtain text representing information related to the image. The text representing information related to the image may include text representing “what is included in the image.” The electronic device (201) according to one embodiment may display and provide guides for a plurality of pre-designated effects that may be applied to the image to a user. The electronic device (201) according to one embodiment may, based on an input of selecting at least one guide among the guides, check text representing at least one effect corresponding to at least one guide. According to one embodiment, the text representing at least one effect may include text representing “what effect.” The electronic device (201) according to one embodiment may obtain text information representing “what effect is applied to what is included in the image” by combining text representing information related to the image with text representing at least one effect. In other words, the electronic device (201) according to one embodiment may obtain text information using an artificial intelligence model without requiring a user to directly obtain text information. An electronic device (201) according to one embodiment can input an image to be edited and text information into an artificial intelligence model to obtain an image with an effect applied.

[0096] FIG. 3 is a schematic block diagram of an image generation module according to one embodiment.

[0097] Referring to FIG. 3, according to one embodiment, the image generation module (300) may be stored in the memory (210) (e.g., the memory (210) of FIG. 2). According to one embodiment, the image generation module (300) may be implemented as software. Depending on the implementation, at least a portion of the image generation module (300) may be implemented as hardware. According to one embodiment, the image generation module (300) may include a first artificial intelligence model (310) and a second artificial intelligence model (320). The first artificial intelligence model (310) may include a model that acquires information related to an image (e.g., text information). According to one embodiment, the first artificial intelligence model (310) may include an image to segmentation model or a detector model. According to one embodiment, the second artificial intelligence model (320) may include a generative artificial intelligence model. For example, the second artificial intelligence model (320) may include a diffusion model.

[0098] Depending on the implementation, the first artificial intelligence model (310) and the second artificial intelligence model (320) may be implemented as the same artificial intelligence model. Depending on the implementation, the first artificial intelligence model (310) and the second artificial intelligence model (320) may be implemented as a single artificial intelligence model.

[0099] According to one embodiment, the processor (220) (e.g., the processor (220) of FIG. 2) may input a first image to be edited into the first artificial intelligence model (310) to obtain first text representing information related to the first image. According to one embodiment, the first text representing information related to the first image may include text based on at least one of the resolution of the first image, at least one object (e.g., a subject) included in the first image, background, color, brightness, or composition. According to one embodiment, the first artificial intelligence model (310) may include a model that converts an image input to the first artificial intelligence model (310) into text representing information related to the image. Alternatively, the processor (220) may obtain the first text representing information related to the first image based on a preset method for acquiring information related to the first image.

[0100] According to one embodiment, the processor (220) may verify a user input for selecting at least one guide corresponding to at least one effect to be applied to a first image from among a plurality of guides corresponding to a plurality of effects pre-stored in the memory (210). According to one embodiment, the processor (220) may verify a second text indicating at least one effect corresponding to the selected at least one guide.

[0101] According to one embodiment, the processor (220) may obtain text information for editing the first image by combining the first text and the second text. According to one embodiment, the processor (220) may obtain text information for editing the first image, such as “apply a certain effect to something included in the first image.” For example, the processor (220) may obtain information about “what included in the first image” using the first text, and obtain information about “apply a certain effect” using the second text.

[0102] According to one embodiment, the processor (220) may input text information for editing a first image and the first image to a second artificial intelligence model (320), thereby obtaining a second image to which at least one effect is applied to the first image. According to one embodiment, the processor (220) may also input the first text, the second text, and the first image to the second artificial intelligence model (320), thereby obtaining a second image to which at least one effect is applied to the first image.

[0103] In one embodiment, the size of the second image (e.g., 256x256) may be smaller than the size of the first image (e.g., 512x512). In one embodiment, the size of the second image may be greater than or equal to the size of the first image.

[0104] The operations of the electronic device (201) described in the drawings below may be performed by the processor (220). However, for convenience of explanation, the operations performed by the processor (220) will be described as being performed by the electronic device (201).

[0105] FIG. 3b is a diagram illustrating an operation of an electronic device applying an effect to a first image according to one embodiment.

[0106] Referring to FIG. 3B, according to one embodiment, a first artificial intelligence model (310) (e.g., the first artificial intelligence model (310) of FIG. 3A), a text encoder, an image encoder, a second artificial intelligence model (320) (e.g., the second artificial intelligence model (320) of FIG. 3B), and an image decoder may be stored in a memory (210) (e.g., the memory (210) of FIG. 2). Alternatively, the first artificial intelligence model (310), the text encoder, the image encoder, the second artificial intelligence model (320), and the image decoder may be stored in a separate server or external device. According to one embodiment, at least some of the first artificial intelligence model (310), the text encoder, the image encoder, the second artificial intelligence model (320), and the image decoder may be implemented as a hardware configuration or a software configuration.

[0107] According to one embodiment, the first artificial intelligence model (310) may output a first text (332) representing information related to the first image (331) based on being provided with the first image (331).

[0108] According to one embodiment, text (340, 350, 360, 370) representing at least one effect corresponding to at least one guide and a first text (332) may be provided to a text encoder. Alternatively, text information for editing a first image (331) in which text (340, 350, 360, 370) representing at least one effect corresponding to at least one guide and the first text (332) are combined may be provided to the text encoder.

[0109] According to one embodiment, the first image (331) may be provided to an image encoder.

[0110] According to one embodiment, the second artificial intelligence model (320) may be provided with a first image (331) obtained from an image encoder and text information for editing the first image (331) obtained from a text encoder. Alternatively, according to one embodiment, the second artificial intelligence model (320) may be provided with a first image (331) obtained from an image encoder, texts (340, 350, 360, 370) obtained from a text encoder, and a first text (332).

[0111] According to one embodiment, the second artificial intelligence model (320) can output at least one image (341, 351, 361, 371) to which at least one effect is applied via an image decoder.

[0112] According to one embodiment, an image (341) to which a deblur effect is applied may represent an image to which a deblur effect is applied to a first image (331) based on a guide (340) indicating the deblur effect.

[0113] According to one embodiment, the image (351) to which the bokeh effect is applied may represent an image to which the bokeh effect is applied to the first image (331) based on a guide (350) representing the bokeh effect.

[0114] According to one embodiment, an image (361) to which an HDR effect is applied may represent an image to which an HDR effect is applied to a first image (331) based on a guide (360) representing an HDR effect.

[0115] According to one embodiment, an image (371) to which a pop art effect is applied may represent an image to which a pop art effect is applied to a first image (331) based on a guide (370) representing a pop art effect.

[0116] FIG. 4A is a flowchart illustrating an operation of an electronic device editing an image according to one embodiment.

[0117] Referring to FIG. 4A, according to one embodiment, in operation 411, the electronic device (201) (e.g., the electronic device (201) of FIG. 2) may confirm a first input for editing a first image through a screen for editing an image. According to one embodiment, the electronic device (201) may display a screen for editing an image stored in a memory (210) (e.g., the memory (210) of FIG. 2) through a display (260) (e.g., the display (260) of FIG. 2). According to one embodiment, the screen for editing an image may include an execution screen of an application for editing an image. For example, the first input for editing the first image may indicate a user input for selecting an image to be edited.

[0118] According to one embodiment, in operation 413, the electronic device (201) may provide the first image to the first artificial intelligence model (310) (e.g., the first artificial intelligence model (310) of FIG. 3A) to obtain first text representing information related to the first image. According to one embodiment, the first text representing information related to the first image may include text based on at least one of the resolution of the first image, at least one object (e.g., subject) included in the first image, background, color, brightness, or composition. For example, the first text may represent “a girl wearing a pink dress with her head tilted to the left, with a background including a house, a lawn, and a blue sky.” For example, the first artificial intelligence model (310) may be stored on a separate server or a separate device.

[0119] According to one embodiment, in operation 415, the electronic device (201) may display a plurality of first guides corresponding to a plurality of first effects that may be applied to the first image.

[0120] According to one embodiment, the plurality of first effects may include a high dynamic range (HDR) effect, a bokeh effect, a blur effect for the background, an effect that changes to a black and white image, and a deblur effect. For example, the plurality of first guides may include text indicating an HDR effect, text indicating a bokeh effect, text indicating a blur effect, text indicating a black and white effect, and text indicating a deblur effect. However, this is an example, and the plurality of first effects are not limited to the above example and may include various effects. According to one embodiment, the plurality of first guides may include texts indicating the plurality of first effects. According to one embodiment, the plurality of first guides may include images designated for the plurality of first effects. For example, the designated images may represent designated images to which the plurality of first effects are applied.

[0121] According to one embodiment, the plurality of first effects may include predefined effects corresponding to at least one of the resolution of the first image, at least one object included in the first image, background, color, brightness, or composition.

[0122] According to one embodiment, the plurality of first effects may include effects that change the style of the image, the style of the image, or the expression technique of the image. According to one embodiment, the plurality of first effects may include an illustration effect, a watercolor effect, a sketch effect, a pop art effect, and a 3D cartoon effect. However, this is an example, and the plurality of first effects are not limited thereto and may include various effects. For example, the plurality of first guides may include texts or designated images representing an illustration effect, a watercolor effect, a sketch effect, a pop art effect, and a 3D cartoon effect.

[0123] According to one embodiment, in operation 417, the electronic device (201) may identify a second input selecting at least one guide from among a plurality of first guides.

[0124] According to one embodiment, in operation 419, the electronic device (201) may, based on the second input, identify a second text indicating at least one effect corresponding to at least one guide.

[0125] According to one embodiment, in operation 421, the electronic device (201) may provide the first image, the first text, and the second text to the second artificial intelligence model (320) (e.g., the second artificial intelligence model (320) of FIG. 3A) to obtain a second image having at least one effect applied to the first image. According to one embodiment, the second artificial intelligence model (320) may include a diffusion model. According to one embodiment, the electronic device (201) may display the second image through the display (260). For example, the second artificial intelligence model (320) may be stored in a separate server or a separate device.

[0126] FIG. 4b is a flowchart illustrating an operation of an electronic device applying an effect to an image using an artificial intelligence model, according to one embodiment.

[0127] Referring to FIG. 4B, according to one embodiment, in operation 431, the electronic device (201) (e.g., the electronic device (201) of FIG. 2) may provide a first image to a first artificial intelligence model (310) (e.g., the first artificial intelligence model (310) of FIG. 3A) to obtain first text representing information related to the first image. According to one embodiment, the first text representing information related to the first image may include text based on at least one of the resolution of the first image, at least one object (e.g., a subject) included in the first image, a background, a color, brightness, or a composition. For example, the first text may represent “a girl wearing a pink dress with her head tilted to the left, with a background including a house, a lawn, and a blue sky.” For example, the first artificial intelligence model (310) may be stored on a separate server or a separate device.

[0128] According to one embodiment, in operation 433, the electronic device (201) may identify a second text indicating at least one effect corresponding to at least one guide.

[0129] For example, text indicating an HDR effect may be “HDR image, High contrast.” For example, text indicating a bokeh effect may be “Out-focused image, DSLR, 100mm, macro.” For example, text indicating a blur effect may be “Blurred image, background.” For example, text indicating a black and white effect may be “Black and white image, entire.” For example, text indicating a deblur effect may be “De-blurred image, super clear, 4k.”

[0130] According to one embodiment, in operation 435, the electronic device (201) may obtain text information for editing the first image by combining the first text and the second text. According to one embodiment, the electronic device (201) may obtain text information for editing the first image, such as “apply a certain effect to something included in the first image.” According to one embodiment, the processor (220) may obtain information about “what included in the first image” using the first text, and obtain information about “apply a certain effect” using the second text. For example, the text information may indicate “apply an out-of-focus effect to a background including a house, a lawn, and a blue sky.”

[0131] According to one embodiment, in operation 437, the electronic device (201) may provide a first image and text information for editing the first image to a second artificial intelligence model (320) (e.g., the second artificial intelligence model (320) of FIG. 3A) to obtain a second image to which at least one effect is applied. For example, the second image may include an image to which an out-of-focus effect is applied to a background included in the first image. For example, the second artificial intelligence model (320) may be stored on a separate server or device.

[0132] FIG. 5 is a diagram for explaining a template and a first text input to a first artificial intelligence model according to one embodiment.

[0133] Referring to FIG. 5, according to one embodiment, an electronic device (201) (e.g., the electronic device (201) of FIG. 2) may provide a first image to a first artificial intelligence model (310) (e.g., the first artificial intelligence model (310) of FIG. 3A). According to one embodiment, the electronic device (201) may further provide a template including a query to the first artificial intelligence model (310).

[0134] According to one embodiment, the first artificial intelligence model (310) may output a first text representing information related to the first image based on the template.

[0135] For example, questions included in a template might include, "Are you wearing glasses?", "What is your gender?", "What is your race?", and "Are you wearing a mask?". However, these are just examples and may not be limited to this, and various other questions may be included.

[0136] According to one embodiment, the first artificial intelligence model (310) may analyze the first image based on a query included in the template and output the result as first text representing information related to the first image.

[0137] In one embodiment, the first text may include an answer corresponding to the query. For example, the first artificial intelligence model (310) may output "No" as the first text to the question "Do you wear glasses?", "Female" as the answer to the question "What is your gender?", "Black" as the answer to the question "What is your race?", and "No" as the answer to the question "Are you wearing a mask?"

[0138] According to one embodiment, the query included in the template may be set by the user or automatically set by the electronic device (201). The query included in the template may be stored on a server or stored in a separate external device.

[0139] Through this, the electronic device (201) according to one embodiment can avoid acquiring unnecessary information about the first image, thereby reducing resources for the first text.

[0140] FIG. 6 is a drawing for explaining a screen for editing an image by an electronic device according to one embodiment.

[0141] Referring to FIG. 6, according to one embodiment, an electronic device (201) (e.g., the electronic device (201) of FIG. 2) may display a screen for editing an image through a display (260) (e.g., the display (260) of FIG. 2). According to one embodiment, the screen for editing an image may include an execution screen of an application for editing an image.

[0142] According to one embodiment, the electronic device (201) can confirm a user input for editing a first image (610) stored in the memory (210) through a screen for editing an image. For example, the user input for editing the first image (610) may represent a user input for selecting an image to be edited.

[0143] According to one embodiment, when a user input for editing a first image (610) is confirmed, the electronic device (201) may display a plurality of guides (620, 630, 640) corresponding to the first image (610) and a plurality of effects for editing the first image (610). According to one embodiment, the plurality of guides (620, 630, 640) may include texts indicating the plurality of effects.

[0144] In one embodiment, the plurality of effects may include a high dynamic range (HDR) effect, a bokeh effect, and a deblur effect. In one embodiment, the plurality of guides (620, 630, 640) may include a guide for an HDR effect (620), a guide for a bokeh effect (630), and a guide for a deblur effect (640).

[0145] FIG. 7 is a diagram illustrating an operation of an electronic device obtaining text information for editing an image, according to one embodiment.

[0146] Referring to FIG. 7, according to one embodiment, when a user input for editing a first image (610) (e.g., the first image (610) of FIG. 6) is confirmed, the electronic device (201) (e.g., the electronic device (201) of FIG. 2) may input the first image (610) into a first artificial intelligence model (310) (e.g., the first artificial intelligence model (310) of FIG. 3A) to obtain a first text (710) representing information related to the first image (610).

[0147] According to one embodiment, the first text (710) representing information related to the first image (610) may include text based on at least one of the subject, background, color, brightness, or composition included in the first image (610). According to one embodiment, the first image (910) may represent an image including a background including a house, a lawn, and a blue sky, and a girl wearing pink clothes and with her head tilted to the left. For example, the first text (710) may represent “a girl wearing pink clothes and with her head tilted to the left, with a background including a house, a lawn, and a blue sky.”

[0148] According to one embodiment, the electronic device (201) can confirm a user input for selecting a guide (630) for a bokeh effect among a plurality of guides (620, 630, 640). According to one embodiment, the electronic device (201) can confirm a second text (or text command) (720) indicating a bokeh effect based on the user input for selecting the guide (630) for the bokeh effect. For example, the second text (or text command) (720) can indicate “Out-focused image, DSLR, 100mm, macro”.

[0149] According to one embodiment, the electronic device (201) can combine a first text (710) and a second text (720). According to one embodiment, the electronic device (201) can obtain text information (or text command, text prompt) (730) for editing the first image (610) based on the combination of the first text (710) and the second text (720).

[0150] According to one embodiment, the text information (or text command, text prompt) (730) may include information that instructs the AI ​​model (e.g., the second artificial intelligence model (320) (e.g., the second artificial intelligence model (320) of FIG. 3A) to “apply what effect to what is included in the first image.” According to one embodiment, the text information (or text command, text prompt) (730) may include information that instructs the AI ​​model (e.g., the second artificial intelligence model) to “apply an out-of-focus effect to a background including a house, a lawn, and a blue sky.”

[0151] In one embodiment, the electronic device (201) may provide the first text (710) and the second text (720) to the second artificial intelligence model (320) instead of text information (or text command, text prompt) (730) for editing the first image (610) that combines the first text (710) and the second text (720).

[0152] FIG. 8 is a diagram for explaining an operation of an electronic device to obtain an image with a bokeh effect applied, according to one embodiment.

[0153] Referring to FIG. 8, according to one embodiment, an electronic device (201) (e.g., the electronic device (201) of FIG. 2) may input a first image (610) and text information (730) for editing the first image (610) (e.g., the text information (730) for editing the first image (730) of FIG. 7) into a second artificial intelligence model (320) (e.g., the second artificial intelligence model (320) of FIG. 3A) to obtain a second image (810) to which a bokeh effect is applied to the first image (610). According to one embodiment, the second image (810) to which the bokeh effect is applied may include an image to which an out-of-focus effect is applied to a background (830) including a house, a lawn, and a blue sky.

[0154] According to one embodiment, the electronic device (201) can display the second image (810) through a display (260) (e.g., the display (260) of FIG. 2). According to one embodiment, the electronic device (201) can store the second image (810).

[0155] FIG. 9A is a diagram illustrating an operation of an electronic device obtaining text information for editing an image, according to one embodiment. FIG. 9B is a diagram illustrating an operation of an electronic device obtaining an image to which a deblur effect has been applied, according to one embodiment.

[0156] Referring to (a) of FIGS. 9A and 9B, according to one embodiment, when a user input for editing a first image (910) is confirmed, the electronic device (201) (e.g., the electronic device (201) of FIG. 2) inputs the first image (910) into a first artificial intelligence model (310) (e.g., the first artificial intelligence model (310) of FIG. 3A) to obtain a first text (901) indicating information related to the first image (910).

[0157] According to one embodiment, the first text (901) representing information related to the first image (910) may include text based on at least one of a subject, a background, a color, brightness, or a composition included in the first image (910). According to one embodiment, the first image (910) may represent an image of a girl wearing pink clothes and tilting her head to the left, with the background including a house, a lawn, and a blue sky, and the girl being photographed shaking. For example, the first text (901) may include “an image of a girl wearing pink clothes and tilting her head to the left, with the background including a house, a lawn, and a blue sky, and the girl being photographed shaking.”

[0158] According to one embodiment, the electronic device (201) can confirm a user input for selecting a guide (640) for a deblur effect among a plurality of guides (620, 630, 640). According to one embodiment, the electronic device (201) can confirm a second text (or text command) (903) indicating a deblur effect based on the user input for selecting the guide (640) for the deblur effect. According to one embodiment, the second text (903) can indicate “De-blurred image, super clear, 4k”.

[0159] According to one embodiment, the electronic device (201) can combine the first text (901) and the second text (903). According to one embodiment, the electronic device (201) can obtain text information (or text command, text prompt) (905) for editing the first image (910) based on the combination of the first text (901) and the second text (903). According to one embodiment, the text information (905) for editing the first image (910) can include information such as “apply a deblur effect to a girl.”

[0160] Referring to (b) of FIG. 9B, according to one embodiment, the electronic device (201) may input text information (905) for editing the first image (910) and the first image (910) into a second artificial intelligence model (320) (e.g., the second artificial intelligence model (320) of FIG. 3A) to obtain a second image (920) to which a deblur effect is applied to a girl (921). According to one embodiment, the electronic device (201) may display the second image (920) on the display (260).

[0161] In one embodiment, the second artificial intelligence model (320) may include a generative artificial intelligence model. In one embodiment, the electronic device (201) may obtain text information (905) indicating a prompt text command using the first artificial intelligence model (310). For example, the text information (905) indicating the prompt text command may include a text command that includes information that commands the second artificial intelligence model (320) to apply a specific effect (e.g., a deblur effect) to the first image (910).

[0162] According to one embodiment, the electronic device (201) may input the acquired prompt text command into the second artificial intelligence model (320) to command the second artificial intelligence model (320). According to one embodiment, instead of text information (or text command, text prompt) (905) for editing the first image (910) that combines the first text (901) and the second text (903), the electronic device (201) may provide the first text (901) and the second text (903) to the second artificial intelligence model (320).

[0163] According to one embodiment, the electronic device (201) can obtain a second image (920) with a specific effect applied to the first image (910) through the second artificial intelligence model (320).

[0164] FIG. 10A is a diagram illustrating an operation of an electronic device obtaining text information for editing an image, according to one embodiment. FIG. 10B is a diagram illustrating an operation of an electronic device obtaining an image with an HDR effect applied, according to one embodiment.

[0165] Referring to (a) of FIGS. 10A and 10B, according to one embodiment, when a user input for editing a first image (1040) is confirmed, the electronic device (201) (e.g., the electronic device (201) of FIG. 2) inputs the first image (1040) into a first artificial intelligence model (310) (e.g., the first artificial intelligence model (310) of FIG. 3A) to obtain a first text (1010) indicating information related to the first image (1040).

[0166] According to one embodiment, the first text (1010) representing information related to the first image (1040) may include text based on at least one of a subject, a background, a color, brightness, or a composition included in the first image (1040). According to one embodiment, the first image (1040) may represent an image having a first resolution, including a background including a house, a lawn, and a blue sky, a girl wearing pink clothes and tilting her head to the left. For example, the first text (1010) may include “an image having a first resolution, a background including a house, a lawn, and a blue sky, a girl wearing pink clothes and tilting her head to the left.”

[0167] According to one embodiment, the electronic device (201) can confirm a user input for selecting a guide (620) for an HDR effect among a plurality of guides (620, 630, 640). According to one embodiment, the electronic device (201) can confirm a second text (or text command) (1020) indicating an HDR effect based on the user input for selecting the guide (620) for an HDR effect. According to one embodiment, the second text (1020) can indicate “HDR image, High contrast.”

[0168] According to one embodiment, the electronic device (201) can combine the first text (1010) and the second text (1020). According to one embodiment, the electronic device (201) can obtain text information (or text command, text prompt) (1030) for editing the first image (1040) based on the combination of the first text (1010) and the second text (1020). According to one embodiment, the text information (1030) for editing the first image (1040) can include information such as “Change the resolution of the image including a girl wearing a pink dress and with her head tilted to the left, with a background including a house, a lawn, and a blue sky, from the first resolution to HDR.”

[0169] Referring to (b) of FIG. 10B, according to one embodiment, the electronic device (201) may input text information (1030) for editing the first image (1040) and the first image (1040) into the second artificial intelligence model (320) (e.g., the second artificial intelligence model (320) of FIG. 3A) to obtain a second image (1050) to which an HDR effect is applied. According to one embodiment, the electronic device (201) may display the second image (1050) on the display (260). According to one embodiment, instead of text information (or text command, text prompt) (1030) for editing the first image (1040) which is a combination of the first text (1010) and the second text (1020), the electronic device (201) may provide the first text (1010) and the second text (1030) to the second artificial intelligence model (320).

[0170] In one embodiment, the first artificial intelligence model (310) may include an image segmentation model or a detector model. In one embodiment, the second artificial intelligence model (320) may include a generative artificial intelligence model. For example, the second artificial intelligence model (320) may include a diffusion model.

[0171] Depending on the implementation, the first artificial intelligence model (310) and the second artificial intelligence model (320) may be identical. Depending on the implementation, the first artificial intelligence model (310) and the second artificial intelligence model (320) may be implemented as a single artificial intelligence model.

[0172] FIG. 11A is a diagram illustrating an operation of an electronic device obtaining text information for editing an image, according to one embodiment. FIG. 11B is a diagram illustrating an operation of an electronic device obtaining an image with a black-and-white effect applied, according to one embodiment.

[0173] Referring to (a) of FIGS. 11A and 11B, according to one embodiment, when a user input for editing a first image (610) is confirmed, the electronic device (201) (e.g., the electronic device (201) of FIG. 2) may provide the first image (1040) to a first artificial intelligence model (310) (e.g., the first artificial intelligence model (310) of FIG. 3A) to obtain a first text (1110) representing information related to the first image (610).

[0174] According to one embodiment, the first text (1110) representing information related to the first image (610) may include text based on at least one of a subject, a background, a color, brightness, or a composition included in the first image (610). According to one embodiment, the first image (610) may represent an image having a first resolution, including a background including a house, a lawn, and a blue sky, and a girl wearing pink clothes and with her head tilted to the left. For example, the first text (1110) may include “an image including a background including a house, a lawn, and a blue sky, and a girl wearing pink clothes and with her head tilted to the left.”

[0175] According to one embodiment, the electronic device (201) can confirm a user input for selecting a guide (650) for a black and white effect among a plurality of guides (620, 630, 640, 650). According to one embodiment, the electronic device (201) can confirm a second text (or text command) (1120) indicating a black and white effect based on the user input for selecting the guide (650) for the black and white effect. According to one embodiment, the second text (1120) can indicate “Black and white photo, high contrast, high details, studio quality, dramatic lighting, 8k, hires”.

[0176] According to one embodiment, the electronic device (201) can combine the first text (1110) and the second text (1120). According to one embodiment, the electronic device (201) can obtain text information (or text command, text prompt) (1130) for editing the first image (610) based on the combination of the first text (1110) and the second text (1120). According to one embodiment, the text information (1130) for editing the first image (610) can include information such as “Change the image including a girl wearing a pink dress and her head tilted to the left, with a background including a house, a lawn, and a blue sky, to a black and white image.”

[0177] Referring to (b) of FIG. 11B, according to one embodiment, the electronic device (201) may provide text information (1130) for editing the first image (610) and the first image (610) to the second artificial intelligence model (320) (e.g., the second artificial intelligence model (320) of FIG. 3A) to obtain a second image (1140) with a black and white effect applied. According to one embodiment, the electronic device (201) may display the second image (1140) on the display (260). According to one embodiment, instead of text information (or text command, text prompt) (1130) for editing the first image (610) that combines the first text (1110) and the second text (1120), the electronic device (201) may also provide the first text (1110) and the second text (1120) to the second artificial intelligence model (320).

[0178] FIG. 12 is a flowchart illustrating an operation of an electronic device editing a sketch according to one embodiment.

[0179] Referring to FIG. 12, according to one embodiment, in operation 1211, the electronic device (201) (e.g., the electronic device (201) of FIG. 2) may obtain a sketch (or drawing). According to one embodiment, the electronic device (201) may display a screen for obtaining the sketch (or drawing) by the user's body or a stylus pen through the display (260) (e.g., the display (260) of FIG. 2). For example, the sketch may be obtained based on a plurality of strokes based on a touch input. For example, the stroke may represent a continuous stroke from a point where a touch input is started to a point where the touch input is released. According to one embodiment, the screen for obtaining the sketch (or drawing) may include an execution screen of an application for obtaining the sketch (e.g., a memo-related application, a drawing-related application). According to one embodiment, the electronic device (201) may display a screen for editing an acquired sketch (or drawing) through a display (260). For example, the sketch may include at least one of a person, an animal, an object, a plant, or a natural environment.

[0180] According to one embodiment, in operation 1213, the electronic device (201) may provide the sketch to the first artificial intelligence model (310) (e.g., the first artificial intelligence model (310) of FIG. 3A) to obtain third text representing information related to the sketch. According to one embodiment, the third text representing information related to the sketch may include text based on at least one of the resolution of the sketch, at least one object (e.g., the subject) included in the sketch, the background, the color, the brightness, or the composition. For example, the first artificial intelligence model (310) may be stored on a server or an external device.

[0181] According to one embodiment, the electronic device (201) may further provide a template containing a query along with the sketch to the first artificial intelligence model (310). According to one embodiment, the first artificial intelligence model (310) may analyze the sketch based on the query included in the template and output the result as a third text representing information related to the sketch. According to one embodiment, the third text may include an answer corresponding to the query.

[0182] In one embodiment, at operation 1215, the electronic device (201) may display a plurality of second guides corresponding to a plurality of second effects that may be applied to the sketch via the display (260) (e.g., the display (260) of FIG. 2 ). In one embodiment, the plurality of second guides may include texts representing the plurality of second effects or a plurality of images designated for the plurality of second effects. For example, the designated images may represent designated images to which the plurality of effects are applied.

[0183] According to one embodiment, the plurality of second effects may include effects that change the image quality of the image (or sketch) and / or effects that change the style of the image (or sketch), the drawing style of the image (or sketch), or the expression technique of the image (or sketch). According to one embodiment, the plurality of second effects may include at least one of a high dynamic range (HDR) effect, a bokeh effect, a blur effect for the background, an effect that changes the image to a black and white image, and a deblur effect. According to one embodiment, the plurality of second effects may include an illustration effect, a watercolor effect, a sketch effect, a pop art effect, and a 3D cartoon effect. However, this is an example, and the plurality of effects may not be limited to the above examples. For example, the illustration effect may include an effect that changes the image into an illustration style. For example, the sketch effect may include an effect that changes the image into a pencil sketched image. For example, a watercolor effect may include a watercolor effect that changes an image into an image that appears to have been painted with watercolor paints. For example, a 3D character effect may include an effect that changes at least one object (e.g., a person, an animal) included in an image into a three-dimensional character. For example, a pop art effect may include an effect of a pop art technique that changes the colors of an image into the colors of acrylic paint and changes at least one object included in the image into a three-dimensional shape.

[0184] According to one embodiment, in operation 1217, the electronic device (201) may confirm an input for selecting at least one guide from among a plurality of second guides. For example, the plurality of second guides may include texts representing a plurality of second effects or designated images representing a plurality of second effects. For example, the designated images may represent designated images to which a plurality of second effects are applied.

[0185] According to one embodiment, in operation 1219, the electronic device (201) may identify a fourth text indicating at least one effect corresponding to at least one guide.

[0186] According to one embodiment, in operation 1221, the electronic device (201) provides the sketch, the third text, and the fourth text to the second artificial intelligence model (320) (e.g., the second artificial intelligence model (320) of FIG. 3A) to obtain a plurality of images to which at least one effect is applied to the sketch. For example, the second artificial intelligence model (320) may be stored on a server or an external device. For example, the number of the plurality of images may be set to four, but this is an example and may be set to various numbers.

[0187] Depending on the implementation, in one embodiment, the electronic device (201) may further provide a fifth text to the second artificial intelligence model (320) regarding effects not to be applied to the sketch. For example, the fifth text may include text indicating that at least one object (e.g., a subject) included in the sketch is not nude, is not wrinkled, and a background included in the sketch is not dirty.

[0188] According to one embodiment, in operation 1223, the electronic device (201) may store the second image in the memory (210) based on confirming a user input for selecting a second image from among the plurality of images. According to one embodiment, the electronic device (201) may store the selected at least one image in the memory (210) based on confirming a user input for selecting at least one image from among the plurality of images.

[0189] FIG. 13A is a diagram illustrating an operation of an electronic device obtaining text information for editing a sketch, according to one embodiment. FIG. 13B is a diagram illustrating an operation of an electronic device obtaining an image with a pop art effect applied, according to one embodiment.

[0190] Referring to (a) and (b) of FIG. 13A, according to one embodiment, an electronic device (201) (e.g., the electronic device (201) of FIG. 2) can obtain a sketch (or drawing) (1300) by a user's body or a stylus pen.

[0191] According to one embodiment, the electronic device (201) can display a screen for editing a sketch (1300) obtained through the display (260) (e.g., the display (260) of FIG. 2) through the display (260).

[0192] According to one embodiment, when a user input for editing a sketch (1300) is confirmed, the electronic device (201) may provide the sketch to a first artificial intelligence model (310) (e.g., the first artificial intelligence model (310) of FIG. 3A) to obtain a third text (1370) representing information related to the sketch (1300).

[0193] According to one embodiment, the third text (1370) representing information related to the sketch (1300) may include text based on at least one of the subject, background, color, brightness, or composition included in the sketch (1300). For example, the third text (1370) may include, “A sketch including a girl wearing a one-piece dress, with her arms outstretched, smiling, with pigtails and short bangs, and a single flower in the lower right corner.”

[0194] According to one embodiment, the electronic device (201) can confirm a user input for selecting a guide (1350) for a pop art effect among a plurality of guides (1310, 1320, 1330, 1340, 1350, 1360). For example, the plurality of guides (1310, 1320, 1330, 1340, 1350, 1360) may include texts indicating a plurality of effects. For example, the plurality of guides (1310, 1320, 1330, 1340, 1350, 1360) may also include images designated for a plurality of effects. The designated images may include images to which a plurality of effects are applied.

[0195] In one embodiment, the electronic device (201) may identify a fourth text (or text command) (1380) representing a pop art effect based on a user input selecting a guide (1350) for a pop art effect. In one embodiment, the fourth text (1380) may represent “3D pop art style, pastel tone.”

[0196] According to one embodiment, the electronic device (201) can combine the third text (1370) and the fourth text (1380). According to one embodiment, the electronic device (201) can obtain text information (or text command, text prompt) (1390) for editing the sketch (1300) based on the combination of the third text (1370) and the fourth text (1380). According to one embodiment, the text information (1390) for editing the sketch (1300) can include information such as “Change the sketch including a girl wearing a one-piece dress, spreading both arms, having a smiling expression, with pigtails and short bangs, and a single flower in the lower right corner to a three-dimensional pop art style and an image with pastel colors applied.”

[0197] Referring to (b) of FIG. 13A, according to one embodiment, the electronic device (201) may input text information (1390) for editing a sketch (1300) and the sketch (1300) into a second artificial intelligence model (320) (e.g., the second artificial intelligence model (320) of FIG. 3A), thereby obtaining a plurality of images (1351, 1352, 1353, 1354) to which a pop art effect is applied. According to one embodiment, the electronic device (201) may display the plurality of images (1351, 1352, 1353, 1354) on a display (260).

[0198] According to one embodiment, when a user input selecting at least one of a plurality of images (1351, 1352, 1353, 1354) is confirmed, the electronic device (201) may store at least one image in the memory (210) (e.g., the memory (210) of FIG. 2).

[0199] FIG. 14A is a diagram illustrating an operation of an electronic device obtaining text information for editing a third image, according to one embodiment. FIG. 14B is a diagram illustrating an operation of an electronic device obtaining an image to which a sketch effect is applied, according to one embodiment.

[0200] Referring to (a) and (b) of FIG. 14A, according to one embodiment, when a user input for editing a third image (1400) is confirmed, the electronic device (201) (e.g., the electronic device (201) of FIG. 2) provides the third image (1400) to the first artificial intelligence model (310) (e.g., the first artificial intelligence model (310) of FIG. 3A) to obtain a fifth text (1470) indicating information related to the third image (1400).

[0201] According to one embodiment, the fifth text (1470) representing information related to the third image (1400) may include text based on at least one of the subject, background, color, brightness, or composition included in the third image (1400). For example, the fifth text (1470) may include “an image including an adult male looking straight ahead, having a beard, and not wearing glasses.”

[0202] According to one embodiment, the electronic device (201) may confirm a user input for selecting a guide (1410) for a sketch (or caricature) effect among a plurality of guides (1410, 1420). For example, the plurality of guides (1410, 1420) may include images designated for a plurality of effects. The designated images may include images to which a plurality of effects are applied. For example, the plurality of guides (1410, 1420) may also include texts indicating a plurality of effects.

[0203] According to one embodiment, the electronic device (201) may identify a sixth text (or text command) (1480) representing a sketch (or caricature) effect based on a user input selecting a guide (1420) for a sketch (or caricature) effect. According to one embodiment, the sixth text (1480) may indicate “a style sketched using a crosshatching technique using a 6B pencil.”

[0204] According to one embodiment, the electronic device (201) can combine the fifth text (1470) and the sixth text (1480). According to one embodiment, the electronic device (201) can obtain text information (or text command, text prompt) (1490) for editing the third image (1400) based on the combination of the fifth text (1470) and the sixth text (1480). According to one embodiment, the text information (1490) for editing the third image (1400) can include information such as “Change the image including an adult male looking straight ahead, having a beard, and not wearing glasses into an image in the style of a cross-hatching sketch using a 6B pencil.”

[0205] Referring to (b) of FIG. 14A, according to one embodiment, the electronic device (201) may provide text information (1490) for editing a third image (1400) and the third image (1400) to a second artificial intelligence model (320) (e.g., the second artificial intelligence model (320) of FIG. 3A), thereby obtaining a plurality of images (1430, 1440, 1450, 1460) to which a sketch (or caricature) effect is applied. According to one embodiment, the electronic device (201) may display the plurality of images (1430, 1440, 1450, 1460) on the display (260).

[0206] According to one embodiment, when a user input selecting at least one image from among a plurality of images (1430, 1440, 1450, 1460) is confirmed, the electronic device (201) may store at least one image in the memory (210) (e.g., the memory (210) of FIG. 2).

[0207] FIG. 15 is a flowchart illustrating an operation of an electronic device editing a sketch according to one embodiment.

[0208] Referring to (a) of FIG. 15, according to one embodiment, an electronic device (201) (e.g., the electronic device (201) of FIG. 2) may execute an application related to a memo. Based on executing the application related to a memo, the electronic device (201) may provide a user interface that allows a user to input a sketch (or drawing) by the user's body or a stylus pen and / or a user interface (e.g., a keypad) that allows a user to input text.

[0209] According to one embodiment, the electronic device (201) can obtain a first sketch (or drawing) (1520) and a second sketch (or drawing) (1530) by the user's body or a stylus pen. According to one embodiment, the electronic device (201) can obtain a text input (1510) based on the user's input. For example, the text input (1510) can represent text obtained by a user interface (e.g., a keypad) or handwriting by the user's body (or a stylus pen). For example, the first sketch (1520), the second sketch (1530), and the text input (1510) can be related to content (e.g., a movie, a comic, a book, an animation, etc.).

[0210] According to one embodiment, the electronic device (201) may display a first sketch (1520), a second sketch (1530), and a text input (1510).

[0211] According to one embodiment, the electronic device (201) may provide a first sketch (1520) and a text input (1510) to a first artificial intelligence model (310) (e.g., the first artificial intelligence model (310) of FIG. 3A) to obtain a first text representing information related to the first sketch. The first artificial intelligence model (310) may analyze the first sketch (1520) and the text input (1510) to obtain a first text representing information related to the first sketch (1520). For example, the first artificial intelligence model (310) may perform a function such as optical character recognition (OCR). For example, the first text may represent “B movie, shark looking left, showing three teeth on top.” According to one embodiment, the electronic device (201) may, based on an input for selecting one of a plurality of guides (e.g., a cartoon), identify a second text indicating an effect (e.g., a cartoon effect) corresponding to one of the guides. Depending on the implementation, according to one embodiment, one of the guides (e.g., a cartoon) may be automatically selected by the electronic device (201). For example, one of the guides (e.g., a cartoon) may be pre-designated according to a type of content (e.g., a movie). For example, one of the guides (e.g., a cartoon) may be pre-designated according to an object included in the sketch (e.g., a person, an animal, an object). According to one embodiment, the electronic device (201) may provide the first sketch (1520), the first text, and the second text to the second artificial intelligence model (320) (e.g., the second artificial intelligence model (320) of FIG. 3A) to obtain an image (1521) in which the cartoon effect is applied to the first sketch (1520).

[0212] According to one embodiment, the electronic device (201) may provide the second sketch (1530) and the text input (1510) to the first artificial intelligence model (310) to obtain a third text representing information related to the first sketch. The first artificial intelligence model (310) may analyze the second sketch (1530) and the text input (1510) to obtain a third text representing information related to the second sketch (1530). For example, the third text may represent “D movie, a man looking straight ahead, with a high nose bridge, brown hair, and thick lips.” According to one embodiment, the electronic device (201) may identify a fourth text representing an effect (e.g., a cartoon effect) corresponding to one of the plurality of guides based on identifying an input for selecting one of the plurality of guides (e.g., a cartoon). According to one embodiment, one of the guides (e.g., a cartoon) may be automatically selected by the electronic device (201). For example, one guide (e.g., cartoon) may be pre-designated according to the type of content (e.g., movie). For example, one guide (e.g., cartoon) may be pre-designated according to the object (e.g., person, animal, object) included in the sketch. According to one embodiment, the electronic device (201) may provide the second sketch (1530), the third text, and the fourth text to the second artificial intelligence model (320), thereby obtaining an image (1531) with a cartoon effect applied to the second sketch (1530).

[0213] Referring to (b) of FIG. 15, according to one embodiment, the electronic device (201) can display a text input (1510), an image (1521) with a cartoon effect applied to a first sketch (1520), and an image (1531) with a cartoon effect applied to a second sketch (1530).

[0214] According to one embodiment, the electronic device (201) may display an image (1521) to which a cartoon effect is applied to the first sketch (1520) or an image (1531) to which a cartoon effect is applied to the second sketch (1530) by applying a visual effect. For example, the visual effect may include a hatching effect, a blinking effect, or an effect of applying a specified color. However, this is just an example, and various effects may be included. For example, when an input (e.g., a touch input) for 'B movie' is confirmed among text inputs (1510), the electronic device (201) may apply a visual effect to an image (1521) to which a cartoon effect is applied to the first sketch (1520). For example, when an input (e.g., a touch input) for 'D movie' is confirmed among text inputs (1510), the electronic device (201) may apply a visual effect to an image (1531) to which a cartoon effect is applied to the second sketch (1530).

[0215] For example, when an input (e.g., touch input) for 'B movie' is confirmed among text inputs (1510), the electronic device (201) can apply an effect in which an image (1531) with a cartoon effect applied to the second sketch (1530) disappears. For example, when an input (e.g., touch input) for 'D movie' is confirmed among text inputs (1510), the electronic device (201) can apply an effect in which an image (1521) with a cartoon effect applied to the first sketch (1520) disappears.

[0216] According to one embodiment, when an input (e.g., touch input) for an image (1521) to which a cartoon effect is applied to a first sketch (1520) is confirmed, the electronic device (201) may apply a visual effect to a text input (e.g., “Start: 1975 Director A, Movie B, First Box Office Revenue of $100 Million”) related to the image (1521) to which a cartoon effect is applied to the first sketch (1520) among text inputs (1510). According to one embodiment, when an input (e.g., touch input) for an image (1531) to which a cartoon effect is applied to the second sketch (1530) is confirmed, the electronic device (201) may apply a visual effect to a text input (e.g., 1977 Director C, Movie D, $180 Million) related to the image (1531) to which a cartoon effect is applied to the second sketch (1530) among text inputs (1510). For example, visual effects may include blinking effects or applying a specified color. However, this is only an example and various other effects may be included.

[0217] FIG. 16 is a diagram illustrating examples of text commands that indicate effects corresponding to the guide provided to an artificial intelligence model, according to one embodiment. According to one embodiment, the guide, the first prompt, and the second prompt may not be limited to those described in FIG. 16.

[0218] Referring to FIG. 16, according to one embodiment, a first prompt (text, text command) corresponding to the guide may be provided to the second artificial intelligence model (320) (e.g., the second artificial intelligence model (320) of FIG. 3A). Depending on the implementation, according to one embodiment, a second prompt (text, text command) corresponding to the guide may also be provided to the second artificial intelligence model (320).

[0219] In one embodiment, the first prompt may indicate a prompt (text, text command) corresponding to an effect to be applied to the image. In one embodiment, the second prompt may indicate a prompt (text, text command) corresponding to an effect not to be applied to the image.

[0220] In one embodiment, a first prompt (text command, text) corresponding to the illustration effect may indicate “illustration style with bold outline, pastel tone.”

[0221] In one embodiment, a first prompt (text command, text) corresponding to a 3D character effect may indicate “3D cartoon character style, pastel tone.”

[0222] In one embodiment, a first prompt (text command, text) corresponding to a watercolor effect may indicate “watercolor style using watercolor paints, soft strokes, pastel tones.”

[0223] According to one embodiment, the first prompt (text command, text) corresponding to the caricature effect may indicate “a style sketched using a cross-hacking technique using a 6B pencil.”

[0224] In one embodiment, a first prompt (text command, text) corresponding to the pop art effect may indicate “3D pop art style, pastel tones.”

[0225] In one embodiment, the second prompt (text command, text) may indicate “nude, wrinkled, dirty, blurry, text.” For example, the second prompt may indicate a text command that instructs the second artificial intelligence model (320) to ensure that at least one object (e.g., a person) included in the output image is not nude, does not have wrinkles, the background of the output image is not dirty, the output image is not blurry, and the output image does not include text.

[0226] According to one embodiment, the second artificial intelligence model (320) may apply an effect corresponding to the first prompt to the image based on the first prompt being provided.

[0227] In one embodiment, the second artificial intelligence model (320) may not apply an effect corresponding to the second prompt to the image based on the second prompt being provided. In one embodiment, the electronic device may include a display, a memory, and at least one processor.

[0228] According to one embodiment, the memory may store instructions that, when individually or collectively executed by the at least one processor, cause the electronic device to, when a first input for editing a first image stored in the memory is identified, provide the first image to a first artificial intelligence model to obtain a first text representing information related to the first image.

[0229] According to one embodiment, the memory may store instructions that, when individually or collectively executed by the at least one processor, cause the electronic device to display, through the display, a plurality of first guides corresponding to a plurality of first effects applicable to the first image.

[0230] According to one embodiment, the memory may store instructions that, when individually or collectively executed by the at least one processor, cause the electronic device to, based on a second input selecting at least one guide from the plurality of first guides, identify a second text indicating at least one effect corresponding to the at least one guide.

[0231] According to one embodiment, the memory may store instructions that, when individually or collectively executed by the at least one processor, cause the electronic device to provide the first image, the first text, and the second text to a second artificial intelligence model to obtain a second image having the at least one effect applied to the first image.

[0232] According to one embodiment, the memory may store instructions that, when individually or collectively executed by the at least one processor, cause the electronic device to obtain text information for editing the first image by combining the first text and the second text.

[0233] According to one embodiment, the memory may store instructions that, when individually or collectively executed by the at least one processor, cause the electronic device to provide the first image and the text information to the second artificial intelligence model to obtain the second image.

[0234] According to one embodiment, the memory may store instructions that, when individually or collectively executed by the at least one processor, cause the electronic device to obtain the first text based on at least one object, background, color, or composition included in the first image using the first artificial intelligence model.

[0235] According to one embodiment, the memory may store instructions that, when individually or collectively executed by the at least one processor, cause the electronic device to identify a plurality of first effects for editing an image based on information related to the first image.

[0236] According to one embodiment, the memory may store instructions that, when individually or collectively executed by the at least one processor, cause the electronic device to display, on the display, a plurality of first guides corresponding to the plurality of first effects among the plurality of guides corresponding to the plurality of effects.

[0237] According to one embodiment, the electronic device may include the plurality of first guides including texts representing the plurality of first effects that can be applied to the first image or designated images representing the plurality of first effects.

[0238] According to one embodiment, the electronic device may be configured such that the plurality of first effects include a high dynamic range (HDR) effect, a bokeh effect, a blur effect for at least one object included in the first image, an effect for changing the first image into a black and white image, a deblur effect, a watercolor effect, a sketch effect, an illustration effect, a 3D character effect for changing the at least one object into a three-dimensional character, and a pop art effect.

[0239] According to one embodiment, the electronic device may include the second artificial intelligence model, which may include a diffusion model.

[0240] According to one embodiment, the memory may store instructions that, when individually or collectively executed by the at least one processor, cause the electronic device to provide the first image, the first text, and the second text to the second artificial intelligence model (320) to obtain a plurality of images including the second image having the at least one effect applied to the first image.

[0241] According to one embodiment, the memory may store instructions that, when individually or collectively executed by the at least one processor, cause the electronic device to store the second image based on identifying a user input selecting the second image from among the plurality of images.

[0242] According to one embodiment, the first artificial intelligence model and the second artificial intelligence model may be stored in the memory or stored on a server.

[0243] According to one embodiment, a method of operating an electronic device may include, when a first input for editing a first image stored in a memory of the electronic device is confirmed, providing the first image to a first artificial intelligence model to obtain a first text representing information related to the first image.

[0244] According to one embodiment, a method of operating an electronic device may include displaying, through a display of the electronic device, a plurality of first guides corresponding to a plurality of first effects that can be applied to the first image.

[0245] According to one embodiment, a method of operating an electronic device may include an operation of identifying a second text indicating at least one effect corresponding to at least one guide based on a second input selecting at least one guide from among the plurality of first guides.

[0246] According to one embodiment, a method of operating an electronic device may include providing the first image, the first text, and the second text to a second artificial intelligence model to obtain a second image having the at least one effect applied to the first image.

[0247] According to one embodiment, a method of operating an electronic device may include an operation of obtaining text information for editing the first image by combining the first text and the second text.

[0248] According to one embodiment, a method of operating an electronic device may include providing the first image and the text information to the second artificial intelligence model to obtain the second image.

[0249] According to one embodiment, a method of operating an electronic device may include an operation of obtaining the first text based on at least one object, background, color, or composition included in the first image using the first artificial intelligence model.

[0250] According to one embodiment, a method of operating an electronic device may include an operation of identifying a plurality of first effects based on information related to the first image among a plurality of effects for editing an image.

[0251] According to one embodiment, a method of operating an electronic device may include an operation of displaying, on the display, a plurality of first guides corresponding to the plurality of first effects among a plurality of guides corresponding to the plurality of effects.

[0252] According to one embodiment, a method of operating an electronic device may include: the plurality of first guides may include texts representing the plurality of first effects that can be applied to the first image or designated images representing the plurality of first effects.

[0253] According to one embodiment, the method of operating the electronic device may include the plurality of first effects including a high dynamic range (HDR) effect, a bokeh effect, a blur effect for at least one object included in the first image, an effect for changing the first image into a black and white image, a deblur effect, a watercolor effect, a sketch effect, an illustration effect, a 3D character effect for changing the at least one object into a three-dimensional character, and a pop art effect.

[0254] According to one embodiment, the method of operating the electronic device may include the second artificial intelligence model including a diffusion model.

[0255] According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor of an electronic device, cause the electronic device to perform at least one operation, wherein the at least one operation may include, when a first input for editing a first image stored in a memory of the electronic device is confirmed, providing the first image to a first artificial intelligence model to obtain a first text representing information related to the first image.

[0256] According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor of an electronic device, cause the electronic device to perform at least one operation, wherein the at least one operation may include displaying, through a display of the electronic device, a plurality of first guides corresponding to a plurality of first effects that can be applied to the first image.

[0257] According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor of an electronic device, cause the electronic device to perform at least one operation, wherein the at least one operation may include an operation of identifying a second text indicating at least one effect corresponding to at least one guide based on a second input selecting at least one guide from among the plurality of first guides.

[0258] According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor of an electronic device, cause the electronic device to perform at least one operation, wherein the at least one operation may include providing the first image, the first text, and the second text to a second artificial intelligence model to obtain a second image having the at least one effect applied to the first image.

[0259] According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor of an electronic device, cause the electronic device to perform at least one operation, wherein the at least one operation may include an operation of obtaining text information for editing the first image by combining the first text and the second text.

[0260] According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor of an electronic device, cause the electronic device to perform at least one operation, wherein the at least one operation may include providing the first image and the text information to the second artificial intelligence model to obtain the second image.

[0261] According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor of an electronic device, cause the electronic device to perform at least one operation, wherein the at least one operation may include an operation of obtaining the first text based on at least one object, background, color, or composition included in the first image using the first artificial intelligence model.

[0262] According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor of an electronic device, cause the electronic device to perform at least one operation, wherein the at least one operation may include an operation of identifying a plurality of first effects based on information related to the first image among a plurality of effects for editing an image.

[0263] According to one embodiment, a storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor of an electronic device, cause the electronic device to perform at least one operation, wherein the at least one operation may include displaying, on the display, a plurality of first guides corresponding to the plurality of first effects among a plurality of guides corresponding to the plurality of effects.

[0264] According to one embodiment, in a storage medium storing computer-readable instructions, the plurality of first guides may include texts representing a plurality of first effects that can be applied to the first image or designated images representing the plurality of first effects.

[0265] According to one embodiment, in a storage medium storing computer-readable instructions, the plurality of first effects may include a high dynamic range (HDR) effect, a bokeh effect, a blur effect for at least one object included in the first image, an effect for changing the first image into a black and white image, a deblur effect, a watercolor effect, a sketch effect, an illustration effect, a 3D character effect for changing the at least one object into a three-dimensional character, and a pop art effect.

[0266] According to one embodiment, in a storage medium storing computer-readable instructions, the second artificial intelligence model may include a diffusion model.

[0267] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments disclosed in this document are not limited to the aforementioned devices.

[0268] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0269] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0270] Various embodiments of the present document may be implemented as software (e.g., program (140)) including one or more commands stored in a storage medium (e.g., built-in memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101, 201)). For example, a processor (e.g., processor (120, 220)) of a machine (e.g., electronic device (101, 201)) may call at least one command among the one or more commands stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the called at least one command. The one or more commands may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means a device in which the storage medium is tangible, It simply means that it does not contain signals (e.g. electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on a storage medium.

[0271] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0272] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (201), display (260); Memory (210) for storing instructions; and comprising at least one processor (220), The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: When a first input for editing a first image stored in the above memory is confirmed, the first image is provided to a first artificial intelligence model (310) to obtain a first text representing information related to the first image, Through the above display, a plurality of first guides corresponding to a plurality of first effects that can be applied to the first image are displayed, Based on a second input selecting at least one guide among the plurality of first guides, a second text indicating at least one effect corresponding to the at least one guide is identified, An electronic device that provides the first image, the first text, and the second text to a second artificial intelligence model (320), thereby causing a second image to be obtained to which at least one effect is applied to the first image.

2. In the first paragraph, the instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An electronic device that causes the first text and the second text to be combined to obtain text information for editing the first image.

3. In any one of paragraphs 1 and 2, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An electronic device that provides the first image and the text information to the second artificial intelligence model to cause the second image to be obtained.

4. In any one of paragraphs 1 to 3, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An electronic device that causes the first text to be obtained based on at least one object, background, color, or composition included in the first image by using the first artificial intelligence model.

5. In any one of paragraphs 1 to 4, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An electronic device that causes a plurality of first effects to be identified based on information related to the first image among a plurality of effects for editing an image.

6. In any one of paragraphs 1 to 5, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An electronic device that causes a plurality of first guides corresponding to the plurality of first effects among a plurality of guides corresponding to the plurality of effects to be displayed on the display.

7. In any one of paragraphs 1 to 6, The above plurality of first guides are: An electronic device comprising texts representing the plurality of first effects that can be applied to the first image or designated images representing the plurality of first effects.

8. In any one of paragraphs 1 to 7, The above plurality of first effects are: An electronic device comprising a HDR (high dynamic range) effect, a Bokeh effect, a blur effect for at least one object included in the first image, an effect for changing the first image into a black and white image, a deblur effect, a watercolor effect, a sketch effect, an illustration effect, a 3D character effect for changing the at least one object into a three-dimensional character, and a pop art effect.

9. In any one of paragraphs 1 to 8, The second artificial intelligence model is an electronic device including a diffusion model.

10. In any one of paragraphs 1 to 9, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Providing the first image, the first text, and the second text to the second artificial intelligence model (320), thereby obtaining a plurality of images including the second image to which at least one effect is applied to the first image, An electronic device that causes storage of the second image based on a user input selecting the second image from among the plurality of images.

11. In any one of paragraphs 1 to 10, An electronic device wherein the first artificial intelligence model and the second artificial intelligence model are stored in the memory or stored in a server.

12. In the operating method of an electronic device (201), When a first input for editing a first image stored in a memory (210) of the electronic device is confirmed, an operation of providing the first image to a first artificial intelligence model (310) to obtain a first text representing information related to the first image; An operation of displaying a plurality of first guides corresponding to a plurality of first effects that can be applied to the first image through a display (260) of the electronic device; An operation of identifying a second text indicating at least one effect corresponding to at least one guide based on a second input selecting at least one guide among the plurality of first guides; and An operating method of an electronic device, comprising: providing the first image, the first text, and the second text to a second artificial intelligence model (320) to obtain a second image to which at least one effect is applied to the first image.

13. In paragraph 12, An operation of obtaining text information for editing the first image by combining the first text and the second text; An operation of providing the first image and the text information to the second artificial intelligence model to obtain the second image; An operation of obtaining the first text based on at least one object, background, color, or composition included in the first image using the first artificial intelligence model; An operation of confirming a plurality of first effects based on information related to the first image among a plurality of effects for editing an image; and / or An operating method of an electronic device further comprising an action of displaying, on the display, a plurality of first guides corresponding to the plurality of first effects among a plurality of guides corresponding to the plurality of effects.

14. A method of operating an electronic device according to claim 12 or 13, wherein the plurality of first guides include texts representing the plurality of first effects that can be applied to the first image or designated images representing the plurality of first effects.

15. In any one of paragraphs 12 to 14, the plurality of first effects are: A method of operating an electronic device comprising a HDR (high dynamic range) effect, a Bokeh effect, a blur effect for at least one object included in the first image, an effect for changing the first image into a black and white image, a deblur effect, a watercolor effect, a sketch effect, an illustration effect, a 3D character effect for changing the at least one object into a three-dimensional character, and a pop art effect.

Images