Electronic device and method for providing image

Abstract

Disclosed are an electronic device and a method for providing an image. The electronic device may receive a drawing according to a first user input and a second user input for image generation via a touch screen display. The electronic device may acquire description information about the drawing by using the drawing on the basis of the second user input. The electronic device may determine the similarity between the drawing and an image to be generated. The electronic device may acquire an image generated on the basis of the similarity and at least a part of the description information, and display the acquired image via the touch screen display.

Description

Electronic device and method for providing images

[0001] The present disclosure relates to an electronic device and method for providing an image.

[0002] Electronic devices have become equipped with complex functions, such as taking photos and videos, playing music and videos, playing games, receiving broadcasts, and supporting wireless Internet, and are being implemented as comprehensive multimedia devices. Accordingly, electronic devices are evolving into new forms, both in hardware and software, to satisfy user needs while enhancing portability and convenience.

[0003] In addition, as artificial intelligence technology advances and generative artificial intelligence models emerge, various attempts are being made to provide services for new content expanded from user input data, rather than being limited to the user's input data.

[0004] The above information may be provided as background information to aid in understanding the present disclosure. None of the above is claimed to be prior art related to the present disclosure or can be used to determine prior art related to the present disclosure.

[0005] An electronic device according to one embodiment may include a touchscreen display, at least one processor including processing circuitry, and a memory storing instructions. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to receive a drawing based on a first user input through the touchscreen display, receive a second user input for generating an image through the touchscreen display, obtain description information of the drawing based on the second user input, determine a similarity between the drawing and an image to be generated, obtain an image generated based on at least a portion of the description information and the similarity, and display the image through the touchscreen display.

[0006] A method for an electronic device to provide an image according to one embodiment may include receiving a drawing according to a first user input through a user interface displayed on a touchscreen display of the electronic device, receiving a second user input for generating an image through the user interface, obtaining description information of the drawing based on the second user input, determining a similarity between the drawing and an image to be generated, obtaining an image generated based on at least a portion of the description information and the similarity, and displaying the image through the touchscreen display.

[0007] A computer-readable non-transitory recording medium according to one embodiment may record a program for executing a method including the steps of: receiving a drawing according to a first user input through a user interface displayed on a touchscreen display of an electronic device; receiving a second user input for generating an image through the user interface; obtaining description information of the drawing based on the second user input; determining a similarity between the drawing and an image to be generated; obtaining an image generated based on at least a portion of the description information and the similarity; and displaying the image through the touchscreen display.

[0008] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0009] FIG. 2 is a block diagram of an electronic device according to one embodiment.

[0010] FIG. 3 is a diagram illustrating a plurality of artificial intelligence models that are linked to an electronic device according to one embodiment.

[0011] FIG. 4 is a flowchart illustrating a method for providing an image of an electronic device according to one embodiment.

[0012] FIG. 5 is an example of a user interface for explaining an image providing function of an electronic device according to one embodiment.

[0013] FIG. 6 is a flowchart illustrating a method for an electronic device to provide an image using multiple artificial intelligence models according to one embodiment.

[0014] FIGS. 7A, 7B, and 7C are examples of user interfaces for describing a process by which an electronic device determines a style according to one embodiment.

[0015] FIG. 8 is a flowchart illustrating a process by which an electronic device generates a first input prompt according to one embodiment.

[0016] FIG. 9 is a flowchart illustrating a process by which an electronic device generates a first input prompt based on a style according to one embodiment.

[0017] FIG. 10A is an example of a first input prompt requesting a brief description of a drawing, according to one embodiment.

[0018] FIG. 10b is an example of a first input prompt requesting a detailed description of a drawing, according to one embodiment.

[0019] FIG. 10c is an example of a first input prompt requesting both a brief description and a detailed description of a drawing, according to one embodiment.

[0020] FIG. 11 is an example of a response to a first input prompt, according to one embodiment.

[0021] FIG. 12 is a flowchart illustrating a process by which an electronic device generates a first input prompt based on proficiency according to one embodiment.

[0022] FIG. 13 is an example of a third input prompt requesting a proficiency determination, according to one embodiment.

[0023] FIG. 14 is a flowchart illustrating a process by which an electronic device generates a second input prompt based on a style according to one embodiment.

[0024] FIGS. 15a, 15b, and 15c are examples of a second input prompt requesting the generation of an image, according to one embodiment.

[0025] FIG. 16 is a flowchart illustrating a process by which an electronic device generates a second input prompt based on style similarity according to one embodiment.

[0026] FIG. 17 is an example of description information for a drawing according to one embodiment.

[0027] Figures 18a and 18b are examples of images acquired based on style-specific similarity according to one embodiment.

[0028] FIG. 19 is a flowchart illustrating a process by which an electronic device according to one embodiment generates a second input prompt based on similarity by skill level.

[0029] FIG. 20 is an example of a user interface for explaining a process by which an electronic device determines similarity according to one embodiment.

[0030] FIGS. 21A, 21B, 21C, and 21D are examples of user interfaces for describing a process of displaying an image by an electronic device according to one embodiment.

[0031] FIG. 22a is an example of a first input prompt requesting a description of a drawing, according to one embodiment.

[0032] FIG. 22b is an example of a second input prompt requesting the generation of an image, according to one embodiment.

[0033] FIG. 23a is an example of a hardware configuration of an electronic device according to one embodiment.

[0034] Figure 23b is a block diagram of an electronic pen, which is an input means according to one embodiment.

[0035] FIG. 24 is an example of a hardware configuration of a foldable electronic device according to one embodiment.

[0036] FIGS. 25A and 25B are examples of drawings and images displayed on an electronic device according to one embodiment.

[0037] FIG. 26 is a diagram illustrating an external electronic device that interfaces with an electronic device according to one embodiment.

[0038] Fig. 27 is a flowchart illustrating a method for providing an image of an electronic device according to one embodiment.

[0039] FIG. 28 is an example of a user interface for explaining a process by which an electronic device receives a drawing via an electronic pen according to one embodiment.

[0040] FIG. 29 is an example of a user interface for describing a process by which an electronic device receives a drawing through an edge region according to one embodiment.

[0041] FIG. 30 is an example of a user interface for describing a process by which an electronic device receives a drawing through a designated application according to one embodiment.

[0042] FIG. 31 is an example of a user interface for explaining an image providing function of an electronic device according to one embodiment.

[0043] FIG. 32 is an example of a user interface for explaining a process by which an electronic device determines a style according to one embodiment.

[0044] FIG. 33 is an example of a user interface for explaining a process by which an electronic device determines similarity based on transparency according to one embodiment.

[0045] FIG. 34 is a diagram illustrating a system including a generative artificial intelligence model according to one embodiment.

[0046] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0047] Electronic devices can provide drawing functionality. This allows users to use the display surface of the electronic device like paper to naturally and conveniently draw. The electronic device can detect user input using an input tool (e.g., an electronic pen or finger) and process and / or display the corresponding data.

[0048] Meanwhile, electronic devices can use generative AI models to provide users with content (e.g., text, images, or other media) that responds to prompts.

[0049] When attempting to generate images using text prompts, it can be difficult for typical users to accurately describe the desired result in text or implement an appropriate prompt structure. Users may experience inconvenience as they must familiarize themselves with the complex parameters included in the prompts. This can lead to users receiving results that differ from their intended intent, resulting in lower user satisfaction.

[0050] Various embodiments of the present disclosure may provide an electronic device and method for providing an image that allows a user to express a desired direction through a simple drawing without having to learn a complex prompt structure or perform cumbersome text input.

[0051] Various embodiments of the present disclosure can provide an electronic device and method for providing an image that can implement an image that accurately reflects a user's intention while improving user convenience.

[0052] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with the electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (104) or the server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0053] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0054] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0055] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0056] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0057] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0058] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0059] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0060] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0061] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0062] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0063] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0064] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0065] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0066] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0067] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0068] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). The corresponding communication module within these communication modules can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can use subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196) to verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199).

[0069] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0070] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0071] According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0072] At least some of the above components may be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)).

[0073] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0074] FIG. 2 is a block diagram of an electronic device (200) according to one embodiment.

[0075] Referring to FIG. 2, an electronic device (200) according to one embodiment may include a touchscreen display (210), a processor (220), and a memory (230). The electronic device (200) may further include a communication circuit (240).

[0076] The electronic device (200) of FIG. 2 may correspond to the electronic device (101) illustrated in FIG. 1. The touchscreen display (210), processor (220), memory (230), and communication circuit (240) included in the electronic device (200) of FIG. 2 may correspond to the display module (160), processor (120), memory (130), and communication module (190) of FIG. 1, respectively.

[0077] The electronic device (200) may include additional components in addition to the components illustrated in FIG. 2. At least one of the components illustrated in FIG. 2 may be omitted. In some embodiments, some of the components illustrated in FIG. 2 may be integrated into one, or one component may be separated into two or more.

[0078] The touchscreen display (210), processor (220), memory (230), and communication circuit (240) included in the electronic device (200) may be electrically and / or operatively connected to each other to exchange signals (e.g., commands or data) with each other.

[0079] According to one embodiment, an electronic device (200) (e.g., electronic device (101) of FIG. 1) can perform an image providing function. For example, the electronic device (200) can perform the image providing function using at least one generative artificial intelligence (AI) model. For example, the electronic device (200) can execute an application (e.g., a drawing application, a note application) and perform the image providing function using the application. The application executed in the electronic device (200) can operate independently or in conjunction with an external electronic device (e.g., electronic device (102), electronic device (104) of FIG. 1, or server (108) of FIG. 1).

[0080] According to one embodiment, the touchscreen display (210) may include a touch panel for providing a touch function and a display panel for providing a display function. The laminated structure of the touchscreen display (210) may vary. For example, the touch panel and the display panel may be independently configured and then packaged as one. The touch panel may also be implemented as an in-cell type integrated with the pixel layer of the display panel.

[0081] In some embodiments, the touch panel may be composed of multiple layers. For example, the touch panel may include a first touch-sensitive layer for detecting touch input using a user's hand (e.g., a finger or nail) and a second touch-sensitive layer for detecting touch input using an electronic pen (e.g., the electronic pen (500) of FIG. 5).

[0082] However, the touch detection method and / or the layer structure of the touch panel are not limited thereto and may be modified in various ways. For example, touch input may be detected through a single touch detection layer regardless of the type of input tool (e.g., an electronic pen, a finger). As another example, the touch panel may include a first touch detection layer and a second touch detection layer, and each layer may be activated alternately depending on the situation. In this case, the second touch detection layer may be activated when the electronic pen mode is turned on or when the approach of the electronic pen is detected.

[0083] According to one embodiment, the touch panel may include one or more touch sensors configured to detect touch. According to various embodiments, the touch sensors may detect touch in at least one of a contact-type capacitive method, a pressure-type resistive method, an infrared sensing method, a surface ultrasonic conduction method, and / or a piezoelectric effect method. However, the touch sensing method is not limited thereto.

[0084] According to one embodiment, the communication circuit (240) may support a short-range wireless communication connection of the electronic device (200). For example, the communication circuit (240) may support a short-range wireless communication (e.g., Bluetooth, Bluetooth low energy (BLE), NFC, Wi-Fi direct, or infrared data association (IrDA)) connection between the electronic device (200) and an electronic pen (e.g., the electronic pen (500) of FIG. 5).

[0085] In one embodiment, the communication circuit (240) may support a long-distance wireless communication connection of the electronic device (200). For example, the communication circuit (240) may support a long-distance wireless communication (e.g., cellular communication, network communication such as the Internet) connection between the electronic device (200) and an external server (e.g., server (108) of FIG. 1).

[0086] According to one embodiment, the processor (220) may include at least one processor including processing circuitry. The processor (220) may execute various functions supported by the electronic device (200). The processor (220) may control at least some of the touchscreen display (210), the processor (220), the memory (230), and the communication circuit (240). The processor (220) may execute an application and control various hardware by executing code written in a programming language stored in the memory (230) of the electronic device (200). For example, the processor (220) may refer to a plurality of processors that collectively perform a plurality of operations by dividing them among the processors.

[0087] According to one embodiment, the processor (220) may include a driver circuit for touch detection. The driver circuit may be electrically connected to a touch panel. The driver circuit may detect a user's touch input (e.g., a touch-down, a long touch, a swipe, a drag, a touch release) received through the touch panel.

[0088] According to one embodiment, the driver circuit can identify data related to a user's touch input received on the touch panel using a touch sensor within the touch panel. For example, the driver circuit can identify data related to characteristics related to the user's touch input (e.g., at least one of whether a touch has been made, the location (coordinates) of the touch point, the touch area, the touch sensitivity, the movement distance, the touch holding time, and / or whether the touch has been released).

[0089] According to one embodiment, the driver circuit can drive (e.g., apply power to) a touch sensor within the touch panel, and identify an electrical value (e.g., at least one of a voltage value or a current value) or a change in the electrical value generated by the touch sensor based on an input received based on the applied power. The driver circuit can identify data related to a user's touch input based on the identified electrical value or change in the electrical value.

[0090] In some embodiments, the driver circuit may include a first driver circuit and a second driver circuit. For example, the driver circuit may be configured with a first driver circuit and a second driver circuit for detecting different types of touch inputs. The first driver circuit may be capable of detecting a touch input using a user's hand (e.g., a finger or a nail) in conjunction with a first touch-sensitive layer of the touch panel. The second driver circuit may be capable of detecting a touch input using an electronic pen (e.g., an electronic pen (500)) in conjunction with a second touch-sensitive layer of the touch panel. However, the structure of the driver circuit is not limited thereto. For example, a single driver circuit may detect a touch input through a single touch-sensitive layer regardless of the type of input tool (e.g., a finger or an electronic pen).

[0091] According to one embodiment, the memory (230) (e.g., the memory (130) of FIG. 1) may store instructions that, when individually or collectively executed by the processor (220), cause the electronic device (200) to perform various operations. For example, at least one processor (or one or more processors) within the processor (220) of the electronic device (200) may individually or collectively execute instructions stored in the memory (230) to perform an image providing function.

[0092] According to one embodiment, the processor (220) can display a user interface for receiving a drawing via the touchscreen display (210).

[0093] According to one embodiment, the processor (220) may display a user interface for receiving a drawing on the touchscreen display (210) when a triggering event occurs. For example, the triggering event may include at least one of a first event detecting a hovering input of an electronic pen (e.g., the electronic pen (500) of FIG. 5), a second event detecting a user input (e.g., a touch) for selecting a specified visual element within an edge region of the touchscreen display (210), or a third event detecting a user input (e.g., a touch) for selecting a specified visual element within an application execution screen.

[0094] According to one embodiment, the processor (220) can receive (or detect) a user's drawing in response to a first user input (e.g., a touch, a swipe, a drag, a gesture, a stroke input, a picture input, a handwriting input) via the touchscreen display (210).

[0095] According to one embodiment, the processor (220) can determine the style of the image to be generated (image style) based on the user's drawing.

[0096] According to one embodiment, the processor (220) may display a first interface element for selecting a style of an image to be generated from among a plurality of designated styles on the touchscreen display (210). The first interface element may be for selecting a style of an image to be generated (image style) from among a plurality of pre-designated styles (e.g., watercolor, animation, pencil, cartoon, neon, or photorealistic). The processor (220) may select a specific style (image style) from among the plurality of designated styles based on a user input to the first interface element.

[0097] According to one embodiment, the processor (220) may receive a second user input (e.g., a touch on a designated visual element) for generating an image corresponding to the user's drawing via the touchscreen display (210).

[0098] According to one embodiment, the processor (220) may obtain description information of the drawing (or description information about the drawing) using the drawing based on the second user input.

[0099] According to one embodiment, the processor (220) may obtain description information about the drawing by providing a first input prompt including the drawing to a first artificial intelligence model (e.g., the first artificial intelligence model (310) of FIG. 3) in response to a second user input for image generation.

[0100] According to one embodiment, the processor (220) can determine (or set) the similarity between the user's drawing and the image to be generated.

[0101] According to various embodiments, similarity may also be referred to by other terms of the same or similar concept, such as correlation, dependency, or drawing reflection ratio for an image.

[0102] In one embodiment, the similarity between the drawing (or source image) and the image to be generated (or output image) may have a set default value.

[0103] According to one embodiment, the similarity between the drawing and the image to be generated can be varied based on at least one of user input, skill level of the drawing, style of the image to be generated, or transparency of the drawing area.

[0104] In one embodiment, the processor (220) may determine the similarity between a drawing and an image to be generated based on user input. The processor (220) may display a second interface element on the touchscreen display (210) for selecting the similarity. The processor (220) may determine the similarity based on the user input for the second interface element.

[0105] In one embodiment, the processor (220) may adjust the similarity to a value lower than the default value based on a first user input (e.g., a swipe to the left) to the second interface element. For example, if the user is a relatively novice user who is not good at drawing, the user may use the second interface element to adjust the similarity between the user's drawing and the image to be generated to a value lower than the default value. Accordingly, an image may be generated (or acquired) that is relatively less influenced by the user's drawing in the image to be generated and relatively more dependent on the learning status of the artificial intelligence model.

[0106] In one embodiment, the processor (220) may adjust the similarity to a value higher than the default value based on a second user input (e.g., a swipe to the right) for the second interface element. For example, if the user is a relatively expert user who is familiar with drawing, the user may use the second interface element to adjust the similarity between the user's drawing and the image to be generated to a value higher than the default value. Accordingly, an image may be generated (or acquired) in which the user's drawing is relatively more reflected in the image to be generated, and which is relatively less dependent on the learning status of the artificial intelligence model.

[0107] According to one embodiment, the processor (220) can determine the similarity between the drawing and the image to be generated based on the style of the image to be generated (image style).

[0108] According to one embodiment, the processor (220) may display a first interface element for selecting an image style from among a plurality of styles on the touchscreen display (210). The processor (220) may set the similarity differently depending on the style selected by the user input for the first interface element. The processor (220) may set the length of the description information differently depending on the style selected by the user input for the first interface element.

[0109] According to one embodiment, the processor (220) may determine similarity based on the drawing skill level. The processor (220) may set the similarity level differently depending on the drawing skill level. For example, if the drawing skill level is low, the similarity may have a first value. If the drawing skill level is high, the similarity may have a second value higher than the first value.

[0110] According to one embodiment, the processor (220) may determine the similarity based on the transparency of the drawing area for receiving the drawing. For example, a reference image that the user can draw along may be displayed as a background image in the drawing area of ​​the touchscreen display (210). The processor (220) may adjust the transparency of the drawing area according to a user input (e.g., left and right swiping) for the third interface element. As the transparency of the drawing area increases (becomes more transparent), the drawing reflection ratio for the image to be generated may decrease, and the reference image reflection ratio for the image may increase. As the transparency of the drawing area decreases (becomes more opaque), the drawing reflection ratio for the image to be generated may increase, and the reference image reflection ratio for the image may decrease.

[0111] According to one embodiment, the processor (220) can obtain (or generate) an image corresponding to a drawing based on at least one of the drawing, description information about the drawing, similarity between the drawing and the image to be generated, style of the image to be generated (image style), skill in drawing, or transparency of the drawing area.

[0112] According to one embodiment, the processor (220) can obtain an image (an image corresponding to a drawing) generated based on a drawing and image style.

[0113] According to one embodiment, the processor (220) may obtain (or generate) an image corresponding to the drawing by providing an input prompt including drawing and image style information to an artificial intelligence model (e.g., the first artificial intelligence model (310) and / or the second artificial intelligence model (320) of FIG. 3). In one embodiment, the input prompt may further include at least a portion of reference text input by a user (e.g., text input through an input window), a reference image (e.g., a background image displayed in a drawing area, a captured image of a background application execution screen), form factor information of the electronic device (200), or form factor information of an external electronic device (e.g., the first external electronic device (2610), the second external electronic device (2620) of FIG. 26) connected to the electronic device (200) via short-range wireless communication.

[0114] According to one embodiment, the processor (220) can obtain an image (an image corresponding to the drawing) generated based on at least a portion of the description information about the drawing and the similarity between the drawing and the image to be generated.

[0115] According to one embodiment, the processor (220) may obtain description information about the drawing by providing a first input prompt including a drawing to a first artificial intelligence model (e.g., the first artificial intelligence model (310) of FIG. 3). The processor (220) may obtain (or generate) an image corresponding to the drawing by providing a second input prompt including at least a portion of the description information about the drawing and a similarity to the drawing to the artificial intelligence model (e.g., the second artificial intelligence model (320) of FIG. 3). In one embodiment, the second input prompt may further include at least a portion of the drawing, image style information, reference text input by a user, a reference image, form factor information of the electronic device (200), or form factor information of an external electronic device connected to the electronic device (200) via short-range wireless communication.

[0116] According to one embodiment, the processor (220) can display an image obtained based on the drawing (an image corresponding to the drawing) through the touchscreen display (210).

[0117] According to one embodiment, the electronic device (200) may be a fixed form factor type electronic device (e.g., the electronic device (2300) of FIG. 23A).

[0118] According to one embodiment, the electronic device (200) may be an electronic device of a flexible form factor type whose form factor can be transformed (e.g., a foldable electronic device (2400) of FIG. 24). The processor (220) of the electronic device (200) may include form factor information of the foldable electronic device in an input prompt to an artificial intelligence model (e.g., the first artificial intelligence model (310) and / or the second artificial intelligence model (320) of FIG. 3). The processor (220) may acquire a plurality of images corresponding to the form factor information of the foldable electronic device in response to the input prompt.

[0119] According to one embodiment, the processor (220) may include form factor information of an external electronic device (or target electronic device) in an input prompt to an artificial intelligence model (e.g., the first artificial intelligence model (310) and / or the second artificial intelligence model (320) of FIG. 3). For example, the external electronic device may be the first external electronic device (2610) and / or the second external electronic device (2620) of FIG. 26. The processor (220) may obtain an image corresponding to the form factor information of the external electronic device in response to the input prompt.

[0120] According to one embodiment, the processor (220) may display a user interface including at least one keyword related to description information about a drawing via the touchscreen display (210). The processor (220) may receive a user input for the at least one keyword via the user interface. The processor (220) may edit (e.g., delete, modify, add) the description information about the drawing based on the user input. The processor (220) may acquire an image based on at least a portion of the edited description information.

[0121] According to one embodiment, the processor (220) may store an image corresponding to a drawing in the memory (230). When storing the image corresponding to the drawing, the processor (220) may store at least one keyword related to the description information of the drawing in the metadata of the image. In one embodiment, the metadata of the image may further include detailed information such as the location (or place), date, and time of the electronic device (200) corresponding to the time of storing the image.

[0122] According to one embodiment, the processor (220) may provide a search function for a plurality of images corresponding to drawings using at least one keyword stored in image-specific metadata. The processor (220) may additionally provide a search function for the plurality of images using image-specific detailed information.

[0123] According to one embodiment, the electronic device (200) can obtain (or generate) an image corresponding to a drawing using at least one artificial intelligence model (e.g., the first artificial intelligence model (310) and / or the second artificial intelligence model (320) of FIG. 3).

[0124] According to one embodiment, the first artificial intelligence model (310) and / or the second artificial intelligence model (320) of FIG. 3 may be implemented in hardware and / or software to perform a predetermined function.

[0125] According to one embodiment, the electronic device (200) of FIG. 2 may be configured to include at least a portion of the first artificial intelligence model (310) of FIG. 3 or the second artificial intelligence model (320) of FIG. 3. The first artificial intelligence model (310) and / or the second artificial intelligence model (320) may be included in the electronic device (200) (e.g., memory (230)) in the form of an on-device artificial intelligence model, but is not limited thereto. For example, at least a portion of the first artificial intelligence model (310) or the second artificial intelligence model (320) may be included in an external server (e.g., server (108) of FIG. 1). In some embodiments, the functions of the first artificial intelligence model (310) of FIG. 3 and the functions of the second artificial intelligence model (320) may be provided through a single integrated artificial intelligence model.

[0126] FIG. 3 is a diagram illustrating a plurality of artificial intelligence models (310, 320) that are linked to an electronic device (200) according to one embodiment.

[0127] According to one embodiment, the electronic device (200) can perform an image providing function using a first artificial intelligence model (310) and / or a second artificial intelligence model (320).

[0128] Referring to FIG. 3, the first artificial intelligence model (310) may be a generative artificial intelligence model trained for the purpose of generating and / or providing descriptive information (303) for a drawing (301) (or source image). For example, the first artificial intelligence model (310) may be a large multimodal model (LMM) artificial intelligence model.

[0129] According to one embodiment, the electronic device (200) can obtain description information (303) about the drawing (301) by providing a first input prompt (302) including the drawing (301) to a first artificial intelligence model (310).

[0130] In one embodiment, the first artificial intelligence model (310) may receive a first input prompt (302) comprising a drawing (301). The first input prompt (302) may further include a text prompt for the drawing (301), such as text requesting a brief description and / or detailed description of the drawing (301), or text indicating the length of the description information.

[0131] According to one embodiment, the first artificial intelligence model (310) can generate description information (303) about the drawing (301) based on the drawing (301) and / or text prompt included in the first input prompt (302). The first artificial intelligence model (310) can transmit the generated description information (303) to the electronic device (200).

[0132] Referring to FIG. 3, the second artificial intelligence model (320) may be a generative artificial intelligence model trained for the purpose of generating and / or providing an image (305) (or output image) corresponding to a drawing (301) (or source image). For example, the second artificial intelligence model (320) may be an image generation artificial intelligence model (e.g., a text to image diffusion model AI model).

[0133] According to one embodiment, the electronic device (200) can obtain an image (305) corresponding to the drawing (301) by providing a second input prompt (304) including at least a portion of the description information (303) about the drawing (301) to the second artificial intelligence model (320).

[0134] In one embodiment, the second artificial intelligence model (320) may receive a second input prompt (304) that includes at least a portion of the description information (303) about the drawing (301). The second input prompt (304) may further include a similarity between the drawing (301) and the image (305) to be generated.

[0135] In one embodiment, the second input prompt (304) may further include at least some of the drawing (301), image style information, reference text input by the user, reference image, form factor information of the electronic device (200), or form factor information of an external electronic device connected to the electronic device (200) via short-range wireless communication.

[0136] In one embodiment, the second artificial intelligence model (320) can generate an image (305) corresponding to the drawing (301) based on the second input prompt (304). The second artificial intelligence model (320) can transmit the image (305) generated in response to the second input prompt (304) to the electronic device (200).

[0137] According to one embodiment, the functions of the first artificial intelligence model (310) and the second artificial intelligence model (320) of FIG. 3 may be provided through a single integrated artificial intelligence model. In this case, the integrated artificial intelligence model may skip the process of generating description information about a drawing and then transmitting it to an electronic device (200) (e.g., processor (220)). The integrated artificial intelligence model may directly generate an image corresponding to the drawing using the generated description information internally. In some embodiments, the drawing may be used as is to generate an image corresponding to the drawing without a process of generating description information about the drawing.

[0138] According to one embodiment, the electronic device (200) may obtain (or generate) an image corresponding to the drawing by providing an input prompt including at least one of a drawing (301), description information about the drawing (303), similarity between the drawing (301) and an image to be generated, or image style information to an artificial intelligence model (e.g., a second artificial intelligence model (320) or an integrated artificial intelligence model). In one embodiment, the input prompt may further include at least some of reference text input by a user, a reference image, form factor information of the electronic device (200), or form factor information of an external electronic device connected to the electronic device (200) via short-range wireless communication.

[0139] FIG. 4 is a flowchart illustrating an image providing method of an electronic device (200) according to one embodiment.

[0140] According to one embodiment, the electronic device (200) can generate an image corresponding to the drawing based on at least some of the user's drawing, description information about the drawing, and similarity.

[0141] Referring to FIG. 4, an image providing method (400) of an electronic device (200) according to one embodiment may include operations 410, 420, 430, 440, 450, and 460. In some embodiments, some of the illustrated operations may be omitted, some operations may be integrated, the order of some operations may be changed, or other operations may be added.

[0142] In operation 410, the electronic device (200) (e.g., the processor (220)) may display a user interface for drawing reception (e.g., the first screen (510) of FIG. 5) on the touchscreen display (210). The electronic device (200) may receive (or detect) a drawing (e.g., a drawing (551, 552) of FIG. 5) according to a first user input (e.g., a touch, a swipe, a drag, a gesture, a stroke input, a picture input, a handwriting input) through the touchscreen display (210).

[0143] According to one embodiment, a drawing may include at least one line (or stroke) and / or shape. The drawing may include a first drawing corresponding to a sketch (or picture) (e.g., a first drawing (551) of FIG. 5 ). The drawing may further include a second drawing corresponding to text (e.g., a second drawing (552) of FIG. 5 ). The drawing may further include color data.

[0144] According to one embodiment, the electronic device (200) can receive a drawing (e.g., drawing (551, 552) of FIG. 5) input by a user through a user interface (e.g., second screen (520) of FIG. 5) displayed on a touchscreen display (210).

[0145] According to one embodiment, the electronic device (200) (e.g., the processor (220)) can receive a drawing that a user inputs on the touchscreen display (210) using an electronic pen (e.g., the electronic pen (500) of FIG. 5) (or a finger). The electronic device (200) can display the received drawing on the touchscreen display (210) in real time. The electronic device (200) can detect the drawing drawn by the user through a touch sensor in the touchscreen display (210). However, the present invention is not limited thereto. For example, the electronic device (200) can detect the drawing drawn by the user by sensing a user gesture using a motion sensor (e.g., a motion sensor, a gesture sensor, a proximity sensor in the sensor module (176)) and / or a camera (e.g., at least one camera in the camera module (180)). For example, the electronic device (200) can also detect a drawing drawn by the user using a separate input pad (drawing pad) and / or an input tool such as a mouse.

[0146] In operation 420, the electronic device (200) (e.g., the processor (220)) may receive (or detect) a second user input for generating an image corresponding to the drawing via the touchscreen display (210). For example, the electronic device (200) may receive a second user input for generating an image (e.g., an image corresponding to the drawing (551, 552)) when the user touches an image generation button (541) within a user interface such as the second screen (520) of FIG. 5.

[0147] In operation 430, the electronic device (200) (e.g., processor (220)) may obtain description information of the drawing (or description information about the drawing) using the drawing (e.g., drawing (551, 552) of FIG. 5) based on the second user input received through operation 420.

[0148] In one embodiment, the description information for the drawing may include text describing the drawing. For example, the text describing the drawing may include at least a portion of one or more keywords related to the drawing or an object within the drawing (e.g., an animal object, an object object, a person object, a background object, a handwriting object), a natural language sentence(s) related to the one or more keywords, or a natural language sentence(s) describing the context of the drawing or an object within the drawing.

[0149] According to one embodiment, at operation 430, the electronic device (200) may obtain description information about the drawing by providing a first input prompt including the drawing to a first artificial intelligence model (e.g., the first artificial intelligence model (310) of FIG. 3) in response to a second user input (e.g., a touch on the image generation button (541) of FIG. 5) received via operation 420.

[0150] According to one embodiment, the electronic device (200) may display a first interface element (e.g., the first interface element (511) of FIG. 5) on the touchscreen display (210) for selecting a style of an image to be generated from among a plurality of styles. The first interface element may be for selecting a style of an image to be generated (image style) from among a plurality of pre-designated styles (e.g., watercolor, animation, pencil, cartoon, neon, or photorealistic). The electronic device (200) may determine the length of the description information based on the style selected by the user input to the first interface element. For example, if the style selected by the user is the first style (e.g., cartoon or neon, where intuitiveness is important), a brief description may be advantageous for generating the image. In this case, the electronic device (200) may set the length of the description information to a relatively short first length, thereby allowing a brief version of the description information to be acquired. For example, if the style selected by the user is a second style (e.g., watercolor or pencil), where precise detail is important, the more detailed the description, the higher the quality of the generated image. In this case, the electronic device (200) can set the length of the description information to a relatively long second length, thereby enabling the acquisition of a detailed version of the description information.

[0151] In one embodiment, at least some of the descriptive information about the drawing may be input by the user. For example, if the user inputs text using a keyboard in an input window within a user interface displayed on the touchscreen display (210), the text may be used as part of the descriptive information about the drawing.

[0152] At operation 440, the electronic device (200) (e.g., processor (220)) may determine (or identify) a similarity between the drawing and the image to be generated.

[0153] According to various embodiments, similarity may also be referred to by other terms of the same or similar concept, such as correlation, dependency, or drawing reflection ratio for an image.

[0154] The term "similarity" has been chosen for convenience of explanation, and it is apparent to a person of ordinary skill in the art to which the present disclosure pertains that it may be referred to by various terms indicating the degree to which a user's drawing is taken into account (or reflected) in an image generated by artificial intelligence.

[0155] In one embodiment, the similarity may be a value indicating how similar the drawing (or source image) is to the image to be generated (or output image). The similarity may be a value indicating the degree to which the drawing and the image to be generated are related (correlation). The similarity may be a value indicating the degree to which the image to be generated is influenced and / or dependent on the drawing (dependence). The similarity may be a value indicating the ratio of the drawing to the image to be generated.

[0156] In one embodiment, the similarity may be represented as a weight value in the range of greater than 0 and less than 1. The similarity may be represented as a percentage value in the range of greater than 0% and less than 100%.

[0157] According to one embodiment, the closer the similarity is to 0 or 0%, the more the image (or output image) can be obtained with a relatively large difference (or distance, e.g., Euclidean distance, structural similarity index (SSIM), histogram difference, feature information difference) from the drawing (or source image). The closer the similarity is to 1 or 100%, the more the image (or output image) can be obtained with a relatively small difference (or distance) from the drawing (or source image).

[0158] In one embodiment, the similarity may be a value representing a drawing reflection ratio for the image to be generated. The similarity may be related to a ratio in which the drawing is reflected in the image or a probability that characteristic data of the drawing is included in the image. For example, the characteristic data of the drawing may include data regarding at least some of the number of strokes (or lines) in the drawing, properties of the strokes (e.g., length, position, shape, thickness, or complexity), the outline of an object in the drawing, the size of the object, the position of the object, the point / line / plane forming the object, whether it is colored, or the color.

[0159] In one embodiment, as the similarity decreases, the probability that the image will contain the drawing reflection ratio or the drawing characteristic data may decrease. As the similarity increases, the probability that the image will contain the drawing reflection ratio or the drawing characteristic data may also increase.

[0160] In one embodiment, the similarity between the drawing and the image to be generated may be determined as a predetermined default value (e.g., one of 0.4, 0.6, 0.8, 1).

[0161] In one embodiment, the similarity between the drawing and the image to be generated may be determined based on at least one of user input, skill level of the drawing, style of the image to be generated, or transparency of the drawing area.

[0162] According to one embodiment, the similarity between a drawing and an image to be generated may be determined based on a user input. The electronic device (200) may display a user interface including a second interface element (e.g., the second interface element (2020) of FIG. 20) for selecting a similarity on the touchscreen display (210). The electronic device (200) may determine the similarity based on a user input for the second interface element. The electronic device (200) may adjust the similarity to a lower value than a default value according to the user input for the second interface element. For example, if the user selects a lower value (e.g., 0.4 or lower) among the low, medium, and high ranges for the similarity using the second interface element (2020) of FIG. 20, the similarity may be adjusted to a first value (e.g., 0.4) that is lower than the default value (e.g., 0.6) in the absence of a separate user input for the second interface element (2020). For example, if a user selects a high range value for the similarity (e.g., 0.8 or higher) using the second interface element (2020) of FIG. 20, the similarity may be adjusted to a second value higher than the default value (e.g., 0.6) (e.g., 0.8).

[0163] According to one embodiment, the similarity between the drawing and the image to be generated may be determined based on the style of the image to be generated. The electronic device (200) may determine the similarity differently depending on the style of the image to be generated.

[0164] According to one embodiment, the electronic device (200) may display a user interface including a first interface element (e.g., the first interface element (2010) of FIG. 20) for selecting a style of an image to be generated on the touchscreen display (210). The electronic device (200) may select a style of an image to be generated according to a user input to the first interface element. The electronic device (200) may set a similarity level differently according to the selected style. For example, when the style selected by the user is the first style (e.g., a cartoon or neon requiring intuitive description), the electronic device (200) may set the similarity level between the drawing and the image to be generated to a first value (e.g., 0.4) that falls within a relatively low range. For example, if the style selected by the user is a second style (e.g., watercolor or pencil that requires precise drawing), the electronic device (200) may set the similarity between the drawing and the image to be generated to a second value (e.g., 0.8) that is within a relatively high range and higher than the first value.

[0165] In one embodiment, the similarity between a drawing and an image to be generated may be determined based on the drawing skill level. The electronic device (200) may set the similarity between the drawing and the image to be generated differently depending on the drawing skill level.

[0166] In one embodiment, the higher the drawing skill, the higher the similarity between the drawing and the image to be generated may be set to. The lower the drawing skill, the lower the similarity between the drawing and the image to be generated may be set to.

[0167] In one embodiment, when the drawing skill is low, the similarity between the drawing and the image to be generated may have a first value (e.g., 0.4). When the drawing skill is high, the similarity between the drawing and the image may have a second value (e.g., 0.8) that is higher than the second value.

[0168] According to one embodiment, the electronic device (200) may determine the similarity between the drawing and the image to be generated based on the transparency of the drawing area for receiving the drawing. For example, a reference image that the user can draw along may be displayed as a background image in the drawing area of ​​the touchscreen display (210). In this state, as the transparency of the drawing area increases (becomes more transparent), the reflection ratio of the drawing for the image to be generated may decrease, and the reflection ratio of the reference image for the image may increase. As the transparency of the drawing area decreases (becomes more opaque), the reflection ratio of the drawing for the image to be generated may increase, and the reflection ratio of the reference image for the image may decrease.

[0169] At operation 450, the electronic device (200) (e.g., processor (220)) may obtain an image (e.g., image (560) of FIG. 5) generated based on at least a portion of the description information obtained through operation 430 and the similarity between the drawing and the image determined through operation 440.

[0170] According to one embodiment, the electronic device (200) may obtain description information about the drawing by providing a first input prompt including a drawing (e.g., a drawing (551, 552) of FIG. 5) to a first artificial intelligence model (e.g., the first artificial intelligence model (310) of FIG. 3) in response to a second user input for generating an image (e.g., a touch on an image generation button (541)). The electronic device (200) may obtain an image corresponding to the drawing by providing a second input prompt including at least a portion of the description information and a similarity to a second artificial intelligence model (e.g., the second artificial intelligence model (320) of FIG. 3).

[0171] According to one embodiment, the similarity may have at least one of a predetermined default value, a user-defined value, a value corresponding to the style of the image to be generated (or the image style), a value corresponding to the drawing skill, and a value corresponding to the transparency of the drawing area.

[0172] According to one embodiment, the second input prompt may further include at least some of the drawing, image style information, reference text input by the user (e.g., text input through an input window), reference image (e.g., background image displayed in a drawing area, captured image for a background application execution screen), form factor information of the electronic device (200), or form factor information of an external electronic device (e.g., first external electronic device (2610), second external electronic device (2620) of FIG. 26) connected to the electronic device (200) via short-range wireless communication.

[0173] According to one embodiment, the electronic device (200) may display a user interface including at least one keyword related to description information about a drawing via a touchscreen display (210). The electronic device (200) may receive a user input for the at least one keyword via the user interface. The electronic device (200) may edit the description information based on the user input. In this case, an image corresponding to the drawing may be acquired (or generated) based on at least a portion of the edited description information.

[0174] In operation 460, the electronic device (200) can display an image obtained through operation 450 (e.g., image (560) of FIG. 5) through the touchscreen display (210).

[0175] FIG. 5 is an example of a user interface for explaining an image providing method of an electronic device (200) according to one embodiment.

[0176] Referring to FIG. 5, the electronic device (200) can display user interfaces, such as a first screen (510), a second screen (520), and a third screen (530), via a touchscreen display (210). The first screen (510), the second screen (520), and the third screen (530) may be for an image providing function. For example, the first screen (510), the second screen (520), and the third screen (530) may be execution screens of a designated application (e.g., a drawing application, a note application) running on the electronic device (200).

[0177] The first screen (510) may be a user interface including a drawing area for receiving a drawing (551, 552) from a user.

[0178] The second screen (520) may be a user interface that displays in real time a drawing (551, 552) drawn by the user in the drawing area with an electronic pen (500) (or finger).

[0179] The second screen (520) may include an image creation button (541).

[0180] The third screen (530) may be a user interface that displays an image (560) acquired (or generated) based on a drawing (551, 552). When the user touches the image generation button (541), the electronic device (200) may acquire an image (560) corresponding to the drawing (551, 552) input by the user and display the acquired image (560).

[0181] According to one embodiment, the electronic device (200) can receive drawings (551, 552) input by a user through a drawing area displayed through a touchscreen display (210). The drawings (551, 552) can include a first drawing (551) and / or a second drawing (552). The first drawing (551) can correspond to a sketch (or picture). For example, the sketch can include at least one line (or stroke) and / or shape. The second drawing (552) can correspond to text. For example, the text can include at least one of handwriting, letters, symbols, or memos input by the user.

[0182] In one embodiment, the electronic device (200) can use the second drawing (552) together with the first drawing (551) to generate an image. For example, if a user sketches a cat and then writes “Russian” in handwriting, the electronic device (200) can obtain an image (560) of a Russian Blue cat using the first drawing (551) of the cat and the second drawing (552) of “Russian.” As another example, if a user draws a triangle over a square to represent a house and then writes “brick” in handwriting, the electronic device (200) can obtain an image of a brick house using the first drawing representing the house and the second drawing of the brick.

[0183] FIG. 6 is a flowchart illustrating a method in which an electronic device (200) according to one embodiment provides an image using multiple artificial intelligence models (310, 320).

[0184] Referring to FIG. 6, an image providing method (600) of an electronic device (200) according to one embodiment may include operations 610, 620, 630, 640, 650, 660, and 670.

[0185] According to one embodiment, at least some of the operations illustrated in FIG. 6 may correspond to the operations illustrated in FIG. 4. For example, operation 610 of FIG. 6 may correspond to operation 410 of FIG. 4. Operations 630 and 640 of FIG. 6 may correspond to operation 430 of FIG. 4. Operations 650 and 660 of FIG. 6 may correspond to operation 450 of FIG. 4. Operation 670 of FIG. 6 may correspond to operation 460 of FIG. 4. At least some of the operations illustrated in FIG. 6 may be performed in combination with at least some of the operations illustrated in FIG. 4.

[0186] In operation 610, the electronic device (200) may receive a drawing according to a first user input through a touchscreen display (210) displaying a user interface (e.g., the first screen (710) of FIG. 7A).

[0187] According to one embodiment, the drawing may include at least one line (or stroke) and / or shape input by the user. The drawing may include a first drawing corresponding to a sketch (or drawing). The drawing may further include a second drawing corresponding to text. The drawing may further include color data.

[0188] In operation 620, the electronic device (200) can determine the style (image style) of the image to be generated based on the drawing.

[0189] According to one embodiment, the electronic device (200) may provide a list of a plurality of styles through a user interface (e.g., a first interface element (721) in a second screen (720) of FIG. 7B ) displayed on a touchscreen display (210), thereby allowing a user to select one of the plurality of styles. For example, the plurality of styles may include two or more of watercolor, animation, pencil, cartoon, neon, or realism. The electronic device (200) may determine a style selected by the user from among the plurality of styles as the style of an image to be generated.

[0190] At operation 630, the electronic device (200) may transmit a first input prompt to a first artificial intelligence model (e.g., the first artificial intelligence module (310) of FIG. 3).

[0191] In one embodiment, the first input prompt may be for requesting a description (e.g., a brief description and / or a detailed description) of the drawing received via operation 610. For example, the first input prompt may be transmitted in response to a user touching the image creation button (741) of FIG. 7C . The first input prompt may include the drawing received via operation 610.

[0192] In one embodiment, the first input prompt may include a length of the description information (e.g., a fixed default length).

[0193] According to one embodiment, the length of the description information included in the first input prompt (e.g., a first length for a detailed description, or a second length for a brief description) may be set differently depending on the style of the image to be generated. The electronic device (200) may set the length of the description information differently depending on the style of the image to be generated.

[0194] According to one embodiment, the electronic device (200) may display a user interface including a first interface element (e.g., the first interface element (721) of FIG. 7B) for selecting a style of an image to be generated on the touchscreen display (210). The electronic device (200) may select a style of an image to be generated based on a user input to the first interface element. The electronic device (200) may determine the length of the description information based on the selected style. The electronic device (200) may set the length of the description information differently depending on the selected style. For example, if the style selected by the user is the first style (e.g., a cartoon or neon for which intuitiveness is important), a brief description may be advantageous for image generation. In this case, the electronic device (200) may request a brief description by setting the length of the description information to a relatively short first length. For example, if the style selected by the user is a second style (e.g., watercolor or pencil, where precise detail is important), the more detailed the description, the higher the quality of the generated image. In this case, the electronic device (200) can request a detailed description by setting the length of the description information to a relatively long second length.

[0195] At operation 640, the electronic device (200) may receive description information about the drawing from a first artificial intelligence model (e.g., the first artificial intelligence module (310) of FIG. 3) in response to the first input prompt transmitted at operation 630.

[0196] At operation 650, the electronic device (200) may transmit a second input prompt to a second artificial intelligence model (e.g., the second artificial intelligence module (320) of FIG. 3).

[0197] In one embodiment, the second input prompt may be for requesting the generation of an image using the description information received through operation 640. The second input prompt may include at least a portion of the description information about the drawing, and a similarity between the drawing and the image to be generated. For example, the similarity may have at least one of a predetermined default value, a user-defined value, a value corresponding to the style of the drawing, a value corresponding to the skill level of the drawing, and a value corresponding to the transparency of the drawing area.

[0198] According to one embodiment, the second input prompt may further include at least some of a drawing, image style information, reference text input by a user, a reference image, form factor information of the electronic device (200), or form factor information of an external electronic device connected to the electronic device (200) via short-range wireless communication.

[0199] According to one embodiment, the electronic device (200) may fully include the description information received via operation 640 in the second input prompt.

[0200] According to one embodiment, the electronic device (200) may include only a portion of the description information about the drawing received through operation 640 in the second input prompt and filter out the remainder of the description information. For example, the electronic device (200) may delete or modify unnecessary text (e.g., words and / or sentences) from the description information about the drawing before transmitting the second input prompt. For example, the electronic device (200) may delete or modify text that has a general or comprehensive meaning. For example, the electronic device (200) may store a list of texts to be filtered (e.g., a simple, a line drawing of, a sketch of, a painting of, a drawing of, black, white, the doodle) in the memory (230) and delete or modify one or more words and / or sentences from the description information about the drawing based on the list of texts.

[0201] At operation 660, the electronic device (200) may receive an image corresponding to the drawing from a second artificial intelligence model (e.g., the second artificial intelligence module (320) of FIG. 3) in response to the second input prompt transmitted at operation 650.

[0202] In operation 670, the electronic device (200) can display the image acquired through operation 660 through the touchscreen display (210).

[0203] FIGS. 7A, 7B, and 7C are examples of user interfaces for explaining a process (e.g., operation 620 of FIG. 6) in which an electronic device (200) determines a style according to one embodiment.

[0204] Referring to FIGS. 7A, 7B, and 7C, the electronic device (200) can display user interfaces such as a first screen (710), a second screen (720), a third screen (730), and a fourth screen (740) for style determination.

[0205] Referring to FIG. 7A, the electronic device (200) can display a first screen (710) through a touchscreen display (210) including a touchscreen (211). The first screen (710) can be a user interface including a drawing area for receiving a sketch (711) (or drawing) from a user. When a user inputs a sketch (711) into the drawing area of ​​the first screen (710) using an electronic pen (712) (or a finger), the electronic device (200) can receive the sketch (711).

[0206] Referring to FIG. 7B, the second screen (720) may be a user interface including a first interface element (721). The first interface element (721) may be for selecting a style of an image to be generated. The first interface element (721) may provide a list of multiple styles, thereby allowing the user to select one of the multiple styles. For example, when the user selects a main menu (e.g., a dialog box) corresponding to the first interface element (721) in the second screen (720), a drop-down menu window (731) may appear, as in the third screen (730). The drop-down menu window (731) may display a list of submenus representing the multiple styles. For example, the multiple styles may include at least some of water color, animation, pencil drawing, cartoon, or neon (3D neon).

[0207] When a user selects one of a plurality of styles included in a drop-down menu window (731) in a third screen (730), the electronic device (200) can determine the selected style as the style of an image to be generated based on the sketch (711).

[0208] Referring to FIG. 7c, the fourth screen (740) may be a user interface including an image creation button (741). The image creation button (741) may be an interface element for requesting image creation. When a user selects a desired style through the first interface element (721) of FIG. 7b and then touches the image creation button (741) of FIG. 7c, the electronic device (200) may obtain an image corresponding to the sketch (711). The image may be an image of the style selected through the first interface element (721).

[0209] FIG. 8 is a flowchart illustrating a process by which an electronic device (200) generates a first input prompt according to one embodiment.

[0210] According to one embodiment, a process (800, e.g., operation 630 of FIG. 6 ) in which an electronic device (200) transmits a first input prompt may include operations 810 and 820. Operations 810 and 820 may be for generating a first input prompt requesting description information about a drawing.

[0211] Referring to FIG. 8, in operation 810, the electronic device (200) may convert a drawing input by a user (e.g., a sketch (711) of FIG. 7a) into image data of a specified first format. For example, in response to a user input to an image generation button (741) of FIG. 7c, the electronic device (200) may convert the sketch (711) into image data of a first format (e.g., bmp data).

[0212] In operation 820, the electronic device (200) can convert the image data of the first format converted through operation 810 into string data of a specified second format. For example, the electronic device (200) can convert and / or encode the image data of the first format (e.g., bmp data) converted from a drawing (e.g., sketch (711)) into string data of the second format (e.g., base64 type string data).

[0213] According to one embodiment, the electronic device (200) can generate a first input prompt that includes both image data (image prompt) in a first format for the drawing and string data (text prompt) in a second format for the drawing. The electronic device (200) can receive description information about the drawing from the first artificial intelligence model (e.g., the first artificial intelligence model (310) of FIG. 3) by providing the first input prompt to the first artificial intelligence model.

[0214] FIG. 9 is a flowchart illustrating a process by which an electronic device (200) generates a first input prompt based on a style according to one embodiment.

[0215] According to one embodiment, a process (900, e.g., operation 630 of FIG. 6 ) in which the electronic device (200) transmits a first input prompt may include operations 910, 920, 930, 940, and 950. Operations 910, 920, 930, 940, and 950 may be for generating a first input prompt requesting description information about a drawing.

[0216] Referring to FIG. 9, in operation 910, the electronic device (200) can identify the style of an image (or output image) to be generated based on a drawing (or source image). For example, the electronic device (200) can identify a style selected by a user through a user interface (e.g., the first interface element (721) of FIG. 7b) among a plurality of pre-designated styles.

[0217] At operation 920, the electronic device (200) can identify whether the style identified through operation 910 belongs to a first style (e.g., cartoon, neon requiring intuitive description) or a second style (e.g., watercolor, pencil requiring precise description).

[0218] If the style identified through operation 920 is a first style (e.g., comics, neon, where intuitiveness is important), the electronic device (200) may proceed to operation 930. In operation 930, the electronic device (200) may select a prompt requesting a brief description from among a plurality of prompts (e.g., text prompts) pre-stored in the memory (230).

[0219] If the style identified through operation 920 is a second style (e.g., watercolor or pencil requiring detailed description), the electronic device (200) may proceed to operation 940. In operation 940, the electronic device (200) may select a prompt requesting detailed description from among a plurality of pre-stored prompts (e.g., text prompts).

[0220] At operation 950, the electronic device (200) may transmit the prompt selected through operation 930 or operation 940 to the first artificial intelligence model (310) as a first input prompt requesting a description of the drawing.

[0221] FIG. 10A is an example of a first input prompt requesting a brief description of a drawing, according to one embodiment.

[0222] According to one embodiment, the first input prompt requesting a description of the drawing may include a first text prompt (1010) requesting a brief description (e.g., the prompt selected at 930 of FIG. 9).

[0223] Referring to FIG. 10A, the first text prompt (1010) may include one or more natural language sentences. The first text prompt (1010) may include a phrase (1011) requesting a brief description.

[0224] FIG. 10b is an example of a first input prompt requesting a detailed description of a drawing, according to one embodiment.

[0225] In one embodiment, a first input prompt requesting a description of a drawing may include a second text prompt (1020) requesting a detailed description (e.g., the prompt selected at 940 of FIG. 9).

[0226] Referring to FIG. 10b, the second text prompt (1020) may include one or more natural language sentences. The second text prompt (1020) may include a phrase (1021) requesting a detailed description.

[0227] FIG. 10c is an example of a first input prompt requesting both a brief description and a detailed description of a drawing, according to one embodiment.

[0228] In one embodiment, a first input prompt requesting a description of a drawing may include a third text prompt (1030) requesting both a brief description and a detailed description of a single drawing.

[0229] Referring to FIG. 10c, the third text prompt (1030) may include one or more natural language sentences. The third text prompt (1030) may include a phrase (1031) requesting a detailed description of the drawing, and a phrase (1032) requesting a brief description of the drawing.

[0230] FIG. 11 is an example of a response (1101) to a first input prompt according to one embodiment.

[0231] According to one embodiment, the electronic device (200) may transmit a first input prompt requesting a description of the sketch (1110) to the first artificial intelligence model (310).

[0232] The electronic device (200) may receive a response (1101) to the first input prompt from the first artificial intelligence model (310). The response (1101) may include a sketch (1110) (or a source image, drawing). The response (1101) may include a brief description portion (1120) and / or a detailed description portion (1130) of the sketch (1110).

[0233] FIG. 12 is a flowchart illustrating a process by which an electronic device (200) generates a first input prompt based on proficiency according to one embodiment.

[0234] According to one embodiment, a process (1200, e.g., operation 630 of FIG. 6 ) in which an electronic device (200) transmits a first input prompt may include operations 1210, 1220, 1230, 1240, and 1250. Operations 1210, 1220, 1230, 1240, and 1250 may be for generating a first input prompt requesting description information about a drawing.

[0235] Referring to FIG. 12, at operation 1210, the electronic device (200) can determine the skill level of the drawing (or source image).

[0236] According to one embodiment, the electronic device (200) may transmit a request (e.g., the third input prompt (1310) of FIG. 13) to an artificial intelligence model (e.g., the first artificial intelligence model (310) of FIG. 3) to inquire about the drawing skill level, and receive the skill level in response to the request.

[0237] According to one embodiment, the electronic device (200) can score and / or determine the skill level (e.g., either low level or high level) of the drawing by analyzing characteristic data of the drawing. For example, the characteristic data of the drawing can include data regarding at least some of the number of stroke lines in the drawing, properties of the stroke lines (e.g., length, position, shape, thickness, or complexity), outline lines of objects in the drawing, sizes of the objects, positions of the objects, points / lines / planes forming the objects, or whether the objects are colored.

[0238] According to one embodiment, the electronic device (200) may determine the drawing skill level based on user information stored in the memory (230). For example, the user information may include at least one of a user level (e.g., an amateur level or a professional level), the user's subscription period to a drawing application, and the user's accumulated drawing amount.

[0239] In operation 1220, the electronic device (200) can identify whether the drawing skill determined through operation 1210 is higher than a predefined reference value.

[0240] As a result of the identification of operation 1220, if the drawing skill is lower than the reference value (if the skill is at a low level), the electronic device (200) may proceed to operation 1230. In operation 1230, the electronic device (200) may select a prompt requesting a brief description from among a plurality of prompts (e.g., text prompts) pre-stored in the memory (230).

[0241] As a result of the identification of operation 1220, if the drawing skill is higher than the reference value (if the skill is at a high level), the electronic device (200) may proceed to operation 1240. In operation 1240, the electronic device (200) may select a prompt requesting a detailed description from among a plurality of prompts (e.g., text prompts) pre-stored in the memory (230).

[0242] At operation 1250, the electronic device (200) may transmit the prompt selected through operation 1230 or operation 1240 to the first artificial intelligence model (310) as a first input prompt requesting a description of the drawing.

[0243] FIG. 13 is an example of a third input prompt requesting a proficiency determination, according to one embodiment.

[0244] In one embodiment, the electronic device (200) may determine the drawing proficiency based on the drawing input by the user. For example, the electronic device (200) may transmit a third input prompt (1310) to the first artificial intelligence model (310) to inquire about the drawing proficiency.

[0245] Referring to FIG. 13, the third input prompt (1310) may include a portion (1311) requesting an item-by-item scoring evaluation for the drawing and / or a proficiency level according to the item-by-item scoring evaluation. For example, the portion (1311) may be for requesting a scoring evaluation (e.g., one of low level, middle level, or high level) for at least one item of a quality evaluation for the drawing, the number of stroke lines in the drawing, the length of the stroke lines, or the complexity of the stroke lines, and / or a proficiency level according to the scoring evaluation (e.g., one of low level or high level).

[0246] FIG. 14 is a flowchart illustrating a process by which an electronic device (200) generates a second input prompt based on a style according to one embodiment.

[0247] According to one embodiment, a process (1400, e.g., operation 650 of FIG. 6 ) by which an electronic device (200) transmits a second input prompt may include operations 1410, 1420, 1430, 1440, 1450, and 1460. Operations 1410, 1420, 1430, 1440, 1450, and 1460 may be for generating a second input prompt requesting generation of an image corresponding to the drawing.

[0248] Referring to FIG. 14, in operation 1410, the electronic device (200) can identify the style of an image (or output image) to be generated based on a drawing (or source image). For example, the electronic device (200) can identify a style selected by a user through a user interface (e.g., a first interface element (721) of FIG. 7b) from among a plurality of pre-designated styles (e.g., at least two of watercolor, animation, pencil, cartoon, or neon).

[0249] At operation 1420, the electronic device (200) can identify whether the style identified through operation 1410 belongs to a first style (e.g., cartoon, neon requiring intuitive description) or a second style (e.g., watercolor, pencil requiring precise description).

[0250] If the style identified through operation 1420 is a first style (e.g., comics, neon, where intuitiveness is important), the electronic device (200) may proceed to operation 1430. In operation 1430, the electronic device (200) may select a style prompt (or first image style information, e.g., text representing one of comics or neon) representing the style identified in operation 1410.

[0251] If the style identified through operation 1420 is a second style (e.g., watercolor or pencil requiring precise drawing), the electronic device (200) may proceed to operation 1440. In operation 1440, the electronic device (200) may select a style prompt (or second image style information, e.g., text representing one of watercolor or pencil styles) representing the style identified in operation 1410.

[0252] In operation 1450, the electronic device (200) may generate a prompt by combining the style prompt (or image style information) selected through operation 1430 or operation 1440 and the description information about the drawing. For example, the electronic device (200) may generate a prompt (e.g., one or more natural language sentences) by combining the style prompt (or first image style information) selected through operation 1430 and a brief description portion extracted from the description information about the drawing (e.g., text corresponding to the brief description portion (1120) of FIG. 11). For example, the electronic device (200) may generate a prompt (e.g., one or more natural language sentences) by combining the style prompt (or second image style information) selected through operation 1440 and a detailed description portion extracted from the description information about the drawing (e.g., text corresponding to the detailed description portion (1130) of FIG. 11).

[0253] At operation 1460, the electronic device (200) may transmit the prompt generated through operation 1450 to the second artificial intelligence model (320) as a second input prompt requesting generation of an image.

[0254] FIGS. 15a, 15b, and 15c are examples of a second input prompt requesting the generation of an image, according to one embodiment.

[0255] Referring to FIG. 15A, a second input prompt requesting generation of an image may include a first text prompt (1510). For example, if a user inputs a drawing in the shape of a cat (e.g., a sketch (711) of FIG. 7A) and selects a watercolor style through a user interface (e.g., a first interface element (721) of FIG. 7B), the electronic device (200) may generate a first text prompt (1510) as in FIG. 15A. The first text prompt (1510) may include a style prompt for the watercolor style (or first style information, e.g., a natural language sentence representing the watercolor style) (1511) and description information for a cat object in the drawing (e.g., one or more natural language sentences describing the cat object) (1512).

[0256] Referring to FIG. 15B, the second input prompt requesting the generation of an image may include a second text prompt (1520). For example, if a user inputs a drawing in the shape of a cat (e.g., a sketch (711) of FIG. 7A) and selects a pencil style through a user interface (e.g., a first interface element (721) of FIG. 7B), the electronic device (200) may generate a second text prompt (1520) as in FIG. 15B. The second text prompt (1520) may include a style prompt for the pencil style (or second style information, e.g., a natural language sentence representing the pencil style) (1521) and description information about a cat object in the drawing (e.g., one or more natural language sentences describing the cat object) (1522).

[0257] Referring to FIG. 15C, the second input prompt requesting the generation of an image may include a third text prompt (1530). For example, if a user inputs a drawing in the shape of an ice cream and selects a pencil style through a user interface (e.g., the first interface element (721) of FIG. 7B), the electronic device (200) may generate a third text prompt (1530) as in FIG. 15C. The third text prompt (1530) may include a style prompt for the pencil style (or second style information, e.g., a natural language sentence representing the pencil style) (1531) and description information for an ice cream object in the drawing (e.g., one or more natural language sentences describing the ice cream object) (1532).

[0258] In one embodiment, the order of sentences in the second input prompt may vary depending on the selected style. For example, in the watercolor style, sentences may be arranged in the order of style information - description information, and in the pencil style, sentences may be arranged in the order of description information - style information.

[0259] FIG. 16 is a flowchart illustrating a process by which an electronic device (200) generates a second input prompt based on style similarity according to one embodiment.

[0260] According to one embodiment, a process (1600, e.g., operation 650 of FIG. 6 ) in which an electronic device (200) transmits a second input prompt may include operations 1610, 1620, and 1630.

[0261] Referring to FIG. 16, in operation 1610, the electronic device (200) can identify the style of an image (or output image) to be generated based on a drawing (or source image). For example, the electronic device (200) can identify a style selected by a user through a user interface (e.g., a first interface element (721) of FIG. 7b) from among a plurality of pre-designated styles (e.g., at least two of watercolor, animation, pencil, cartoon, or neon).

[0262] In operation 1620, the electronic device (200) may determine the similarity between the drawing and the image to be generated based on the style identified through operation 1610. The similarity may vary depending on the style of the image to be generated. The electronic device (200) may determine the similarity to be included in the second input prompt differently depending on the style of the image to be generated. For example, if a first style (e.g., cartoon or neon requiring intuitive description) is selected from among a plurality of styles, the electronic device (200) may determine the similarity between the drawing and the image to be generated as a first value (e.g., 0.4) belonging to a relatively low range. For example, if a second style (e.g., watercolor or pencil requiring precise description) is selected from among a plurality of styles, the electronic device (200) may determine the similarity between the drawing and the image to be generated as a second value (e.g., 0.8) belonging to a relatively high range and higher than the first value.

[0263] In operation 1630, the electronic device (200) may generate a prompt including the similarity determined through operation 1620. The electronic device (200) may transmit the generated prompt to the second artificial intelligence model (320) as a second input prompt requesting generation of an image.

[0264] FIG. 17 is an example of description information for a drawing according to one embodiment.

[0265] Referring to FIG. 17, an electronic device (200) may receive a sketch (1710) (or source image, drawing) in the shape of a puppy input by a user. The electronic device (200) may convert the sketch (1710) into image data (1720) (e.g., bmp data) of a specified first format. The electronic device (200) may obtain description information (1730) about the sketch (1710) by providing the image data (1720) (e.g., bmp data) of the first format to a first artificial intelligence model (310). For example, the description information (1730) about the sketch (1710) may include one or more natural language sentences describing a puppy object in the sketch (1710).

[0266] Although not shown in FIG. 17, according to one embodiment, the electronic device (200) can convert and / or encode the first format image data (1720) converted from the sketch (1710) into a second format string data (e.g., base64 type string data).

[0267] According to one embodiment, the electronic device (200) can generate a first input prompt that includes both image data (1720) of a first format for the sketch (1710) and string data (text prompt) of a second format for the sketch (1710). The electronic device (200) can receive description information (1730) for the sketch (1710) from the first artificial intelligence model (310) by providing the first input prompt to the first artificial intelligence model (310).

[0268] According to one embodiment, the electronic device (200) may display a user interface (e.g., a message window, a speech bubble) including at least one keyword related to description information for a drawing (e.g., a sketch (1710) of FIG. 17). The electronic device (200) may receive a user input for the at least one keyword through the user interface. The electronic device (200) may edit (e.g., delete, modify, add) the description information based on the user input. The electronic device (200) may obtain an image generated based on the edited description information by providing the edited description information to a second artificial intelligence model (e.g., the second artificial intelligence model (320) of FIG. 3).

[0269] Figures 18a and 18b are examples of images acquired based on style-specific similarity according to one embodiment.

[0270] According to one embodiment, the electronic device (200) can obtain multiple images (e.g., multiple versions of output images) corresponding to a drawing (or source image) input by a user based on style and / or style similarity.

[0271] Referring to FIG. 18A, when a user inputs a sketch (1710) in the shape of a puppy as shown in FIG. 17 and a watercolor style is selected from among a plurality of styles, the electronic device (200) may determine the similarity between the drawing and the image to be generated as a second value (e.g., 0.8) that matches the watercolor style. The electronic device (200) may generate a prompt that includes description information (1730) about the sketch (1710), a second value (e.g., 0.8) representing the determined similarity, and the number of required images (e.g., 4). The electronic device (200) may obtain multiple versions of images (1811, 1812, 1813, 1814) as shown in FIG. 18A from the second artificial intelligence model (320) by providing the generated prompt as a second input prompt requesting generation of an image.

[0272] Referring to FIG. 18A, the first image (1811), the second image (1812), the third image (1813), and the fourth image (1814) may be watercolor-style images generated based on description information (1730) regarding the puppy-shaped sketch (1710) of FIG. 17 and having a similarity of a second value (e.g., 0.8) with respect to the sketch (1710). For example, the various versions of the images (1811, 1812, 1813, 1814) illustrated in FIG. 18A may be images representing different breeds of puppies generated to correspond to the posture or features of major parts (e.g., the direction the head is looking, the shape of the legs or tail) of the puppy object in the sketch (1710).

[0273] Referring to FIG. 18b, when a user inputs a sketch (1710) in the shape of a puppy as shown in FIG. 17 and a neon style is selected from among a plurality of styles, the electronic device (200) may determine the similarity between the drawing and the image to be generated as a first value (e.g., 0.4) that matches the neon style. The electronic device (200) may generate a prompt that includes description information (1730) about the sketch (1710), the first value (e.g., 0.4) representing the determined similarity, and the number of required images (e.g., 4). The electronic device (200) may obtain multiple versions of images (1821, 1822, 1823, 1824) as shown in FIG. 18b from the second artificial intelligence model (320) by providing the generated prompt as a second input prompt requesting generation of an image.

[0274] Referring to FIG. 18b, the first image (1821), the second image (1822), the third image (1823), and the fourth image (1824) may be neon-style images generated based on description information (1730) about the puppy-shaped sketch (1710) of FIG. 17 and having a similarity of a first value (e.g., 0.4) with respect to the sketch (1710). For example, the various versions of the images (1821, 1822, 1823, 1824) illustrated in FIG. 18b may be images representing simplified puppy characters of different shapes (e.g., characters with different colors and effects applied to neon lights) generated to correspond to the posture or features of major parts (e.g., the direction the head is looking, the shape of the legs or tail) of the puppy object in the sketch (1710).

[0275] The images (1821, 1822, 1823, 1824) of FIG. 18b (watercolor style images generated based on similarity 0.8) may reflect the user's sketch (1710) relatively less than the images (1811, 1812, 1813, 1814) of FIG. 18a (neon style images generated based on similarity 0.4).

[0276] Thus, as the similarity approaches 0.1, the resulting image may be more random than at a similarity of 1.0. Furthermore, as the similarity approaches 0.1, the resulting image may be more dependent on the AI ​​model's learning state than at a similarity of 1.0.

[0277] FIG. 19 is a flowchart illustrating a process in which an electronic device (200) according to one embodiment generates a second input prompt based on similarity by proficiency.

[0278] According to one embodiment, a process (1900, e.g., operation 650 of FIG. 6 ) in which an electronic device (200) transmits a second input prompt may include operations 1910, 1920, 1930, 1940, and 1950.

[0279] Referring to FIG. 19, in operation 1910, the electronic device (200) may determine the proficiency of the drawing (or source image). Operation 1910 may correspond to operation 1210 of FIG. 12.

[0280] At operation 1920, the electronic device (200) can identify whether the drawing skill determined through operation 1910 is higher than a predefined reference value.

[0281] As a result of the identification of operation 1920, if the drawing skill is lower than the reference value (if the skill is at a low level), the electronic device (200) may proceed to operation 1930. In operation 1930, the electronic device (200) may determine the similarity between the drawing and the image to be generated as a first value (e.g., 0.4).

[0282] As a result of the identification of operation 1920, if the drawing skill is higher than the reference value (if the skill is at a high level), the electronic device (200) may proceed to operation 1940. In operation 1940, the electronic device (200) may determine the similarity between the drawing and the image to be generated as a second value (e.g., 0.8) higher than the first value (e.g., 0.4).

[0283] In this way, the electronic device (200) can determine the similarity between the drawing and the image to be generated differently depending on the drawing skill.

[0284] At operation 1950, the electronic device (200) may generate a prompt including the similarity determined through operation 1930 or operation 1940. The electronic device (200) may transmit the generated prompt to the second artificial intelligence model (320) as a second input prompt requesting generation of an image.

[0285] FIG. 20 is an example of a user interface for explaining a process by which an electronic device (200) determines similarity according to one embodiment.

[0286] Referring to FIG. 20, the electronic device (200) may display a user interface, such as a first screen (2001), a second screen (2002), or a third screen (2003). The first screen (2001), the second screen (2002), or the third screen (2003) may include a first interface element (2010) and / or a second interface element (2020).

[0287] The first interface element (2010) may be for selecting a style (image style) of the image to be generated from among a plurality of specified styles (e.g., watercolor, illustration, sketch, pop art, 3D cartoon, no style, or random style).

[0288] The second interface element (2020) may be for varying the similarity depending on user input (e.g., touch input, swipe input, direct number input, input for selecting a number, input for selecting one of the levels of upper / middle / lower, or whether an option is checked).

[0289] According to one embodiment, the electronic device (200) may determine a similarity between the user's drawing and an image to be generated based on an image style selected via the first interface element (2010) and / or a user input for the second interface element (2020). The electronic device (200) may include the similarity in a second input prompt requesting image generation. The electronic device (200) may provide the second input prompt including the similarity to the second artificial intelligence model (320).

[0290] In one embodiment, the lower the similarity between the drawing and the image to be generated, the lower the probability that the feature data of the drawing will be included in the image (or the ratio of the drawing's reflection to the image). The higher the similarity between the drawing and the image to be generated, the higher the probability that the feature data of the drawing will be included in the image (or the ratio of the drawing's reflection to the image).

[0291] According to one embodiment, the electronic device (200) may set the similarity differently depending on the image style selected through the first interface element (2010).

[0292] The electronic device (200) may set a default similarity value that matches each of a plurality of styles. For example, a relatively high default value (e.g., 0.8) may be set for a first style (e.g., watercolor, illustration). A mid-range default value (e.g., 0.6) may be set for a second style (e.g., sketch, pop art, 3D cartoon). A relatively low default value (e.g., 0.4) may be set for a third style (e.g., random style, no style).

[0293] For example, in the first screen (2001), if the watercolor style is selected as the image style and no user input occurs for the second interface element (2020), the similarity may be set to a default value (e.g., 0.8) matching the watercolor style. If the user changes the image style to a 3D cartoon style using the first interface element (2010), the similarity may be changed to a default value (e.g., 0.6) matching the 3D cartoon style.

[0294] According to one embodiment, the electronic device (200) may set the similarity differently depending on the user input to the second interface element (2020).

[0295] For example, if no user input occurs for the second interface element (2020), the similarity may be set to a predetermined default value (e.g., 0.5). The first screen (2001) represents a state in which the similarity indicated by the second interface element (2020) is set to a mid-range default value (e.g., 0.5).

[0296] For example, when a swipe input in the first direction (e.g., a left swipe) occurs on the second interface element (2020), the similarity may be changed to a low range value (e.g., 0.3 or less). The second screen (2002) indicates a state in which the similarity indicated by the second interface element (2020) has been changed to the lowest value (0 or 0%) in response to the swipe input in the first direction. When the similarity is set to the lowest value, the second artificial intelligence model (320) may generate an image using only the description information about the drawing without separate consideration of the characteristic data of the drawing.

[0297] For example, when a swipe input in a second direction opposite to the first direction (e.g., a right swipe) occurs on the second interface element (2020), the similarity may change to a high range value (e.g., 0.7 or higher). The third screen (2003) indicates a state in which the similarity indicated by the second interface element (2020) has changed to the highest value (1 or 100%) in response to the swipe input in the second direction. When the similarity is set to the highest value, the second artificial intelligence model (320) may generate an image by fully reflecting the characteristic data of the drawing together with the description information about the drawing.

[0298] FIGS. 21A, 21B, 21C, and 21D are examples of user interfaces for explaining a process of displaying an image (e.g., operation 460 of FIG. 4 or operation 670 of FIG. 6) by an electronic device (200) according to one embodiment.

[0299] Referring to FIG. 21A, the electronic device (200) may display a first screen (2110). The first screen (2110) may display a drawing (2111) input by a user and a first image (2112) generated in response to the drawing (2111). For example, the first image (2112) may be generated (or acquired) based on at least one of the drawing (2111), description information about the drawing (2111), similarity, or image style.

[0300] Referring to FIG. 21B, the electronic device (200) may display a first screen (2120) and / or a second screen (2130). For example, the electronic device (200) may display a first screen (2120) for a first image (2121) generated in response to a drawing. When a user input (e.g., a touch) occurs for the first image (2121), the electronic device (200) may change the first screen (2120) to a second screen (2130). The second screen (2130) may be a screen that temporarily displays a second image (2131) in which the drawing overlaps the first image (2121). The electronic device (200) can change the second screen (2130) back to the first screen (2120) when a specified time (e.g., 10 seconds) has passed or a user input (e.g., touch) occurs for the second image (2131).

[0301] Referring to FIG. 21c, the electronic device (200) can generate (or acquire) multiple images (2151, 2161) corresponding to one drawing (2141) using an artificial intelligence model (e.g., a second artificial intelligence model (320)). The electronic device (200) can display a screen (2150, 2160) for each of the multiple images (2151, 2161).

[0302] According to one embodiment, the electronic device (200) may acquire a plurality of images (2151, 2161) corresponding to the user's drawing (2141), and then sequentially display (or display or highlight with a high priority) images among the plurality of images (2151, 2161) that have a small difference (or a close distance) from the drawing (2141).

[0303] According to one embodiment, the electronic device (200) may alternately display a first screen (2140), a second screen (2150), and a third screen (2160) according to a user input (e.g., a swipe input). The first screen (2140) may be a screen including a drawing (2141) input by the user. The second screen (2150) may be a screen including a first image (2151) generated in response to the drawing (2141). The third screen (2160) may be a screen including a second image (2161) generated in response to the drawing (2141).

[0304] According to one embodiment, when a swipe input in a first direction (e.g., left) occurs while a second screen (2150) for a first image (2151) is displayed, the second screen (2150) for the first image (2151) may be changed to a third screen (2160) for the second image (2161). When a swipe input in a second direction (e.g., right) occurs while a third screen (2160) for the second image (2161) is displayed, the third screen (2160) for the second image (2161) may be changed to a second screen (2150) for the first image (2151).

[0305] In one embodiment, the first screen (2140), the second screen (2150), or the third screen (2160) may include an image regeneration button (2142, 2152, 2162). The image regeneration button (2142, 2152, 2162) may be for generating a new image corresponding to the drawing (2141).

[0306] For example, the electronic device (200) may detect a user input (e.g., touch) to an image regeneration button (2141, 2152, 2162) while any one of a drawing (2141), a first image (2151), and a second image (2161) is displayed. The electronic device (200) may generate and / or display at least one third image in response to the user input. The first image (2151) and the second image (2161) may be previous images generated in response to the drawing (2141). The at least one third image may be a new image generated in response to the drawing (2141).

[0307] For example, the first image (2151) and the second image (2161) may be generated based on at least some of the drawing (2141), descriptive information about the drawing (2141), similarity, image style, drawing skill, or transparency of the drawing area.

[0308] For example, the at least one third image may be generated based on the drawing (2141) and / or a previous image (2151, 2161) generated corresponding to the drawing (2141). For example, the at least one third image may be generated based on at least some of the drawing (2141), description information about the drawing (2141), similarity, image style, drawing skill, transparency of the drawing area, or a previous image (2151, 2161) generated corresponding to the drawing (2111).

[0309] Referring to FIG. 21d, the electronic device (200) can receive multiple versions of images for a single drawing from the second artificial intelligence model (320). The electronic device (200) can display the multiple images on a single screen.

[0310] For example, the plurality of images may be images of different styles corresponding to one drawing. For example, the electronic device (200) may display a first screen (2170) and / or a second screen (2180). The first screen (2170) may be a screen that displays a plurality of images (2171, 2172, 2173, 2174) having different styles in a list format. The second screen (2180) may be a screen that preferentially displays a first image (2171) of a style selected by a user (e.g., displayed in a first size) and displays second images (2172, 2173, 2174) of the remaining styles in the form of thumbnails (e.g., displayed in a second size smaller than the first size).

[0311] For example, after receiving a plurality of images (2171, 2172, 2173, 2174) corresponding to the drawing, the electronic device (200) may sequentially display (or display or highlight with a high priority) images among the plurality of images (2171, 2172, 2173, 2174) that have a small difference (or a close distance) from the drawing.

[0312] The first screen (2170) and the second screen (2180) may include an image regeneration button (2175). The image regeneration button (2175) may be for generating a new image corresponding to the drawing.

[0313] FIG. 22a is an example of a first input prompt requesting a description of a drawing, according to one embodiment.

[0314] Referring to FIG. 22A, a first input prompt requesting a description of a drawing (e.g., the first input prompt of operation 630 illustrated in FIG. 6) may include a first text prompt (2210) in a specified format (e.g., JSON format). The first text prompt (2210) may include a first portion (2211) and / or a second portion (2212).

[0315] The first part (2211) may be a part representing image data for the drawing (base64 type string data converted from the drawing). The second part (2212) may be a part representing description information for the drawing (e.g., one or more natural language sentences describing an object within the drawing).

[0316] FIG. 22b is an example of a second input prompt requesting the generation of an image, according to one embodiment.

[0317] Referring to FIG. 22b, a second input prompt requesting generation of an image corresponding to a drawing (e.g., the second input prompt of operation 650 illustrated in FIG. 6) may include a second text prompt (2220) in a specified format (e.g., JSON format). The second text prompt (2220) may include a first portion (2221), a second portion (2222), a third portion (2223), and / or a fourth portion (2224).

[0318] In one embodiment, the second input prompt may include descriptive information about the drawing.

[0319] The first part (2221) may be a part indicating a size property of the image to be generated (e.g., at least one of size, aspect ratio, or resolution).

[0320] The second portion (2222) may be a portion corresponding to a negative prompt. For example, the negative prompt may be intended to indicate one or more words and / or sentences to be filtered from the description information for the drawing.

[0321] The third part (2223) may be a part representing image data for the drawing (string data of base64 type converted from the drawing).

[0322] The fourth part (2224) is a part that represents description information about the drawing, and may include one or more natural language sentences related to the drawing and / or objects within the drawing.

[0323] In one embodiment, the second input prompt may include a similarity between the drawing and the image to be generated.

[0324] According to one embodiment, the second input prompt may include at least one of drawing or image style information to be used in generating the image.

[0325] In one embodiment, the second input prompt may include additional information to be used in image generation.

[0326] According to one embodiment, the additional information may further include at least a portion of reference text input by a user (e.g., text input through an input window), reference image (e.g., background image displayed in a drawing area, captured image of a background application execution screen), form factor information of the electronic device (200), or form factor information of an external electronic device (e.g., first external electronic device (2610), second external electronic device (2620) of FIG. 26) connected to the electronic device (200) via short-range wireless communication.

[0327] According to one embodiment, the additional information may include at least one of user contents (e.g., sketches, photos, drawings, images, videos, documents drawn by the user in the past) stored in the memory (230) of the electronic device (200) or registered in the user account. When such additional information is used to generate an image, an image having continuity with the previous sketch or the user content, for example, an image including an object similar to an object in the previous sketch or the user content, may be generated.

[0328] According to one embodiment, the additional information may include at least one of location information of the electronic device (200) (e.g., the location where the user is located) or application usage information of the electronic device (200) (e.g., previous conversations in a chat application used by the user before starting drawing). When such additional information is used to generate an image, an image suitable for the user's situation can be generated.

[0329] FIG. 23a is an example of a hardware configuration of an electronic device (200) according to one embodiment.

[0330] According to one embodiment, the electronic device (200) may be a fixed form factor type electronic device (2300) (e.g., a smart phone, a tablet PC (personal computer)).

[0331] Referring to FIG. 23A, the electronic device (2300) may include a display (2310). The display (2310) of the electronic device (2300) may correspond to the display module (160) of FIG. 1 or the touchscreen display (210) of FIG. 2.

[0332] In one embodiment, the display (2310) may include a display panel (2312), a touch panel (2311), and a digitizer module (2313). The touch panel (2311) may be positioned on an upper surface of the display panel (2312) and may detect a capacitive touch input. The digitizer module (2313) may be positioned on a lower surface of the display panel (2312) and may detect an electromagnetic resonance (EMR) pen input.

[0333] According to one embodiment, an electronic pen (2330) may be used as an input tool. The electronic pen (2330) may correspond to the electronic pen (500) of FIG. 5. The electronic device (2300) may detect a touch input by the electronic pen (2330) through the display (2310).

[0334] According to one embodiment, the electronic device (2300) may include a structure into which an electronic pen (2330) may be inserted. The electronic device (2300) may include a housing (2320). A hole (2321) may be formed in a portion of the housing (2320), for example, a portion of a side of the housing (2320). The electronic pen (2330) may be designed to be accommodated in the hole (2321) within the housing (2320) of the electronic device (2300). An internal space, which is a storage space connected to the hole (2321), may be formed on the housing (2320). The electronic pen (2330) may be inserted into the internal space of the hole (2321) and may be stably attached or detached from the internal space.

[0335] According to one embodiment, the electronic pen (2330) may be configured to have a magnetic attachment / detachment function. In this case, a structure (e.g., a hole (2321)) for inserting / detaching the electronic pen (2330) into / from the housing (2320) may be omitted. The electronic pen (2330) may be fixed to a magnet portion formed on one side of the housing (2320) or may be easily detached as needed.

[0336] FIG. 23b is a block diagram of an electronic pen (2330), which is an input means according to one embodiment.

[0337] Referring to FIG. 23b, an electronic pen (2330) according to one embodiment may include, as an input means, a memory (2331), a communication circuit (2332), a processor (2333), a resonant circuit (2334), and a trigger circuit (2335). According to one embodiment, the processor (2333) may be electrically connected to the memory (2331), the communication circuit (2332), the resonant circuit (2334), and / or the trigger circuit (2335). However, the configuration of the electronic pen (2330) is not limited to that illustrated in FIG. 23b, and the electronic pen (2330) may include more or less components other than the illustrated components.

[0338] According to one embodiment, the memory (2331) of the electronic pen (2330) may store information related to the operation of the electronic pen (2330). For example, the information may include information for communication with the electronic device (2300) and frequency information related to the input operation of the electronic pen (2330). The memory (2331) according to one embodiment may also store a communication stack of the communication circuit (2332). Depending on the implementation, the communication circuit (2332) and / or the processor (2333) may include dedicated memory.

[0339] According to one embodiment, the communication circuit (2332) of the electronic pen (2330) may be configured to perform a wireless communication function between the electronic pen (2330) and a communication module of the electronic device (2300) (e.g., the communication circuit (240) of FIG. 2). According to one embodiment, the communication circuit (2332) may transmit information related to the status information, input information, and / or location of the electronic pen (2330) to the electronic device (2300) using a short-range communication method. For example, the communication circuit (2332) may transmit information related to the direction of the electronic pen (2330) obtained through the trigger circuit (2335) (e.g., motion sensor data), audio information input through a microphone, information related to the remaining battery level, and / or information related to the status of a button (e.g., an on / off button) provided in the electronic pen (2330) to the electronic device (2300). According to one embodiment, the short-range communication method may include at least one of, but not limited to, Bluetooth, Bluetooth Low Energy (BLE), NFC, Wi-Fi direct, or IrDA.

[0340] In one embodiment, the processor (2333) of the electronic pen (2330) may be configured to collect various data and calculate a desired output value. In one embodiment, the processor (2333) may support various operations based on at least a portion of user input. For example, the processor (2333) may include a customized hardware module or a generic processor configured to execute software (e.g., an application). The processor (2333) may include, for example, one or a combination of two or more of hardware, software, or firmware.

[0341] According to one embodiment, the processor (2333) of the electronic pen (2330) may be configured to transmit information indicating a pressed state of a button, sensing information acquired by a sensor, and / or information calculated based on the sensing information (e.g., information related to the position of the electronic pen (2330)) to the electronic device (2300) via the communication circuit (2332).

[0342] According to one embodiment, the resonant circuit (2334) of the electronic pen (2330) may resonate based on an electromagnetic field signal generated from the digitizer module (2313) of the electronic device (2300) and may radiate an EMR input signal by resonance. The electronic device (2300) may use the EMR input signal to determine the position of the electronic pen (2330). For example, the electronic device (2300) may determine the position of the electronic pen (2330) based on the magnitude of an induced electromotive force (e.g., an output current) generated by the EMR input signal in each of a plurality of channels (e.g., a plurality of loop coils) within the digitizer module (2313).

[0343] Meanwhile, although the electronic device (2300) and the electronic pen (2330) have been described as operating based on the EMR method as described above, this is merely exemplary and the driving method of the electronic pen (2330) is not limited thereto. For example, the electronic device (2330) may generate a signal based on an electric field based on an electrically coupled resonance (ECR) method. The resonance circuit (2334) of the electronic pen (2300) may be resonated by an electric field. The electronic device (2300) may check the potential in a plurality of channels (e.g., electrodes) due to resonance in the electronic pen (2330) and may also check the position of the electronic pen (2330) based on the potential. As another example, the electronic pen (2330) may be implemented using an active electrostatic (AES) method. As another example, the electronic device (2300) may detect the electronic pen (2330) based on a change in capacitance associated with at least one electrode of the touch panel (2311). In this case, the electronic pen (2330) may not include a resonant circuit (2334).

[0344] According to one embodiment, the resonant circuit (2334) of the electronic pen (2330) may include a coil (or an inductor and / or a capacitor). The resonant circuit (2334) may resonate based on an input electric field and / or magnetic field. When the electronic pen (2330) transmits a signal using the EMR method, the electronic pen (2330) may generate a signal including a resonant frequency based on an electromagnetic field generated from an inductive panel of the electronic device (2300). When the electronic pen (2330) transmits a signal using the AES method, the electronic pen (2330) may generate a signal using capacitive coupling with the electronic device (2300). When the electronic pen (2330) transmits a signal by the ECR method, the electronic pen (2330) can generate a signal including a resonant frequency based on an electric field generated from a capacitive device of the electronic device (2300).

[0345] According to one embodiment, the resonance circuit (2334) of the electronic pen (2330) can be used to change the intensity or frequency of an electromagnetic field depending on the user's operating state. For example, the resonance circuit (2334) can provide various frequencies for recognizing hovering input, drawing input, button input, or erasing input. For example, the resonance circuit (2334) can provide various resonance frequencies depending on the connection combination of a plurality of capacitors, or can provide various resonance frequencies based on a variable inductor and / or a variable capacitor.

[0346] According to one embodiment, the trigger circuit (2335) of the electronic pen (2330) may include at least one button or sensor circuit. According to one embodiment, the processor (2333) may determine the button input method (e.g., touch, press) or type of the electronic pen (2330). According to one embodiment, the trigger circuit (2335) may transmit a trigger signal to an electronic device (2300) connected to wireless communication using an input signal of the button.

[0347] In one embodiment, the electronic pen (2330) may be connected to the electronic device (2300) via short-range wireless communication. For example, the electronic device (2300) and the electronic pen (2330) may be connected via short-range wireless communication using at least one of Bluetooth communication, low-power Bluetooth communication, and / or Wi-Fi communication. In this case, the electronic pen (2330) may be used for the electronic device (2300) to generate user input. For example, the electronic pen (2330) may be brought into contact with the display (2310) of the electronic device (2300) or may be brought into non-contact (proximity) within a reference distance to the electronic device (2300) to generate an input for the electronic device (2300).

[0348] In some embodiments, even if there is no short-range wireless communication connection established between the electronic pen (2330) and the electronic device (2300), the electronic device (2300) may detect touch input by the electronic pen (2330) through the display (2310) or provide a drawing function using the electronic pen (2330).

[0349] FIG. 24 is an example of a hardware configuration of a foldable electronic device (2400) according to one embodiment.

[0350] According to one embodiment, the electronic device (200) may be a flexible form factor type electronic device whose form factor can be transformed. For example, the electronic device (200) may be a foldable electronic device (2400) of FIG. 24.

[0351] Referring to FIG. 24, reference numeral 2401 represents a first state in which the foldable electronic device (2400) is unfolded. The first state may be a fully unfolded state (a state in which the folding angle is substantially 180 degrees) in which the foldable electronic device (2400) is fully unfolded. Reference numeral 2402 represents a second state in which the foldable electronic device (2400) is folded. The second state may be a fully folded state (a state in which the folding angle is substantially 0 degrees) in which the foldable electronic device (2400) is fully folded.

[0352] Referring to FIG. 24, a foldable electronic device (2400) may include a foldable housing (2420) and a display (2410) positioned within a space formed by the foldable housing (2420). The display (2410) of the foldable electronic device (2400) may correspond to the display module (160) of FIG. 1 or the touchscreen display (210) of FIG. 2.

[0353] According to one embodiment, the foldable housing (2420) may include a first housing (2421) and a second housing (2422). The first housing (2421) and / or the second housing (2422) may form at least a portion of the exterior of the foldable electronic device (2400).

[0354] According to one embodiment, the display (2410) may include a first display (2411) (e.g., flexible display, front display, main display) and a second display (2412) (e.g., rear display, auxiliary display).

[0355] According to one embodiment, the first housing (2421) and the second housing (2422) are connected by a hinge (not shown) and can rotate relative to each other about a folding axis (A) by the hinge. Accordingly, the first housing (2421) and / or the second housing (2422) of the foldable electronic device (2400) can be folded or unfolded about the folding axis (A). In addition, the folding angle of the foldable electronic device (2400) and / or the folding state of the foldable electronic device (2400) (e.g., the first state of the drawing reference numeral 2401, the second state of the drawing reference numeral 2402, or an intermediate state between the first state and the second state) can be changed.

[0356] In one embodiment, the surface on which the first display (2411) of the display (2410) is visually exposed is defined as the front surface of the foldable electronic device (2400). The surface opposite the front surface is defined as the back surface of the foldable electronic device (2400). In addition, the surface surrounding at least a portion of the space between the front surface and the back surface is defined as the side surface of the foldable electronic device (2400).

[0357] According to one embodiment, when the folding state of the foldable electronic device (2400) is a first state (2401) (fully unfolded state), the first display (2411) may be visually exposed or activated through the front of the foldable electronic device (2400). When the folding state of the foldable electronic device (2400) is a second state (2402) (fully folded state), the first display (2411) may not be exposed to the outside of the foldable electronic device (2400), and the second display (2412) may be visually exposed through the back of the foldable electronic device (2400).

[0358] In one embodiment, the first housing (2421) can provide relative motion to the second housing (2422), and the second housing (2422) can provide relative motion to the first housing (2421).

[0359] In one embodiment, the first housing (2421) can rotate relative to the second housing (2422) using a hinge. Accordingly, the foldable electronic device (2400) can be changed into a first state (2401) or a second state (2402).

[0360] According to one embodiment, the first housing (2421) and the second housing (2422) are arranged on both sides with the folding axis (A) as the center, and may have a shape that is overall symmetrical with respect to the folding axis (A).

[0361] According to one embodiment, the angle or distance between the first housing (2421) and the second housing (2422) may vary depending on the folding state of the foldable electronic device (2400). The angle between the first housing (2421) and the second housing (2422) may change depending on whether the folding state of the foldable electronic device (2400) is the first state (2401) (fully unfolded state), the second state (2402) (fully folded state), or an intermediate state between the first state (2401) and the second state (2402). The angle between the first housing (2421) and the second housing (2422) may be interpreted as the folding angle of the foldable electronic device (2400).

[0362] According to one embodiment, the display (2410) of the foldable electronic device (2400) may include a first display (2411) (e.g., a flexible display, a front display, a main display). According to one embodiment, the first display (2411) may have a structure in which at least a portion thereof may be deformed into a flat or curved surface. For example, the first display (2411) may be formed to be variable in response to the relative movement of the second housing (2422) with respect to the first housing (2421).

[0363] According to one embodiment, the display (2410) of the foldable electronic device (2400) may include a second display (2412) (e.g., a rear display, a secondary display). The second display (2412) may be arranged to face a different direction from the first display (2411). For example, the first display (2411) may be visually exposed through the front of the foldable electronic device (2400), and the second display (2412) may be visually exposed through the back of the foldable electronic device (2400).

[0364] FIGS. 25A and 25B are examples of drawings and images displayed on an electronic device (200) according to one embodiment.

[0365] In one embodiment, the size properties (e.g., at least one of size, aspect ratio, or resolution) of a drawing input by a user and an image generated corresponding to the drawing may be different from each other. In such a case, the electronic device (200) may adjust the size properties of the drawing or adjust the size properties of the image generated corresponding to the drawing.

[0366] According to one embodiment, the electronic device (200) may be a foldable electronic device (2400) of FIG. 24. The folding state of the foldable electronic device (2400) may change. As the folding state of the foldable electronic device (2400) changes, the size of the display area may change. The size of the display area may refer to the size of the display area used (or activated) for a user's drawing and / or image display.

[0367] For example, when the foldable electronic device (2400) is in the first state (2401) (fully unfolded state) of FIG. 24, the first display (2411) can be visually exposed and activated through the front of the foldable electronic device (2400). In this case, the size of the display area of ​​the foldable electronic device (2400) can correspond to the first size, which is the size of the first display (2411).

[0368] For example, when the foldable electronic device (2400) is in the second state (2402) (fully folded state) of FIG. 24, the second display (2412) may be visually exposed and activated through the back of the foldable electronic device (2400). In the second state (2402) (fully folded state), the first display (2411) may not be visually exposed from the outside of the foldable electronic device (2400). In this case, the size of the display area of ​​the foldable electronic device (2400) may correspond to the second size, which is the size of the second display (2412). The second size may have a smaller value (e.g., 50% or less) than the first size.

[0369] In the case of a foldable electronic device (2400), the size of a drawing actually drawn by a user may vary. For example, depending on the folding state of the foldable electronic device (2400), the size attribute of the drawing may have a value corresponding to the size of the first display (2411) or corresponding to the size of the second display (2412) (e.g., an aspect ratio of 16:9). On the other hand, an image generated corresponding to the drawing may have a designated size attribute (e.g., an aspect ratio of 1:1). In this way, the size attribute of the drawing and the size attribute of the image may be different from each other.

[0370] According to one embodiment, the foldable electronic device (2400) can adjust the size properties of the drawing and / or the image to match the size properties of the drawing and the image with each other.

[0371] Referring to FIG. 25A, when the folding state of the foldable electronic device (2400) is a first state (2401), the first display (2411) is activated, and the display mode of the first display (2411) is a first situation in which the portrait mode is used, a user can input a drawing (2511) through the first display (2411) in the portrait mode. The drawing (2511) can have a first size property (e.g., an aspect ratio of 3:4). The foldable electronic device (2400) can generate a source image (2510) with left and right padding areas (2512) added to the drawing (2511). The source image (2510) can have a first size property (e.g., an aspect ratio of 1:1). The foldable electronic device (2400) can obtain a first image (2520) and a second image (2530) using a source image (2510) having a first size attribute (e.g., an aspect ratio of 1:1). The first image (2520) can have the same size attribute (e.g., an aspect ratio of 3:4) as the drawing (2511). The second image (2530) is obtained by adding left and right padding areas (2532) to a basic area (2531) corresponding to the first image (2520), and can have the same size attribute (e.g., an aspect ratio of 1:1) as the source image (2510). The foldable electronic device (2400) can obtain the first image (2520) and the second image (2530) having different size attributes from an artificial intelligence model (e.g., the second artificial intelligence model (320) of FIG. 3). The foldable electronic device (2400) can select a first image (2520) from among the first image (2520) and the second image (2530) based on a first situation (e.g., current folding state and / or portrait mode). The foldable electronic device (2400) can display the selected first image (2520).

[0372] Referring to FIG. 25B, when the folding state of the foldable electronic device (2400) is a first state (2401), the first display (2411) is activated, and the display mode of the first display (2411) is a landscape mode, a user can input a drawing (2551) through the first display (2411) in the landscape mode. The drawing (2551) can have a first size property (e.g., an aspect ratio of 4:3). The foldable electronic device (2400) can generate a source image (2550) with upper and lower padding areas (2552) added to the drawing (2551). The source image (2550) can have a first size property (e.g., an aspect ratio of 1:1). The foldable electronic device (2400) can obtain a first image (2560) and a second image (2570) using a source image (2550) having a first size attribute (e.g., an aspect ratio of 1:1). The first image (2560) can have the same size attribute (e.g., an aspect ratio of 4:3) as the drawing (2551). The second image (2570) is obtained by adding upper and lower padding areas (2572) to a basic area (2571) corresponding to the first image (2560), and can have the same size attribute (e.g., an aspect ratio of 1:1) as the source image (2550). The foldable electronic device (2400) can obtain the first image (2560) and the second image (2570) having different size attributes from an artificial intelligence model (e.g., the second artificial intelligence model (320) of FIG. 3). The foldable electronic device (2400) can select a first image (2560) from among the first image (2560) and the second image (2570) based on a second situation (e.g., a current folding state and / or a landscape mode). The foldable electronic device (2400) can display the selected first image (2560).

[0373] According to one embodiment, the foldable electronic device (2400) can obtain a plurality of images corresponding to the form factor information by providing the form factor information of the foldable electronic device (2400) to an artificial intelligence model (e.g., the second artificial intelligence model (320) of FIG. 3) together with a drawing.

[0374] According to one embodiment, the form factor information may include device information of the foldable electronic device (2400) (e.g., at least one of a device type, a model name, a display shape (e.g., a square, a circle), or display sizes according to changes in a folding state). For example, the plurality of images may include an image corresponding to a first size, which is a size of the first display (2411), and an image corresponding to a second size, which is a size of the second display (2412). The foldable electronic device (2400) may display at least one of the acquired plurality of images through the first display (2411) or the second display (1412).

[0375] Although not shown, the electronic device (200) according to one embodiment may be a rollable (or slideable) electronic device. According to one embodiment, the rollable electronic device may obtain a plurality of images corresponding to the form factor information by providing form factor information of the rollable electronic device (e.g., at least one of a device type, a model name, a display shape (e.g., a square, a circle), or display sizes according to changes in a rolling (or sliding) state) together with a drawing to an artificial intelligence model (e.g., the second artificial intelligence model (320) of FIG. 3).

[0376] The display size of a rollable (or slideable) electronic device may vary depending on a rolling (or sliding) motion. For example, in a first state (e.g., a fully collapsed state), the display size may be a first size. In a second state (e.g., a fully expanded state), the display size may be a second size larger than the first size. For example, the rollable electronic device may, in the first state, receive a user input for selecting an interface element (e.g., an image generation button) for requesting generation of an image corresponding to a drawing. After receiving the user input, if a rolling (or sliding) motion that changes from the first state to the second state is detected, the rollable electronic device may transmit the second size, which is the display size in the second state, to an artificial intelligence model (e.g., the second artificial intelligence model (320) of FIG. 3), thereby obtaining and / or displaying an image generated corresponding to the second size. In this way, if a change in the rolling state occurs while performing an image providing function, an image suitable for the changed display size can be provided.

[0377] FIG. 26 is a diagram illustrating an external electronic device (2610, 2620) (or target electronic device) that is connected to an electronic device (200) according to one embodiment.

[0378] According to one embodiment, the electronic device (200) may be the electronic device (2300) of FIG. 23a.

[0379] Referring to FIG. 26, the electronic device (2300) can provide images in conjunction with a first external electronic device (2610) (e.g., an extended reality display device such as a head mounted display (HMD) device) and / or a second external electronic device (2620) (e.g., a wearable device such as a smart watch).

[0380] According to one embodiment, the electronic device (2300) may be connected to a first external electronic device (2610) and / or a second external electronic device (2620) via short-range wireless communication (e.g., Bluetooth, Wi-Fi, etc.) through a communication circuit (e.g., the communication circuit (240) of FIG. 2).

[0381] According to one embodiment, the electronic device (2300) can receive a drawing (2601) input by a user through a display (2310). The electronic device (2300) can obtain at least one image (e.g., a first image (2602) and / or a second image (2603)) corresponding to the form factor information by providing the form factor information of the first external electronic device (2610) and / or the second external electronic device (2620) together with the drawing (2601) to an artificial intelligence model (e.g., a second artificial intelligence model (320) of FIG. 3).

[0382] The form factor information may include device information of the first external electronic device (2610) (e.g., at least one of a device type, a model name, a display shape (e.g., a square, a circle), or a size of the first external display (2611)). The form factor information may include device information of the second external electronic device (2620) (e.g., at least one of a device type, a model name, or a size of the second external display (2621)).

[0383] The image obtained from the above artificial intelligence model may include a first image (2602) corresponding to the size of the first external display (2611) and / or a second image (2603) corresponding to the size of the second external display (2621).

[0384] The electronic device (2300) can transmit a first image (2602) obtained from the artificial intelligence model to a first external electronic device (2610) through a communication circuit (e.g., the communication circuit (240) of FIG. 2), thereby causing the first image (2602) to be displayed through a first external display (2611) of the first external electronic device (2610). The electronic device (2300) can transmit a second image (2603) obtained from the artificial intelligence model to a second external electronic device (2620), thereby causing the second image (2603) to be displayed through a second external display (2621) of the second external electronic device (2620).

[0385] According to one embodiment, the electronic device (2300) may provide form factor information of an external electronic device (e.g., the first external electronic device (2610) and / or the second external electronic device (2620) of FIG. 26) that is registered in a user account of the electronic device (2300) and has been connected (has a connection history) to the electronic device (2300) as an artificial intelligence model (e.g., the second artificial intelligence model (320) of FIG. 3) together with a drawing (2601). By doing so, the electronic device (2300) may obtain at least one image (e.g., the first image (2602) and / or the second image (2603)) corresponding to the form factor information. Thereafter, the electronic device (2300) may connect to the external electronic device via short-range wireless communication (e.g., Bluetooth, Wi-Fi, etc.) through a communication circuit (e.g., the communication circuit (240) of FIG. 2). The electronic device (2300) can transmit the at least one image (e.g., the first image (2602) and / or the second image (2603)) via the short-range wireless communication, thereby causing the at least one image to be displayed through a display of the external electronic device (e.g., the first external display (2611) and / or the second external display (2612)).

[0386] FIG. 27 is a flowchart illustrating an image providing method of an electronic device (200) according to one embodiment.

[0387] According to one embodiment, the electronic device (200) can generate an image corresponding to the drawing based on at least some of the user's drawing and style information (image style information) of the image to be generated.

[0388] Referring to FIG. 27, an image providing method (2700) of an electronic device (200) according to one embodiment may include operations 2710, 2720, 2730, 2740, and 2750. In some embodiments, some of the illustrated operations may be omitted, some operations may be integrated, the order of some operations may be changed, or other operations may be added.

[0389] According to one embodiment, at least some of the operations illustrated in FIG. 27 may correspond to the operations illustrated in FIG. 4. For example, operation 2710 of FIG. 27 may correspond to operation 410 of FIG. 4. Operation 2730 of FIG. 27 may correspond to operation 420 of FIG. 4. Operation 2750 of FIG. 27 may correspond to operation 460 of FIG. 4. At least some of the operations illustrated in FIG. 27 may be performed in combination with at least some of the operations illustrated in FIG. 4.

[0390] In operation 2710, the electronic device (200) (e.g., processor (220)) may receive (or detect) a drawing according to a first user input (e.g., touch, gesture, stroke input, picture input, handwriting input) via the touchscreen display (210).

[0391] According to one embodiment, the electronic device (200) may display a user interface for receiving a drawing (e.g., a third screen (2803) of FIG. 28, a third screen (2903) of FIG. 29, a first screen (3001) of FIG. 30, a first screen (3101) of FIG. 31) on the touchscreen display (210) when a triggering event occurs. For example, the triggering event may include at least one of an event detecting a hovering input of an electronic pen (e.g., an electronic pen (500) of FIG. 5) (e.g., a hovering input of the electronic pen (500) on the first screen (2801) of FIG. 28), an event detecting a user input selecting a designated visual element within an edge area of ​​a touchscreen display (210) (e.g., a sketch conversion icon (2940) of FIG. 29), and an event detecting a user input selecting a designated visual element within an application execution screen (e.g., a sketch conversion menu (3040) of FIG. 30). The electronic device (200) may receive a user's drawing through the user interface.

[0392] At operation 2720, the electronic device (200) (e.g., processor (220)) may determine (or identify) the style (image style) of the image to be generated.

[0393] According to one embodiment, the electronic device (200) may display, on the touchscreen display (210), a first interface element (e.g., the first interface element (2871) of FIG. 28, the first interface element (2971) of FIG. 29, the first interface element (3071) of FIG. 30, the first interface element (3151) of FIG. 31, and the first interface element (3231) of FIG. 32) for selecting a style of an image to be generated from among a plurality of specified styles. The electronic device (200) may select a specific style (image style) from among the plurality of specified styles according to a user input to the first interface element.

[0394] In operation 2730, the electronic device (200) (e.g., processor (220)) may receive (or detect) a second user input for image generation (e.g., a touch on the image generation button (2872) of FIG. 28, a touch on the image generation button (2972) of FIG. 29, a touch on the image generation button (3082) of FIG. 30, a touch on the image generation button (3152) of FIG. 31) via the touchscreen display (210).

[0395] At operation 2740, the electronic device (200) (e.g., processor (220)) may obtain an image corresponding to the drawing based on the drawing received through operation 2710 and the style determined through operation 2720.

[0396] According to one embodiment, the electronic device (200) may obtain (or generate) an image of a specific style corresponding to the drawing by providing the user's drawing received through operation 2710 and the image style information (specific style information) determined through operation 2720 to an artificial intelligence model (e.g., the first artificial intelligence model (310) and / or the second artificial intelligence model (320) of FIG. 3) in response to the second user input of operation 2730.

[0397] At operation 2750, the electronic device (200) (e.g., processor (220)) may display the image acquired through operation 2740 through the touchscreen display (210).

[0398] FIG. 28 is an example of a user interface for explaining a process in which an electronic device (200) receives a drawing through an electronic pen (500) according to one embodiment.

[0399] According to one embodiment, the electronic device (200) may perform an operation (e.g., operation 410 of FIG. 4) of receiving a user's drawing via a visual element (e.g., a floating button (2810)) linked to an electronic pen (500).

[0400] Referring to FIG. 28, the electronic device (200) can display user interfaces such as a first screen (2801), a second screen (2802), a third screen (2803), and a fourth screen (2804) through a touchscreen display (210). The first screen (2801), the second screen (2802), the third screen (2803), and the fourth screen (2804) can be user interfaces for receiving drawings.

[0401] The electronic device (200) can detect a hovering input of an electronic pen (500) on a first screen (2801) (e.g., a home screen) while the first screen (2801) is being displayed on a touchscreen display (210). When the hovering input of the electronic pen (500) is detected, the electronic device (200) can display a floating button (2810) on the first screen (2801). The floating button (2810) can be a visual element (or graphic element) linked to the electronic pen (500).

[0402] A user can select a floating button (2810) using an electronic pen (500). When a user input (2820) (e.g., a touch) for selecting the floating button (2810) is received, the electronic device (200) can display a second screen (2802).

[0403] The electronic device (200) may display a pop-up window (2830) on the second screen (2802). The pop-up window (2830) may include at least one visual element indicating a function selectable and / or executable by the electronic pen (500).

[0404] At least one of the visual elements within the pop-up window (2830) may be related to the image providing function. For example, the sketch conversion menu (2840) within the pop-up window (2830) may be a visual element related to the image providing function. The user may select the sketch conversion menu (2840) using the electronic pen (500). When a user input (2850) (e.g., a touch) for selecting the sketch conversion menu (2840) is received, the electronic device (200) may display a third screen (2803).

[0405] The third screen (2803) may display text (e.g., "Draw freely") (2860) guiding the user to input a drawing.

[0406] A user can input a drawing (2811) on a third screen (2803) using an electronic pen (500). The electronic device (200) can receive the drawing (2811) according to the user input. When the drawing (2811) is received, the third screen (2803) can be switched to a fourth screen (2804). The fourth screen (2804) can include a first interface element (2871) and an image generation button (2872) for selecting a style of an image to be generated. For example, the first interface element (2871) can be for selecting any one of watercolor, illustration, sketch, pop art, 3D cartoon, no style, or random style.

[0407] The first interface element (2871) of FIG. 28 may correspond to the first interface element (721) of FIG. 7b. The image creation button (2872) of FIG. 28 may correspond to the image creation button (741) of FIG. 7c.

[0408] According to one embodiment, the fourth screen (2804) may further display a menu (not shown) for loading content (e.g., photos, pictures, still images, videos, documents) stored in the memory (230) or registered in a user account. The electronic device (200) may load content desired by the user in response to a user input for the menu and use the content as reference content (e.g., reference image).

[0409] For example, the electronic device (200) may load a picture to be used as reference content in response to a user input for the menu, and display the picture. The user of the electronic device (200) may draw a sketch on the picture. For example, the electronic device (200) may transmit the picture to an artificial intelligence model (e.g., the second artificial intelligence model (320) of FIG. 3) as additional information to be referenced in the style of an image to be generated. The artificial intelligence model may determine the style of the picture (e.g., watercolor, illustration) as the style of the image to be generated.

[0410] According to one embodiment, the fourth screen (2804) may further include an input window (not shown) for inputting text to be used as additional information for image generation. For example, a user may input text to be used as additional information (e.g., "cat picture", "draw with a beach in the background", "make the breed XX", related link (url), title of reference document (e.g., paper title)) through the input window. The electronic device (200) may transmit the text to an artificial intelligence model (e.g., the second artificial intelligence model (320) of FIG. 3) so that the text may be used as additional information for image generation.

[0411] According to one embodiment, the fourth screen (2804) may further include a menu (not shown) that can determine characteristics of an image to be generated (e.g., at least one of resolution, size, format (e.g., extension), or type (e.g., still image type, slide show type, animation type, video type)). The characteristics of the image to be generated (e.g., at least one of resolution, size, format, or type) may be determined based on user input through the menu. The characteristics of the image to be generated may also be automatically determined based on the style of the image.

[0412] According to one embodiment, the electronic device (200) can receive drawing input through a drawing area (blank space) where no content is displayed.

[0413] According to one embodiment, the electronic device (200) may receive a drawing drawn on top of content while the content is being displayed in the drawing area. For example, the content being displayed may be content stored in the memory (230) of the electronic device (200) or registered in a user account (e.g., sketches, photos, drawings, images, videos, documents drawn by the user in the past). In this case, the electronic device (200) may transmit the content together with the drawing as additional information to an artificial intelligence model (e.g., the second artificial intelligence model (320) of FIG. 3) so that the content is reflected in image generation.

[0414] For example, in an extended reality (XR) environment provided by an electronic device (200), an image of an external environment may be displayed on an XR user interface (drawing area) of a touchscreen display (210), and a drawing may be drawn on the image of the external environment. In this case, the electronic device (200) may transmit the image of the external environment as additional information together with the drawing to an artificial intelligence model (e.g., the second artificial intelligence model (320) of FIG. 3), thereby allowing the image of the external environment to be reflected in the generation of an image corresponding to the drawing.

[0415] FIG. 29 is an example of a user interface for explaining a process in which an electronic device (200) receives a drawing through an edge region according to one embodiment.

[0416] According to one embodiment, the electronic device (200) may perform an operation (e.g., operation 410 of FIG. 4) of receiving a user's drawing through an edge area of ​​the touchscreen display (210).

[0417] Referring to FIG. 29, the electronic device (200) can display user interfaces such as a first screen (2901), a second screen (2902), a third screen (2903), and a fourth screen (2904) through a touchscreen display (210). The first screen (2901), the second screen (2902), the third screen (2903), and the fourth screen (2904) can be user interfaces for receiving drawings.

[0418] An electronic device (200) may display a first screen (2901) (e.g., a home screen) on a touchscreen display (210). An edge panel handler (2910) may be displayed on an edge area of ​​one side of the touchscreen display (210). The edge panel handler (2910) may be a visual element (or graphic element) for executing a shortcut function.

[0419] The electronic device (200) may display a second screen (2902) including an edge panel (2930) in response to a user input (e.g., a swipe gesture) (2920) to an edge region (or edge panel handler (2910)) of the touchscreen display (210). The edge panel (2930) may include at least one visual element representing an executable shortcut function.

[0420] At least one of the visual elements within the edge panel (2930) may be related to the image providing function. For example, the sketch transformation icon (2940) within the edge panel (2930) may be a visual element related to the image providing function.

[0421] When a user input (2950) for the sketch transformation icon (2940) is received, the electronic device (200) may display a third screen (2903).

[0422] The third screen (2903) may display text (e.g., "Draw freely") (2960) guiding the user to input a drawing.

[0423] The electronic device (200) may receive a drawing (2911) based on user input. When the drawing (2911) is received, the third screen (2903) may be switched to a fourth screen (2904). The fourth screen (2904) may include a first interface element (2971) for selecting the style of the image to be generated and an image generation button (2972).

[0424] The first interface element (2971) of FIG. 29 may correspond to the first interface element (721) of FIG. 7b. The image creation button (2972) of FIG. 29 may correspond to the image creation button (741) of FIG. 7c.

[0425] FIG. 30 is an example of a user interface for explaining a process in which an electronic device (200) according to one embodiment receives a drawing through a designated application.

[0426] According to one embodiment, the electronic device (200) may perform an operation (e.g., operation 410 of FIG. 4) of receiving a user's drawing using a specified application.

[0427] Referring to FIG. 30, the electronic device (200) can display user interfaces, such as a first screen (3001), a second screen (3002), a third screen (3003), and a fourth screen (3004), through the touchscreen display (210) while a designated application (e.g., a note application) is running. The first screen (3001), the second screen (3002), the third screen (3003), and the fourth screen (3004) are user interfaces for receiving drawings and may be execution screens of the same application (e.g., a note application).

[0428] An electronic device (200) can display a first screen (3001) on a touchscreen display (210). The electronic device (200) can receive a drawing (3011) based on a user input to the first screen (3001). A generative AI icon (3010) can be displayed in one area within the first screen (3001).

[0429] The electronic device (200) may display a first pop-up window (3030) on the first screen (3001) in response to a user input (3020) (e.g., a touch) to the generative AI icon (3010). The first pop-up window (3030) may include at least one visual element representing an executable function in conjunction with the generative AI model.

[0430] At least one of the visual elements within the first pop-up window (3030) may be related to the image providing function. The sketch transformation menu (3040) within the first pop-up window (3030) may be a visual element related to the image providing function.

[0431] When user input for the sketch transformation menu (3040) is received, the electronic device (200) can display a second screen (3002).

[0432] A second screen (3002) may display a second pop-up window (3050) containing text (e.g., "Draw around the drawing to be converted") guiding the user to select an area containing a drawing (3011) in a specified manner for image generation.

[0433] The electronic device (200) can receive a user input (3060) for selecting an area including a drawing (3011) (e.g., an area in the form of a closed curve surrounding the drawing (3011)) on the third screen (3003).

[0434] The electronic device (200) may select a drawing area (3070) in response to the user input (3060) and display a fourth screen (3004) representing the drawing area (3070). The fourth screen (3004) may include a first interface element (3081) for selecting a style of an image to be generated for the drawing area (3070) and an image generation button (3082).

[0435] The first interface element (3081) of FIG. 30 may correspond to the first interface element (721) of FIG. 7b. The image creation button (3082) of FIG. 30 may correspond to the image creation button (741) of FIG. 7c.

[0436] FIG. 31 is an example of a user interface for explaining an image providing function of an electronic device (200) according to one embodiment.

[0437] Referring to FIG. 31, the electronic device (200) can display user interfaces such as a first screen (3101), a second screen (3102), a third screen (3103), a fourth screen (3104), a fifth screen (3105), and a sixth screen (3106) through a touchscreen display (210).

[0438] The first screen (3101) may be a user interface for receiving a drawing and / or generating an image. The first screen (3101) may include a drawing area for receiving a drawing input from a user, and / or visual elements (3110) indicating functions that can be executed in relation to the drawing. For example, the functions that can be executed in relation to the drawing may include at least some of the following: clearing the drawing, undo / redo, redrawing the drawing, saving / sharing the drawing, transparency, minimizing, copying, settings, or closing the drawing.

[0439] The first screen (3101) may include a first interface element (3151) for selecting the style of the image to be generated, a second interface element (3153) for selecting (or setting) similarity, and an image generation button (3152).

[0440] According to one embodiment, the electronic device (200) can display in real time a drawing (3111, 3112) drawn by a user with an electronic pen (500) (or a finger). For example, the drawing (3111, 3112) can include a first drawing (3111) corresponding to a sketch (or picture) and a second drawing (3112) corresponding to text (e.g., handwriting).

[0441] In one embodiment, the electronic device (200) can use the second drawing (3112) together with the first drawing (3111) to generate an image. For example, the first drawing (3111) may be a sketch of a cup (mug). The second drawing (3112) may be a handwritten input of "Coffee." The electronic device (200) can obtain at least one image (3113, 3114, 3115, 3116) of a cup containing coffee (e.g., an image reflecting the color of the coffee) using the first drawing (3111) sketching the cup and the second drawing (3112) titled "Coffee."

[0442] According to one embodiment, the electronic device (200) can select an image style (a style of an image to be generated) based on a user input (e.g., a touch) to the first interface element (3151). The electronic device (200) can request an artificial intelligence model (e.g., the second artificial intelligence model (320) of FIG. 3) to generate an image based on a user input selecting an image generation button (3152).

[0443] According to one embodiment, the electronic device (200) may select (or set) the similarity between the drawing (3111, 3112) and the image to be generated, based on user input to the second interface element (3153).

[0444] For example, if a touch is made on the image generation button (3152) while the option corresponding to the second interface element (3153) in the first screen (3101) is unchecked (or disabled), the similarity may be set to a first value (a first default value). For example, if a touch is made on the image generation button (3152) while the option corresponding to the second interface element (3153) in the first screen (3101) is checked (or enabled) by the user, the similarity may be set to a second value (a second default value) lower than the first value.

[0445] According to one embodiment, the electronic device (200) may select (or set) the similarity based on a style selected through the first interface element (3151) and / or a user input (e.g., whether to check an option) for the second interface element (3153).

[0446] For example, if a touch is made on the image generation button (3152) while the option corresponding to the second interface element (3153) in the first screen (3101) is unchecked (or disabled), the similarity may be set to a third value according to the style selected through the first interface element (3151). The third value may be a value obtained by adjusting (e.g., moving up / down) the first value (first default value) according to the selected style. For example, if a touch is made on the image generation button (3152) while the option corresponding to the second interface element (3153) in the first screen (3101) is checked (or enabled), the similarity may be set to a fourth value according to the style selected through the first interface element (3151). The fourth value may be a value obtained by adjusting (e.g., moving up / down) the second value (second default value) according to the selected style.

[0447] The electronic device (200) can display a second screen (3102) based on a user input to a first interface element (3151), a user input to a second interface element (3153), and a user input to an image generation button (3152).

[0448] The second screen (3102) may provide text and visual elements (3120) with an animated effect indicating that an image is being generated. For example, the text indicating that an image is being generated may be at least one of the following phrases: "Breathing warm air into the sketch...", "Adding a special touch with artistic flair...", or "Becoming a great work of art..."

[0449] According to one embodiment, the text indicating that an image is being generated on the second screen (3102) may include information about the image style selected by the user through the first interface element (3151). For example, if the selected image style is 'Watercolor', the phrase 'Breathing warm air into the sketch with watercolor style...' may be displayed. If the selected image style is 'Pop Art', the phrase 'Adding something special to the pop art style with an artistic sense...' may be displayed. If the selected image style is '3D Cartoon', the phrase 'Becoming a wonderful 3D cartoon work...' may be displayed.

[0450] The electronic device (200) can obtain at least one image (3113, 3114, 3115, 3116) corresponding to a drawing (3111, 3112) input by a user. The electronic device (200) can display the obtained at least one image (3113, 3114, 3115, 3116).

[0451] The electronic device (200) may display a first image (3113) and a portion of a second image (3114) on a third screen (3103). The third screen (3103) may include an indicator (3130) that indicates a drawing area, the number of acquired images (3113, 3114, 3115, 3116), and their positions (e.g., pages 1, 2, 3, and 4 out of a total of 4 pages).

[0452] The electronic device (200) can sequentially display a fourth screen (3104), a fifth screen (3105), and a sixth screen (3106) according to a user's specified input (e.g., a swipe gesture).

[0453] FIG. 32 is an example of a user interface for explaining a process for determining a style by an electronic device (200) according to one embodiment.

[0454] The first screen (3201) can display a drawing (3251) entered by the user.

[0455] The first screen (3201) may include a first interface element (3231) for selecting an image style (the style of the image to be generated) and an image generation button (3232). The first interface element (3231) may correspond to the first interface element (721) of FIG. 7B. The image generation button (3232) may correspond to the image generation button (741) of FIG. 7C.

[0456] The electronic device (200) can select an image style (style of an image to be generated) based on a user input (3252) for a first interface element (3231) while the first screen (3201) is displayed. The electronic device (200) can display a second screen (3202) based on the user input (3252).

[0457] When a user input (3252) for selecting a main menu (e.g., a dialog box) corresponding to a first interface element (3231) is received, the electronic device (200) may display a drop-down menu window (3220) including a list of submenus (3211, 3213, 3215, 3217, 3219) representing a plurality of styles on the second screen (3202). For example, the plurality of styles may include at least some of water color, illustration, sketch, pop art, or 3-dimensional cartoon. For example, each of the submenus (3211, 3213, 3215, 3217, 3219) may include at least some of text or image objects (3212, 3214, 3216, 3218, 3210) representing a specified style of an image to be generated. For example, each image object (3212, 3214, 3216, 3218, 3210) may be associated with a drawing (3251) and a specified style. For example, the image objects (3212, 3214, 3216, 3218, 3210) may correspond to a preview image that indicates how the drawing (3251) is generated with a specified image style (or indicates what style it is).

[0458] The electronic device (200) may, in response to a user input to the first interface element (3231), generate a fourth input prompt including at least one of image data (e.g., bmp data) of a specified first format for the drawing (3251) and / or string data of a second format. The electronic device (200) may obtain and display image objects (3212, 3214, 3216, 3218, 3210) representing multiple styles for the drawing (3251) by providing the fourth input prompt to the first artificial intelligence model (310) and / or the second artificial intelligence model (320) (e.g., an on-device artificial intelligence model of the electronic device (200).

[0459] According to one embodiment, one of the specified styles may be selected based on a user input to a drop-down menu window (3220). Thereafter, when a user input (e.g., a touch) occurs to an image generation button (3232), the electronic device (200) may add an image object for the selected style to the first input prompt and / or the second input prompt. The electronic device (200) may provide the first input prompt and / or the second input prompt with the added image object to the first artificial intelligence model (310) and / or the second artificial intelligence model (320) (e.g., an artificial intelligence model of an external server (e.g., the server (108) of FIG. 1)), thereby generating an image corresponding to the drawing (3251) based on the selected style.

[0460] FIG. 33 is an example of a user interface for explaining a process by which an electronic device (200) determines similarity based on transparency according to one embodiment.

[0461] According to one embodiment, the electronic device (200) may perform an operation (e.g., operation 440 of FIG. 4) of determining a similarity between the user's drawing and the image to be generated based on transparency. The similarity may correspond to a drawing reflection ratio (or drawing reflection ratio) for the image to be generated.

[0462] According to one embodiment, the electronic device (200) may provide a reference image to assist a user's drawing input through a drawing area. The electronic device (200) may determine a drawing reflection ratio based on the transparency of the drawing area in which the reference image is displayed.

[0463] Referring to FIG. 33, the electronic device (200) can display user interfaces such as a first screen (3301), a second screen (3302), a third screen (3303), a fourth screen (3304), a fifth screen (3305), and a sixth screen (3306) through a touchscreen display (210). The first screen (3301), the second screen (3302), the third screen (3303), the fourth screen (3304), the fifth screen (3305), and the sixth screen (3306) can be execution screens of a first application (e.g., a note application).

[0464] The drawing area of ​​the first screen (3301) may display a reference image as a background image along which the user can draw. The drawing area of ​​the first screen (3301) may display a More menu (3310).

[0465] The electronic device (200) may display a second screen (3202) including a pop-up window (3320) for submenus (e.g., transparency, minimize, settings) in response to a user input selecting a more menu (3310) within the first screen (3301).

[0466] The electronic device (200) can display a third screen (3303) including a transparency adjustment bar (3330) in response to a user input selecting transparency from among the submenus of a pop-up window (3320) in a second screen (3302).

[0467] The electronic device (200) can set (or adjust) the transparency of the drawing area according to user input (e.g., a swipe gesture) to the transparency adjustment bar (3330).

[0468] A user can set the transparency of the drawing area to a desired value through the transparency adjustment bar (3330). The electronic device (200) can set the transparency of the drawing area and display a fourth screen (3304) reflecting the transparency according to a user input (e.g., a swipe gesture) that adjusts the transparency through the transparency adjustment bar (3330).

[0469] When user input occurs in the drawing area, the electronic device (200) can display a fifth screen (3305).

[0470] A drawing (3351) can be input by a user input that follows a background image on the fifth screen (3305). When a user input of selecting an image generation button (3350) occurs after the drawing (3351) is input, the electronic device (200) can determine a different drawing reflection ratio for the image to be generated depending on the transparency of the drawing area.

[0471] According to one embodiment, the electronic device (200) may transmit a similarity corresponding to the user's drawing (3351) and the transparency of the drawing area to an artificial intelligence model (e.g., the second artificial intelligence model (320) of FIG. 3). The electronic device (200) may transmit a reference image to the artificial intelligence model as additional information to be additionally used in image generation.

[0472] According to one embodiment, as the transparency of the drawing area increases (becomes more transparent), the drawing reflection ratio for the image to be generated may decrease, and the reference image reflection ratio for the image may increase. As the transparency of the drawing area decreases (becomes more opaque), the drawing reflection ratio for the image to be generated may increase, and the reference image reflection ratio for the image may decrease.

[0473] For example, the first drawing reflection ratio when the transparency is set to 90% may have a lower value than the second drawing reflection ratio when the transparency is set to 30%. The first reference image reflection ratio when the transparency is set to 90% may have a higher value than the second reference image reflection ratio when the transparency is set to 30%.

[0474] The electronic device (200) can display an image (3361) generated based on a drawing (3351), transparency of the drawing area (or similarity corresponding to the transparency), and a reference image through a sixth screen (3306).

[0475] According to one embodiment, the electronic device (200) may additionally provide a menu (not shown) for linking with a second application, which is a background application, using the generated image (3361).

[0476] For example, the electronic device (200) may display a menu (e.g., add, send menu) that supports the use of the generated image (3361) by linking with a second application (e.g., a social network service (SNS) application, a gallery application) running in the background through the sixth screen (3306) so that the second application can use the generated image (3361). For example, if the second application is an SNS application, the generated image (3361) may be converted into an emoji / emoticon and provided to the SNS application. The image / emoticon may be added to a previous conversation in the SNS application, or may be recommended for transmission or sent to a previous conversation partner.

[0477] For example, if the second application is a gallery application, the generated image (3361) can be converted into a sticker and provided to the gallery application. The sticker can be utilized through the gallery application. For example, the sticker can be added to a previous image of the gallery application (an image displayed in the gallery application before displaying the first screen (3301) or immediately before drawing begins).

[0478] According to one embodiment, the electronic device (200) may additionally provide audio (e.g., sound effects, background music) associated with the generated image (3361).

[0479] For example, the audio may be audio that matches (or fits) the generated image (3361). The audio may reflect the style of the generated image (3361).

[0480] For example, the audio may include at least one of audio obtained from the electronic device (200) (e.g., music playing while drawing, audio collected through a microphone while drawing), audio stored in the electronic device (200), and audio generated by an artificial intelligence model trained to output audio based on a second input prompt (e.g., the second artificial intelligence model (320) of FIG. 3).

[0481] When displaying a generated image (3361) on a touchscreen display (210), the electronic device (200) may additionally display an indicator (e.g., a speaker icon) indicating that audio is included, and may play (or output) the audio according to a user input selecting the indicator. For example, the indicator may be displayed overlapping an area of ​​the generated image (3361).

[0482] FIG. 34 is a diagram illustrating a system (3400) including a generative artificial intelligence model according to one embodiment.

[0483] Referring to FIG. 34, the User Query / Response Interface (3410) can receive a user's input. The user's input may be in the form of natural language, images, and / or videos. Additionally, context information may also be transmitted when the user's input is transmitted. Context information may include various additional information at the time of user input. For example, information about the application the user is currently using or information about the user's location. Furthermore, the user's input may be in a mixed form of the aforementioned natural language, images, sounds, and context information. Furthermore, the user's input may also be in a non-natural language form, such as selecting a menu. The User Query / Response Interface (3410) can output the results of a generative artificial intelligence system to the user. The output may be in the form of natural language or specific content, and may also be provided in the form of an action requested by the user. The User Query Interface can output the results of a generative artificial intelligence system to the user. The output may be in the form of natural language or specific content, and may also be provided in the form of an action requested by the user.

[0484] The AI ​​framework (3420) can receive user input and coordinate and control each component necessary to perform the user's intention based on the user's query.

[0485] User input received from the User Query / Response Interface (3410) can be transmitted to the Prompt design component (3421). The Prompt design component (3421) can be used to generate a prompt suitable for inputting the user input into a Large Language Model (LLM) or a large multimodal model (LMM). The Prompt design component (3421) can be an AI component that uses a machine learning algorithm or a neural network to develop better prompts over time. The Prompt design component (3421) can access a knowledge component (e.g., knowledge repositories (3440)) containing user preference data, a prompt library, and prompt examples based on the user input to generate a prompt, and transmit the generated prompt to the LLM or LMM.

[0486] The API / Plug-in management component (3423) can communicate with external information when there is a request for additional information when passing user input as input to a generative model. The API / Plug-in management component (3423) can establish a channel for communicating with the outside of the AI ​​Interface through the API, and can enable access to various data sources (e.g., knowledge repositories (3440)) through the established channel. In addition, if the API / Plug-in management component (3423) needs to perform an action that performs the user input as a final result rather than an intermediate result in an application or service, it can request the action to the application / service component (3430) through the API. Information obtained from an external source can be used to generate a prompt in the prompt design component (3421) together with the user input, or can be passed as input to the generative model.

[0487] The Refiner component (e.g., the output modification component (3425)) can fine-tune the output from a generative model. For example, the Refiner component can verify that the content generated by the LLM and / or LMM is not irrelevant, biased, or harmful. Furthermore, the Refiner component can determine the degree to which the output matches the user's desired result and, if necessary, perform additional processing. The Refiner component can also configure and provide hints to the user to avoid undesirable output.

[0488] Generative AI Model (3450) can generally refer to an artificial intelligence neural network that creates new types of data based on user input information. Generative AI Model (3450) can include an image generation model and / or a language generation model. Representative models for generating images include a generative adversarial network (GAN) and a variational autoencoder (VAE), and examples include a diffusion-based generative model that uses a VAE and a transformer structure. A language generation model is a model that is trained to statistically output the most appropriate output value based on input values, and representative examples include models such as CHAT-GPT 3 and CHAT-GPT 4. In addition, there is also an LMM that can recognize various types of data input, such as text, images, and voice, and generate new data corresponding to them.

[0489] According to one embodiment, the electronic device (101) of FIG. 1 and / or the electronic device (200) of FIG. 2 may be configured to include at least a portion of the User Query / Response Interface (3410), the AI ​​framework (3420), the application / service component (3430), the knowledge repositories (3440), or the Generative AI Model (3450) of FIG. 34. According to one embodiment, at least a portion of the User Query / Response Interface (3410), the AI ​​framework (3420), the application / service component (3430), the knowledge repositories (3440), or the Generative AI Model (3450) of FIG. 33 may be included in another electronic device (e.g., another user's electronic device (e.g., the electronic device (102, 104) of FIG. 1) and / or a server (e.g., the server (108) of FIG. 1).

[0490] In the present disclosure, image providing methods according to various embodiments have been described with reference to the drawings of FIGS. 4, 6, 8, 9, 12, 14, 16, 19, and 27. The operations illustrated in the drawings may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. In some embodiments, some of the illustrated operations may be omitted, some operations may be combined, the order of some operations may be changed, or other operations may be added. At least some of the operations of the image providing methods according to various embodiments described with reference to FIGS. 4, 6, 8, 9, 12, 14, 16, 19, and 27 may correspond to each other. At least some of the operations may be performed in combination with each other.

[0491] In various embodiments of the present disclosure, interface elements, visual elements, graphic elements, menus, pop-up windows, menu windows, icons, dialog boxes, etc. depicted as components of a user interface or screen are merely examples, and the types, numbers, interactions, arrangements, or layouts of the components may be variously modified, altered, or applied. A screen according to various embodiments of the present disclosure may be various types of screens (e.g., a home screen, an application execution screen, a browser screen, a web screen, a full screen, a partial screen, a camera preview screen, an XR environment screen) that can be displayed on an electronic device or that can be displayed to include content (e.g., an image, a video, a document).

[0492] An electronic device (e.g., the electronic device (200) of FIG. 2) according to one embodiment may include a touchscreen display (e.g., the touchscreen display (210) of FIG. 2), at least one processor (e.g., the processor (220) of FIG. 2), and a memory (e.g., the memory (230) of FIG. 2) that stores instructions. The instructions, when executed by the at least one processor, may cause the electronic device to receive a drawing in response to a first user input through the touchscreen display, receive a second user input for generating an image through the touchscreen display, obtain description information about the drawing using the drawing based on the second user input, determine a similarity between the drawing and an image to be generated, obtain an image generated based on at least a portion of the description information and the similarity, and display the image through the touchscreen display.

[0493] In one embodiment, the similarity may be determined as a predetermined default value.

[0494] In one embodiment, a first interface element for selecting a style of the image to be generated from among a plurality of styles may be displayed on the touchscreen display. The similarity may be determined based on a style selected by a user input to the first interface element.

[0495] In one embodiment, a first interface element for selecting a style of the image to be generated from among a plurality of styles may be displayed on the touchscreen display. The length of the description information may be determined based on the style selected by user input to the first interface element.

[0496] In one embodiment, a second interface element for selecting the similarity may be displayed on the touchscreen display. The similarity may be determined based on a user input to the second interface element.

[0497] In one embodiment, the similarity may be determined based on the drawing skill level. If the skill level is low, the similarity may have a first value. If the skill level is high, the similarity may have a second value higher than the first value.

[0498] In one embodiment, the similarity may be determined based on the transparency of a drawing area for receiving the drawing.

[0499] According to one embodiment, the instructions, when executed by the at least one processor, may cause the electronic device to obtain the description information by providing a first input prompt including the drawing to a first artificial intelligence model in response to the second user input, and to obtain the image by providing a second input prompt including at least a portion of the description information and the similarity to a second artificial intelligence model.

[0500] According to one embodiment, the second input prompt may further include at least some of the drawing, image style information, reference text input by the user, reference image, form factor information of the electronic device, or form factor information of an external electronic device connected to the electronic device via short-range wireless communication.

[0501] According to one embodiment, the instructions, when executed by the at least one processor, may cause the electronic device to display a user interface including at least one keyword related to the description information, receive a user input for the at least one keyword through the user interface, and edit the description information based on the user input. The image may be acquired based on at least a portion of the edited description information.

[0502] According to one embodiment, the instructions, when executed by the at least one processor, may cause the electronic device to display a user interface for receiving the drawing on the touchscreen display upon occurrence of a triggering event. The triggering event may include at least one of a first event detecting a hovering input of an electronic pen, a second event detecting a user input selecting a specified visual element within an edge region of the touchscreen display, or a third event detecting a user input selecting a specified visual element within an application execution screen.

[0503] A method for providing an image by an electronic device (e.g., the electronic device (200) of FIG. 2) according to one embodiment may include an operation of receiving a drawing according to a first user input through a user interface displayed on a touchscreen display of the electronic device (e.g., the touchscreen display (210) of FIG. 2), an operation of receiving a second user input for generating an image through the user interface, an operation of obtaining description information about the drawing using the drawing based on the second user input, an operation of determining a similarity between the drawing and an image to be generated, an operation of obtaining an image generated based on at least a portion of the description information and the similarity, and an operation of displaying the image through the touchscreen display.

[0504] According to one embodiment, the user interface may include a first interface element for selecting a style of the image to be generated from among a plurality of styles. The similarity may be determined based on a style selected by a user input to the first interface element.

[0505] According to one embodiment, the user interface may include a first interface element for selecting a style of the image to be generated from among a plurality of styles. The length of the description information may be determined based on the style selected by user input to the first interface element.

[0506] In one embodiment, the user interface may include a second interface element for selecting the similarity. The similarity may be determined based on a user input to the second interface element.

[0507] In one embodiment, the similarity may be determined based on the drawing skill level. If the skill level is low, the similarity may have a first value. If the skill level is high, the similarity may have a second value higher than the first value.

[0508] In one embodiment, the similarity may be determined based on the transparency of a drawing area for receiving the drawing.

[0509] In one embodiment, the operation of obtaining the description information may be performed by providing a first input prompt including the drawing to a first artificial intelligence model. The operation of obtaining the image may be performed by providing a second input prompt including at least a portion of the description information and the similarity to a second artificial intelligence model.

[0510] According to one embodiment, the second input prompt may further include at least some of the drawing, image style information, reference text input by the user, reference image, form factor information of the electronic device, or form factor information of an external electronic device connected to the electronic device via short-range wireless communication.

[0511] A non-transitory computer-readable recording medium according to one embodiment may record a program for executing a method including an operation of receiving a drawing according to a first user input through a user interface displayed on a touchscreen display (e.g., a touchscreen display (210) of FIG. 2) of an electronic device (e.g., an electronic device (200) of FIG. 2), an operation of receiving a second user input for generating an image through the user interface, an operation of obtaining description information about the drawing using the drawing based on the second user input, an operation of determining a similarity between the drawing and an image to be generated, an operation of obtaining an image generated based on at least a portion of the description information and the similarity, and an operation of displaying the image through the touchscreen display.

[0512] According to various embodiments of the present disclosure, it is possible to support a user to express a desired direction through a simple drawing without having to learn a complex prompt structure or perform cumbersome text input.

[0513] According to various embodiments of the present disclosure, an image that accurately reflects a user's intention can be implemented while improving user convenience.

[0514] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly understood by a person having ordinary skill in the art to which the present disclosure belongs from the above description.

[0515] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0516] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0517] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0518] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0519] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0520] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In electronic devices, Touchscreen display; At least one processor comprising processing circuitry; and Contains memory that stores instructions, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Receive a drawing according to a first user input through the above touchscreen display, Receive a second user input for image generation via the above touchscreen display, Obtaining description information of the drawing based on the second user input, Determine the similarity between the above drawing and the image to be generated, Obtaining an image generated based on at least a portion of the above description information and the above similarity, An electronic device that displays the image through the touch screen display.

2. In claim 1, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An electronic device that determines the above similarity as a set default value.

3. In claim 1, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: On the touch screen display, a first interface element for selecting a style of the image to be generated among a plurality of styles is displayed, An electronic device that determines the similarity based on a style selected by user input for the first interface element.

4. In claim 1, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: On the touch screen display, a first interface element for selecting a style of the image to be generated among a plurality of styles is displayed, An electronic device that determines the length of the description information based on a style selected by user input for the first interface element.

5. In claim 1, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: On the above touch screen display, display a second interface element for selecting the similarity, An electronic device that determines the similarity based on a user input to the second interface element.

6. In claim 1, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: The similarity is determined based on the skill of the above drawing, If the above proficiency is at a low level, the similarity is determined as the first value, An electronic device in which the similarity is determined as a second value higher than the first value when the above proficiency is at a high level.

7. In claim 1, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An electronic device for determining the similarity based on the transparency of a drawing area for receiving the drawing.

8. In claim 1, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Obtaining the description information by providing a first input prompt including the drawing to the first artificial intelligence model in response to the second user input, An electronic device that obtains the image by providing a second input prompt including at least a portion of the description information and the similarity to a second artificial intelligence model.

9. In claim 8, The second input prompt above is, An electronic device further comprising at least some of the drawing, image style information, reference text input by a user, reference image, form factor information of the electronic device, or form factor information of an external electronic device connected to the electronic device via short-range wireless communication.

10. In claim 1, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Displaying a user interface that includes at least one keyword related to the above description information, Receiving user input for at least one keyword through the user interface, To edit the above description information based on the above user input, An electronic device wherein the image is obtained based on at least a portion of the edited description information.

11. In claim 1, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: When a triggering event occurs, display a user interface for receiving the drawing on the touchscreen display, An electronic device wherein the triggering event comprises at least one of a first event detecting a hovering input of an electronic pen, a second event detecting a user input selecting a specified visual element within an edge area of ​​the touchscreen display, or a third event detecting a user input selecting a specified visual element within an application execution screen.

12. In a method for an electronic device to provide an image, An action of receiving a drawing according to a first user input through a user interface displayed on a touch screen display of the electronic device; An action of receiving a second user input for image generation through the user interface; An operation of obtaining description information of the drawing based on the second user input; An operation for determining the similarity between the above drawing and the image to be generated; An operation of obtaining an image generated based on at least a portion of the above description information and the above similarity; and A method comprising an action of displaying the image via the touch screen display.

13. In claim 12, The operation of determining the similarity between the above drawing and the image to be generated is: An operation of displaying a first interface element for selecting a style of the image to be generated from among a plurality of styles through the user interface; and A method comprising an action of determining the similarity based on a style selected by user input for the first interface element.

14. In claim 12, The operation of obtaining the description information of the above drawing is: An operation of displaying a first interface element for selecting a style of the image to be generated from among a plurality of styles through the user interface; and A method comprising an action of determining the length of the description information based on a style selected by user input for the first interface element.

15. In claim 12, The operation of determining the similarity between the above drawing and the image to be generated is: An action of displaying a second interface element for selecting the similarity through the user interface; and A method comprising an action of determining the similarity based on a user input to the second interface element.

Images