Method for using attribute of drawing input as prompt and electronic device therefor

Abstract

Various embodiments of the present invention may comprise: a communication module (190); a display (160); a memory (130) for storing instructions; and a processor (120). The instructions, when executed by the processor, cause the electronic device to: detect a drawing input on content displayed on the display; display the detected drawing input on the content; receive a request for generating an object corresponding to the detected drawing input; analyze the content and the detected drawing input in response to the request; generate a prompt on the basis of at least one of information about the content, the position of the drawing input, or an input attribute value of the drawing input as the result of the analysis; and generate the object corresponding to the detected drawing input on the content on the basis of the generated prompt. Various embodiments are possible.

Description

Method of utilizing the properties of drawing input as a prompt and its electronic device

[0001] Various embodiments of the present disclosure disclose methods and electronic devices for utilizing properties of drawing input as prompts.

[0002] With the advancement of digital technology, various types of electronic devices, such as mobile terminals, personal digital assistants (PDAs), electronic notebooks, smartphones, tablet PCs (personal computers), and wearable devices, are becoming widely used. These electronic devices are constantly undergoing improvements in their hardware and / or software to support and enhance their functionality.

[0003] For example, an electronic device can use voice recognition technology to control the execution of an electronic device or application through a user's voice commands, or perform functions using a web-based service. Alternatively, the electronic device can connect to a wireless input / output device (e.g., earphones, headphones) via short-range wireless communication such as Bluetooth, and output the sound of music or videos through the wireless input / output device. Alternatively, an electronic device (e.g., a smartphone) can connect to a wearable display device (e.g., AR glasses) to provide extended reality (XR) content such as virtual reality (VR), augmented reality (AR), and / or mixed reality (MR).

[0004] In one embodiment, a method and device may be disclosed for detecting a drawing input directly drawn by a user on content, analyzing the content and the drawing input, generating a prompt based on the analysis information, and generating and providing an object corresponding to the drawing input on the content based on the generated prompt.

[0005] An electronic device (101) according to an embodiment of the present disclosure includes a communication module (190), a display (160), a memory (130) for storing instructions, and a processor (120), wherein the instructions, when executed by the processor, cause the electronic device to detect a drawing input on content displayed on the display, receive a request to display the detected drawing input on the content, and create an object corresponding to the detected drawing input, analyze the content and the detected drawing input in response to the request, generate a prompt based on at least one of information of the content, a location of the drawing input, or an input attribute value of the drawing input as a result of the analysis, and generate an object corresponding to the detected drawing input on the content based on the generated prompt.

[0006] An operating method of an electronic device (101) according to an embodiment of the present disclosure may include an operation of detecting a drawing input on content displayed on a display (160) of the electronic device, an operation of displaying the detected drawing input on the content, an operation of receiving a request for generating an object corresponding to the detected drawing input, an operation of analyzing the content and the detected drawing input in response to the request, an operation of generating a prompt based on at least one of information of the content, a location of the drawing input, or an input attribute value of the drawing input as a result of the analysis, and an operation of generating an object corresponding to the detected drawing input on the content based on the generated prompt.

[0007] According to one embodiment, by receiving a drawing input drawn directly by a user on content and analyzing the content and the drawing input itself, rather than text, to generate a prompt for generating an object corresponding to the drawing input, an object corresponding to the drawing drawn directly by the user can be provided that is close to the object desired by the user.

[0008] According to one embodiment, various information can be utilized for object creation by generating prompts based on the location, size, shape or input attribute values ​​of drawing input within the content.

[0009] In one embodiment, by receiving the style of the object from the user before requesting object creation for drawing input, the style of the object can be included in the prompt, so that the object can be created according to the style of the object desired by the user.

[0010] FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment.

[0011] FIG. 2 is a block diagram illustrating a generative artificial intelligence system according to one embodiment.

[0012] FIG. 3 is a drawing illustrating an example of creating and providing an object corresponding to a drawing input in an electronic device according to one embodiment.

[0013] Figure 4 is a flowchart illustrating an operating method of an electronic device according to one embodiment.

[0014] FIGS. 5A to 5C are diagrams illustrating an example of creating an object corresponding to a drawing input in an electronic device according to one embodiment.

[0015] FIG. 6 is a flowchart illustrating a method for defining an object corresponding to a drawing input in an electronic device according to one embodiment.

[0016] FIGS. 7A to 7C are diagrams illustrating an example of providing a visual cue for a drawing input in an electronic device according to one embodiment.

[0017] FIGS. 8A and 8B are drawings illustrating an example of providing a guide for drawing input in an electronic device according to one embodiment.

[0018] FIG. 9 is a flowchart illustrating a method for providing guidance for drawing input in an electronic device according to one embodiment.

[0019] FIGS. 10A and 10B are diagrams illustrating an example of providing a result image in an electronic device according to one embodiment.

[0020] FIGS. 11A and 11B are diagrams illustrating an example of providing a result image in an electronic device according to one embodiment.

[0021] FIGS. 12A and 12B are drawings illustrating an example of creating and providing an object corresponding to a drawing input in an electronic device according to one embodiment.

[0022] FIG. 13 is a diagram illustrating an example of generating an object corresponding to a drawing input in an electronic device according to one embodiment.

[0023] FIGS. 14A and 14B are drawings illustrating an example of creating and providing an object corresponding to a drawing input in an electronic device according to one embodiment.

[0024] FIGS. 15A and 15B are drawings illustrating an example of generating and providing an object corresponding to a drawing input in an electronic device according to one embodiment.

[0025] FIGS. 16A and 16B are drawings illustrating an example of generating and providing an object corresponding to a drawing input in an electronic device according to one embodiment.

[0026] FIGS. 17A to 17C are drawings illustrating an example of providing a sketch conversion function in an electronic device according to one embodiment.

[0027] FIG. 18 is a diagram illustrating an example of storing result content in an electronic device according to one embodiment.

[0028] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to various embodiments.

[0029] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0030] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0031] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0032] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0033] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0034] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0035] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0036] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0037] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0038] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0039] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0040] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0041] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0042] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0043] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0044] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0045] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0046] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0047] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0048] According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0049] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0050] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0051] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0052] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0053] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0054] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0055] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as a computer program product. The computer program product may be traded between sellers and buyers as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or may be provided through an application store (e.g., Play Store). TM ) or directly between two user devices (e.g., smart phones), online distribution (e.g., downloading or uploading). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0056] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0057] FIG. 2 is a block diagram illustrating a generative artificial intelligence system according to one embodiment.

[0058] Referring to FIG. 2, a generative artificial intelligence system according to one embodiment may include a user interface (210), a database (220), an application and service component (230), an AI (artificial intelligence) framework (250), and a generative AI model (270). According to one embodiment, the generative artificial intelligence system may be included in an electronic device (e.g., an electronic device (101) of FIG. 1) or an external server (e.g., an intelligent server (e.g., a server (108) of FIG. 1)).

[0059] The user interface (210) can receive a user query. The user query can be in the form of natural language, images, or videos. Additionally, context information can be transmitted along with the user query. As another example, the user query can also be a non-natural language input that does not generate natural language, such as a design request or modification. Furthermore, the query can be a mixed form of natural language, images, sounds, and context information as described above. Furthermore, the user interface (210) can output the results of the generative artificial intelligence system to the user. The output can be in the form of natural language or specific content, and can also be provided in the form of an action requested by the user.

[0060] The AI ​​framework (250) can receive a user query and coordinate and control each component necessary to carry out the user's intent. The AI ​​framework (250) may include a prompt design component (251), an application and plug-in management component (APIs / Plugins Management component) (253), and an output modification component (255).

[0061] A user query or action entered in the user interface (210) may be transmitted to a prompt design component (251). The prompt design component (251) may be used to generate prompts suitable for input into a large language model (LLM) or a large multimodal model. The prompt design component (251) may be an AI component that uses a machine learning algorithm or a neural network to develop better prompts over time. The prompt design component (251) may access a knowledge component containing user preference data, a prompt library, and prompt examples to generate prompts and transmit them to the large language model (LLM) or the large multimodal model (LMM).

[0062] The application and plugin management component (253) can communicate with external information when a request for additional information is made when user input is passed as input to the generative model. The application and plugin management component (253) establishes a channel for communication with the AI ​​Interface externally via an application programming interface (API), thereby enabling access to various data sources. Furthermore, if the application or service needs to perform an action that ultimately fulfills a user query, rather than an intermediate result, the application and plugin management component (253) can request such an action via the API. Information obtained from external sources can be passed as input to the generative model along with user input.

[0063] The output modification component (255) can fine-tune the output from the generative model. For example, the output modification component (255) can verify that content generated through a language model (LLM) or a large-scale multimodal model (LMM) is not irrelevant, does not contain biased content, or does not contain harmful content. In addition, the output modification component (255) can determine the degree to which the content matches the user's desired result and, if necessary, can proceed with additional processing. Additionally, the output modification component (255) can configure and provide hints to the user to avoid undesired output.

[0064] A generative AI model (270) generally refers to an artificial intelligence neural network that creates new types of data based on user input information. Representative models that generate images include generative adversarial networks (GANs) and variational autoencoders (VAEs). Recently, diffusion-based generative models that use VAE and transformer structures are also called generative models. In addition, language models are models trained to statistically output the most appropriate output based on input values, and representative examples include models such as CHAT-GPT 3 and CHAT-GPT 4. In addition, the generative AI model (270) can recognize various types of data input, such as text, images, and voice, and generate new data corresponding to them.

[0065] FIG. 3 is a drawing illustrating an example of creating and providing an object corresponding to a drawing input in an electronic device according to one embodiment.

[0066] Referring to FIG. 3, a processor (e.g., a processor (120) of FIG. 1) of an electronic device (e.g., an electronic device (101) of FIG. 1) according to an embodiment may display a first user interface (310) including content (301) (e.g., an image, a video) on a display (e.g., a display module (160) of FIG. 1). The content (301) may be stored in a memory (e.g., a memory (130) of FIG. 1) of the electronic device (101) or received (or downloaded) from an external device (or server). According to an embodiment, the content (301) may be obtained (or photographed) from a camera (e.g., a camera module (180) of FIG. 1) (e.g., preview content). Alternatively, the content may refer to the entire screen that can be displayed through the display module (160). For example, by drawing a picture on the content included in a web page displayed through an Internet browser application, or by drawing a picture on the screen of a running application displayed on a display module (160), and pressing the create button, an object corresponding to the drawing input can be created using an image captured from the screen of the running application as the target.

[0067] The processor (120) may receive a user input for selecting an edit menu (313) for editing content (301) while displaying the first user interface (310). For example, the edit menu (313) may be for receiving a drawing input for the content (301). When the edit menu (313) is selected, the processor (120) may provide a second user interface (330) for setting (or selecting) an input attribute value (or input attribute information) of the drawing input. Referring to the second user interface (530), the input attribute value of the drawing input may include at least one of a pen type (331), a pen thickness (333), a pen transparency (335), or a pen color (337). The user may set an input attribute value that matches a desired picture and draw (e.g., draw).

[0068] The processor (120) may receive a drawing input (353) from a user and display the drawing input (353) as in a third user interface (350). The processor (120) may display the drawing input (353) by overlaying it on content. The drawing input (353) may include a user input for drawing a picture (or an object). After receiving the drawing input (353), the processor (120) may receive a request to create an object corresponding to the drawing input (353). The request may be selecting a creation menu (351) (e.g., a software button) (or a creation item, a creation object) in the third user interface (350). When the creation menu (351) is selected, the processor (120) may analyze the content (301) and the drawing input (353). The processor (120) can analyze a subject included in the content (301) and identify location information (or coordinate information, pixel information) of the subject within the content (301). In addition, the processor (120) can analyze at least one of the location (or location information), size (or size information), shape, or input attribute value of the drawing input (353) within the content (301).

[0069] The processor (120) may define an object corresponding to the drawing input (353) based on at least one of the position, size, shape, or input attribute value of the drawing input (353). According to one embodiment, the processor (120) may define an object by further referring to previous drawing input history information in addition to the currently input drawing input information / attribute value in order to define an object corresponding to the drawing input. For example, the processor (120) may define an object corresponding to the drawing input by referring to a picture that the user frequently draws (or draws confidently), such as performing a similar drawing input on similar content, subject, or similar location within the subject as the current drawing input content. The processor (120) may define an object corresponding to the drawing input by further referring to / analyzing similarity with the current content, the distance between the subject within the content and the drawing input, the position, the previous drawing input history, or an object created by the previous drawing input.

[0070] The processor (120) may generate a prompt based on at least one of information of the content, the location of the drawing input, or the input attribute value of the drawing input as the analyzed result. The prompt may include at least one of content (e.g., the content itself), content including the drawing input (353), information of an object (e.g., a subject) within the content corresponding to the location where the drawing was input (e.g., a person, an eye), location information of the subject within the content, the location, size, shape, or input attribute value of the drawing input (353), or the above-defined object. An AI engine (e.g., the generative AI model (270) of FIG. 2) may generate an object (371) (e.g., glasses or sunglasses) corresponding to the drawing input (353) based on the generated prompt. The fourth user interface (370) may include an object (371) corresponding to the drawing input (353) within the content (302). The AI ​​engine may generate an object (371) based on the prompt and provide the user with content (302) including the object (371). Alternatively, the AI ​​engine may generate content (302) including the object (371). The AI ​​engine may generate the object (371) and may also generate the entire content (302) including the object (371). When generating the entire content including the object (371), a prompt may be generated to regenerate the content (302) by adding (or overlapping) an object (371) corresponding to the drawing input to the original content without modifying the original content.

[0071] According to one embodiment, the processor (120) may transmit the generated prompt to an intelligent server (e.g., the server (108) of FIG. 1) via a communication module (e.g., the communication module (190) of FIG. 1). When transmitting the generated prompt, the processor (120) may transmit the content (301) and the drawing input (353) to the server (108). The processor (120) may receive the content including the object (371) from the server (108). Alternatively, the processor (120) may transmit the area including the drawing input (353) in the content (301) as a masking area to the server (108) together with the generated prompt. The processor (120) may receive information including the object (371) from the server (108), and may include the object (371) in the content based on the information including the object (371) and provide the content to the user.

[0072] According to one embodiment, the processor (120) may provide a guide for object creation when the location of the drawing input is not a location where object creation is possible on the content (e.g., a face area or an eye, nose, or mouth). The guide may be intended to prevent AI from regenerating a human face and creating a face different from the original face when a drawing input is made within a human face.

[0073] According to one embodiment, the processor (120) may receive a selection of a style of an object corresponding to the drawing input from the user, and include the style of the selected object in the prompt. For example, the style of the object may be illustration, comic, watercolor, or pop art. In response to the request, the processor (120) may analyze the location of the detected drawing input on the content, determine whether the location of the detected drawing input is a location where an object can be created on the content, and if the location of the detected drawing input is not a location where an object can be created on the content, provide a guide for object creation. The guide may include at least one of text, an image, a video, or audio. In response to the request, the processor (120) may analyze the size of the detected drawing input, determine whether the size of the detected drawing input exceeds a specified size, and if the size of the detected drawing input exceeds a specified size, provide a guide for object creation. This may be to prevent the creation of an object corresponding to the drawing input from being impossible if the size of the drawing input is too small, or to prevent the object corresponding to the drawing input from obscuring the original content if the size of the drawing input is too large.

[0074] An electronic device (101) according to an embodiment of the present disclosure includes a communication module (190), a display (160), a memory (130) for storing instructions, and a processor (120), wherein the instructions, when executed by the processor, cause the electronic device to detect a drawing input on content displayed on the display, receive a request to display the detected drawing input on the content, and create an object corresponding to the detected drawing input, analyze the content and the detected drawing input in response to the request, generate a prompt based on at least one of information of the content, a location of the drawing input, or an input attribute value of the drawing input as a result of the analysis, and generate an object corresponding to the detected drawing input on the content based on the generated prompt.

[0075] The above instructions, when executed by the processor, may cause the electronic device to define an object corresponding to the drawing input based on at least one of information of the content, a location of the drawing input, or an input attribute value of the drawing input as the analyzed result.

[0076] The above instructions, when executed by the processor, may cause the electronic device to further analyze a size or shape of the detected drawing input and define an object corresponding to the drawing input based on the further analyzed size or shape of the drawing input.

[0077] The input attribute value of the above drawing input may include at least one of the pen type, color, thickness, or transparency.

[0078] The instructions, when executed by the processor, may cause the electronic device to include a style of a specified object in the prompt in response to the drawing input, or, if a style change of the specified object is requested from a user, to include a style of an object selected by the user in the prompt.

[0079] The instructions, when executed by the processor, may cause the electronic device to analyze the location of the detected drawing input on the content in response to the request, determine whether the location of the detected drawing input is a location where an object can be created on the content, and provide a guide for object creation if the location of the detected drawing input is not a location where an object can be created on the content (e.g., a face area or an eye, nose, or mouth). This may be to prevent AI from regenerating the human face and creating a face different from the original face when the drawing input is made within a human face.

[0080] The above instructions, when executed by the processor, may cause the electronic device to analyze the size of the detected drawing input in response to the request, determine whether the size of the detected drawing input exceeds a specified size, and, if the size of the detected drawing input exceeds the specified size, provide a guide for object creation. This may be to prevent an object corresponding to the drawing input from being created if the size of the drawing input is too small, or an object corresponding to the drawing input from obscuring original content if the size of the drawing input is too large.

[0081] The above instructions, when executed by the processor, may cause the electronic device to set an area including the detected drawing input in the content as a masking area, transmit the generated prompt and the masking area to an intelligent server (108), receive object information including an object generated in response to the detected drawing input from the intelligent server, and generate and provide content including the generated object in the content based on the received object information.

[0082] The above instructions, when executed by the processor, may cause the electronic device to transmit analysis information of the detected drawing input and the generated prompt to an intelligent server, receive object information including an object generated in response to the detected drawing input from the intelligent server, and generate and provide content including the generated object in the content based on the received object information.

[0083] The instructions, when executed by the processor, may cause the electronic device to create a first object corresponding to the detected drawing input, create a second object different from the first object corresponding to the detected drawing input, and, while displaying first content including the first object, display second content including the second object when a specified user input is detected.

[0084] The instructions, when executed by the processor, may cause the electronic device to display the first content and the second content as thumbnail content while displaying the first content including the first object.

[0085] Figure 4 is a flowchart (400) illustrating an operating method of an electronic device according to one embodiment.

[0086] Referring to FIG. 4, in operation 401, a processor (e.g., the processor 120 of FIG. 1) of an electronic device (e.g., the electronic device 101 of FIG. 1) according to an embodiment may detect a drawing input on content (e.g., an image, a video). The processor (120) may detect a drawing input on content. The processor (120) may receive a drawing input from a user while content (e.g., the content 301 of FIG. 3) is displayed on a display (e.g., the display module (160) of FIG. 1). The content may be stored in a memory of the electronic device (101) (e.g., the memory (130) of FIG. 1) or received (or downloaded) from an external device (or server). Depending on the embodiment, the content may be obtained (or photographed) by a camera (e.g., the camera module (180) of FIG. 1) (e.g., a preview image). Alternatively, the content may refer to the entire screen that can be displayed via the display module (160). For example, a drawing may be drawn on content included in a webpage displayed via an Internet browser application, or a drawing may be drawn on the screen of a running application displayed on the display module (160), and by pressing the create button, an object corresponding to the drawing input may be created using an image captured from the screen of the running application.

[0087] The above drawing input may include a user input for drawing a picture (or object). According to one embodiment, the processor (120) may receive (or select) input attribute values ​​(or input attribute information) of the drawing input from the user. The input attribute values ​​may include at least one of the type, color, thickness, or transparency of the pen.

[0088] In operation 403, the processor (120) can display the drawing input on the content. The processor (120) can display the drawing input on the display module (160) as it receives (or detects).

[0089] In operation 405, the processor (120) may receive a request to create an object corresponding to the drawing input. The user may request that the drawing drawn by the user be created by AI. For example, the request may be selecting a menu (or item, object) (e.g., a software button or a physical button) for object creation. As an example of the interface, in the fourth user interface (370) of FIG. 3, after the user inputs a drawing, the AI ​​assistant may be called using a designated input (e.g., a long press of the home button) and, when the user draws a circle / shape around the drawing input using a touch gesture, the AI ​​assistant may immediately operate to create an object corresponding to the drawing input within the content.

[0090] In operation 407, the processor (120) may analyze the content and the drawing input. The processor (120) may analyze a subject included in the content and identify location information of the subject within the content. The subject may refer to a person, an animal, or an object (e.g., a tree, the sea, a mountain, a bird, a desk). In addition, the processor (120) may analyze at least one of the location (or location information), size (or size information), shape, or input attribute value of the drawing input within the content. The processor (120) may define an object corresponding to the drawing input based on at least one of the location, size, shape, or input attribute value of the drawing input.

[0091] According to one embodiment, the processor (120) may define an object corresponding to a drawing input by further referring to previous drawing input history information in addition to the currently input drawing input information / attribute values. For example, the processor (120) may define an object corresponding to a drawing input by referring to a picture that the user frequently draws (or draws confidently), such as performing a similar drawing input on similar content, subject, or similar location within the subject as the current drawing input content. The processor (120) may define an object corresponding to a drawing input by further referring to / analyzing similarity with the current content, the distance between the subject within the content and the drawing input, the location, the previous drawing input history, or an object created by the previous drawing input.

[0092] According to one embodiment, the processor (120) may receive from the user the selection of the style of an object corresponding to the drawing input, and include the style of the selected object in the prompt. For example, the style of the object may be an illustration, a comic book, a watercolor painting, or pop art. The style of the selected object may be included in the input attribute value or may be included in the prompt. The AI ​​engine (or intelligent server) may generate different objects depending on the position of the drawing input. According to one embodiment, even if the drawing input has the same size, shape, and input attribute value, if the position of the drawing input is different, the defined object may be different. For example, if a drawing input in the shape of a figure 8 lying sideways is input on the head of a person in an image of a person including a face, the generated object may be a butterfly-shaped hairpin, and if it is drawn on the person's upper clothes, the generated object may be a butterfly-shaped brooch.

[0093] The processor (120) may also set the pen type as the style of an object corresponding to a drawing input. The processor (120) may reflect pen property information on all or part of the original content (background, object) and display it according to the pen property (pen type). When coloring the entire content with a pen of a specified pen type, the processor (120) may change the entire content by reflecting an effect (style) according to the property of the pen of the specified pen type. For example, when the pen type is a fountain pen, the processor (120) may set the style of the object to a sketch style, and when the pen type is a brush pen, the processor (120) may set the style of the object to a watercolor style. When drawing the outline of a specific object with a pen, the processor (120) may also change the pen property only to the object.

[0094] According to one embodiment, the processor (120) may analyze the location of the detected drawing input on the content in response to the request, determine whether the location of the detected drawing input is a location where an object can be created on the content, and if the location of the detected drawing input is not a location where an object can be created on the content, provide a guide for object creation (e.g., you cannot draw at that location). The guide may include at least one of text, content, video, or audio. The processor (120) may analyze the size of the detected drawing input in response to the request, determine whether the size of the detected drawing input exceeds a specified size, and if the size of the detected drawing input exceeds the specified size, provide a guide for object creation (e.g., please draw smaller, please draw bigger). This may be to prevent an object corresponding to the drawing input from being created if the size of the drawing input is too small, or to prevent an object corresponding to the drawing input from obscuring the original content if the size of the drawing input is too large.

[0095] In operation 409, the processor (120) may generate a prompt based on the analyzed information (or result). The processor (120) may generate a prompt based on at least one of information of the content, the location of the drawing input, or the input attribute value of the drawing input as the analyzed result. The prompt may include content (e.g., the content itself or descriptive information (text) describing the content), content including the drawing input, information of an object (e.g., a subject) within the content corresponding to the location where the drawing is input, location information of the subject within the content, the location, size, shape, or input attribute value of the drawing input (353), or at least one of the objects defined above. The processor (120) may include an object desired to be drawn in the content, a location, size, shape, and input attribute value at which the object within the content is to be drawn, in the prompt based on the analyzed information.

[0096] In operation 411, the processor (120) may generate an object corresponding to the drawing input in the content based on the generated prompt. For example, the object may be a cat, a dolphin, or a pin or tie worn on the head of a person. For example, an AI engine included in the processor (120) (e.g., the generative AI module (270) of FIG. 2) may generate an object based on the prompt and provide content including the generated object. Alternatively, the AI ​​engine may generate content including the generated object.

[0097] According to one embodiment, the processor (120) may transmit the generated prompt to an intelligent server (e.g., the server (108) of FIG. 1) via a communication module (e.g., the communication module (190) of FIG. 1). When transmitting the generated prompt, the processor (120) may transmit the content and the drawing input to the server (108). The processor (120) may receive the content including the generated object from the server (108). Alternatively, the processor (120) may transmit an area including the drawing input in the content as a masking area to the server (108) together with the generated prompt. The processor (120) may receive information including an object that is constantly generated from the server (108), and may include the generated object in the content based on the information including the generated object and provide the content to the user. The processor (120) may set an area including the drawing input in the content as a masking area, transmit the generated prompt and the masking area to the server (108), receive object information including an object generated in response to the drawing input from the server (108), and generate and provide content including the generated object in the content based on the received object information.

[0098] According to one embodiment, the AI ​​engine may create multiple different objects when creating the object. For example, if the object is a white cat, the AI ​​engine may create multiple white cats with different shapes (or appearances) or multiple white cats with different angles (or inclinations). When a specified user input is detected while displaying first content including a first object, the processor (120) may display second content including a second object different from the first object. The first object and the second object may be the same object, but may have different shapes, angles, or positions. The first content and the second content may be the same content (e.g., the content displayed in operation 401), but since the included objects are different, they may be described as first and second. The processor (120) may display the first content and the second content as thumbnail images while displaying the first content including the first object.

[0099] FIGS. 5A to 5C are drawings illustrating an example of generating an object corresponding to a drawing input in an electronic device according to one embodiment.

[0100] Referring to FIG. 5A, a processor (e.g., a processor (120) of FIG. 1) of an electronic device (e.g., an electronic device (101) of FIG. 1) according to an embodiment may display a first user interface (510) including content (501) on a display (e.g., a display module (160) of FIG. 1). When a user selects an editing mode, the first user interface (510) may include a first guide (511) (or a guide message) guiding editing of the content (501). The second user interface (520) may include a menu for editing the content (501) and a second guide (521) for drawing input from the user. The first guide (511) or the second guide (521) may include at least one of text, content, video, or audio. The processor (120) can display the first guide (511) as text on the display module (160) and output audio corresponding to the first guide (511) to a speaker (e.g., the audio output module (155) of FIG. 1).

[0101] The above menu may include at least one menu (or item, object) of pen type, pen color, pen thickness, or pen transparency. The pen type may include, for example, at least one of basic pen, fountain pen, colored pencil, marker, highlighter, mosaic pen, blur pen, and text word art pen. The color of the pen may include 50, 100, 256 colors, etc. The thickness of the pen may be classified into 0 to N levels (e.g., N is 100, 200, etc.), and the transparency of the pen may be classified into 0 to K levels (e.g., K is 50, 100).

[0102] When the pen menu (523) is selected from the menus of the second user interface (520), the processor (120) may provide a third user interface (530). The third user interface (530) may include a content (501) and a pen color menu (531). The user may select (or set) at least one of the pen type, pen color, pen thickness, or pen transparency through the menu. The third user interface (530) illustrates a pen color menu (531), but may also include a pen type and pen thickness menu. The user may select a color that matches (or is suitable for) his or her drawing and draw. The fourth user interface (540) illustrates an example in which a drawing input (541) is detected in the content (501). Referring to the fourth user interface (540), when the drawing input (541) is completed, the user may select a creation menu (543). The creation menu (543) may request the creation of an object corresponding to the drawing input (541).

[0103] Referring to FIG. 5B, the processor (120) may provide a fifth user interface (550) while generating an object corresponding to a drawing input (541). The fifth user interface (550) may include text and visual objects that guide that an object corresponding to the drawing input (541) is being generated. According to one embodiment, audio corresponding to the text that guides that an object is being generated may be output through a speaker (e.g., the audio output module (155) of FIG. 1). While providing the fifth user interface (550), the processor (120) may analyze the content (501) and the drawing input (541), and generate a prompt based on the analyzed information. The processor (120) may analyze an object (e.g., bread, a plate (or tray), a shopping bag) included in the content (501), and identify location information of the object within the content (501). Additionally, the processor (120) can analyze at least one of the position, size, shape, or input attribute value of the drawing input (541) within the content (501). The input attribute value can include at least one of the type, color, thickness, or transparency of the pen. The processor (120) can define a corresponding object of the drawing input (541) based on at least one of the position, size, shape, or input attribute value of the drawing input (541).

[0104] The processor (120) may generate a prompt including an object to be drawn on the content (501) based on the analyzed information, a location, size, shape, and input attribute values ​​of the object to be drawn within the content. The processor (120) may generate an object corresponding to the drawing input (541) in the content (501) through an AI engine (e.g., the generative AI module (270) of FIG. 2) based on the generated prompt. When the object generation is completed, the processor (120) may provide a sixth user interface (560). The sixth user interface (560) may include an object (563) corresponding to the drawing input (541) in the content (501). The object (563) may be 'strawberry'. What the AI ​​engine generates may be an object (563) corresponding to the drawing input (541) or content including the object (563). However, since the content is identical to the original content except for the object (563), the same drawing symbol as the original content can be used here for the drawing symbol of the content.

[0105] When a completion item (561) is selected in the sixth user interface (560), the processor (120) may store the corresponding content, and when an edit item is selected for the stored content, a seventh user interface (570) for content editing may be provided. The seventh user interface (570) may include a menu related to editing new content that includes an object (573) in the content (501). The editing menu for the new content may include conversion (571) (e.g., converting the new content into the content (501)), and whether to apply a filter, tone, decoration, or tool to the new content.

[0106] Referring to FIG. 5C, the processor (120) may receive a user's selection (or setting) of a style of an object corresponding to a drawing input. For example, the style of the object may be illustration, comic book, watercolor, or pop art. The processor (120) may provide an eighth user interface (580) before a drawing input or an object creation request. The eighth user interface (580) may include a recommendation menu (581) (or item, object) and a creation menu (583). When the recommendation menu (581) is selected, the processor (120) may provide a ninth user interface (590). The ninth user interface (590) may include a style list (591) of objects. The style list (591) of objects may include styles of one or more objects, such as illustration, comic book, watercolor, or pop art. When the user selects watercolor as the style of the object, the processor (120) may provide a tenth user interface (595). The tenth user interface (595) may illustrate an example of selecting watercolor (597) as the style of an object. The processor (120) may receive a drawing input while the tenth user interface (595) is displayed, and when the creation of an object corresponding to the drawing input is requested, the style of the selected object may be included in the prompt.

[0107] FIG. 6 is a flowchart (600) illustrating a method for defining an object corresponding to a drawing input in an electronic device according to one embodiment. FIG. 6 may be an embodiment of operation 407 or 409 of FIG. 4.

[0108] Referring to FIG. 6, in operation 601, a processor (e.g., the processor (120) of FIG. 1) of an electronic device (e.g., the electronic device (101) of FIG. 1) according to an embodiment may analyze the location of a drawing input. The drawing input may correspond to a picture drawn by a user on content. The user may change (or set) input attribute values, such as the type, color, thickness, or transparency of a pen, to draw the picture he or she has in mind. The processor (120) may receive a drawing input from the user in a state set with the input attribute values, and analyze the location of the received drawing input. For example, the processor (120) may analyze content including a drawing input to identify where the location of the drawing input is within the content.

[0109] According to one embodiment, the processor (120) may determine whether the location of the drawing input is a location where object creation is possible on the content. For example, if the subject of the content is a 'person', an object may not be created inside the person's face. This may be to prevent AI from regenerating the person's face and creating a face different from the original face when a drawing input is made inside the person's face. If the location of the drawing input is not a location where object creation is possible on the content, the processor (120) may provide a guide for object creation (e.g., drawing is not possible at that location). If the location of the drawing input is inside a person, the processor (120) may provide a guide message indicating that drawing is not possible inside the person. The guide may include at least one of text, content, video, or audio.

[0110] In operation 603, the processor (120) may analyze the size or shape of the drawing input. The processor (120) may analyze whether the size of the drawing input exceeds (or is less than) a specified size of the entire size of the content, or whether the shape of the drawing input corresponds to a violent or sexual form. According to one embodiment, the processor (120) determines whether the size of the drawing input exceeds a specified size (e.g., 30%, 40%, 50% or more of the entire size of the content), and if the size of the drawing input exceeds (or is less than) the specified size, a guide for object creation (e.g., “Please draw smaller” or “Please draw larger”) may be provided. This may be to prevent an object corresponding to the drawing input from being created if the size of the drawing input is too small, or to prevent an object corresponding to the drawing input from obscuring the original content if the size of the drawing input is too large. The processor (120) may provide a guide for object creation (e.g., “You cannot draw that picture”) if the form of the analyzed drawing input corresponds to a specified form (e.g., violent or sexual).

[0111] In operation 605, the processor (120) may analyze the input attribute value of the drawing input. The input attribute value (or input attribute information) may include at least one of a pen type, a pen color, a pen thickness, or a pen transparency. The pen type may include, for example, at least one of a basic pen, a fountain pen, a colored pencil, a marker, a highlighter, a mosaic pen, a blur pen, and a text word art pen. The color of the pen may include 50, 100, 256, or the like. The pen thickness may be divided into steps 0 to N (e.g., N is 100, 200, etc.), and the transparency of the pen may be divided into steps 0 to K (e.g., K is 50, 100). The user may select a color that matches (or is suitable for) his / her drawing and draw the drawing. The processor (120) can recognize the type of pen, the color of the pen, the thickness of the pen, or the transparency of the pen as parameters, and utilize the input information as a prompt. For example, for the same drawing input, when the input attribute values ​​are fountain pen (e.g., pen type) and red (e.g., pen color), and when the input attribute values ​​are colored pencil (e.g., pen type) and blue (e.g., pen color), the objects may be different, or even the same objects may be created slightly differently. The processor (120) can identify (or extract) the input attribute values ​​of the drawing input.

[0112] In operation 607, the processor (120) may define an object corresponding to the drawing input based on the analysis result. For example, if a user draws a fish (e.g., drawing input) on an ocean background (e.g., content) with a basic pen or a colored pencil, the processor (120) may analyze the content, the position, size, shape, and input attribute values ​​of the drawing input to define the fish drawn by the user as a dolphin. Alternatively, if the user writes (e.g., drawing input) on an ocean background (e.g., content) with a fountain pen, the processor (120) may analyze the content, the position, size, shape, and input attribute values ​​of the drawing input to define the handwriting drawn by the user as calligraphy. The processor (120) may include the defined object, the position, size, shape, or input attribute values ​​of the drawing input in the prompt.

[0113] In one embodiment, the processor (120) may analyze the drawing input and related input attribute values ​​to define an object and include it in the prompt, but may also inquire an intelligent server (e.g., server (108) of FIG. 1) about what object corresponds to the drawing input. The processor (120) may include request information requesting a definition of the object in the prompt and request the server (108). The server (108) may define and generate the object and transmit it to the electronic device (101).

[0114] FIGS. 7A to 7C are diagrams illustrating an example of providing a visual cue for a drawing input in an electronic device according to one embodiment.

[0115] Referring to FIG. 7A, a processor (e.g., a processor (120) of FIG. 1) of an electronic device (e.g., an electronic device (101) of FIG. 1) according to an embodiment may display a first user interface (710) including a person image (701) (e.g., content, video) on a display (e.g., a display module (160) of FIG. 1). The first user interface (710) may include a person image (701) (or a photo) including a user's face. When a drawing input is received in the first user interface (710), the processor (120) may provide a second user interface (730) displaying the received drawing input. The second user interface (730) may display (include) a drawing input (731) in the person image (701). The user may continue to draw.

[0116] Referring to FIG. 7b, a third user interface (740) illustrates an example of including a first drawing input (741) in a person image (701). A fourth user interface (750) illustrates an example of including a second drawing input (751) in a person image (701). The second drawing input (751) may be input at a different location than the first drawing input (741).

[0117] Referring to FIG. 7C, when an object creation request is made, the processor (120) may set a masking area (763) including a first drawing input (741) and a second drawing input (751). The processor (120) may analyze the size or shape of the masking area (763). The processor (120) may analyze whether the size of the masking area (763) exceeds a specified size in the overall size of the person image (701) or whether the shape of the drawing input included in the masking area (763) corresponds to a violent or sexual form. If the size of the masking area (763) exceeds the specified size, the processor (120) may provide a fifth user interface (760) including a guide (761) for object creation. Alternatively, the processor (120) may set each of the first drawing input (741) and the second drawing input (751) as a masking area. The processor (120) can analyze whether the size of the masking area (773) including the second drawing input (751) exceeds a specified size from the overall size of the person image (701) or whether the shape of the second drawing input (751) corresponds to a violent or sexual shape. If the size of the masking area (773) exceeds the specified size, the processor (120) can provide a sixth user interface (770) including a guide (771) for object creation. The guide can include at least one of text, an image, a video, or audio.

[0118] FIGS. 8A and 8B are drawings illustrating an example of providing a guide for drawing input in an electronic device according to one embodiment.

[0119] Referring to FIG. 8A, a processor (e.g., a processor (120) of FIG. 1) of an electronic device (e.g., an electronic device (101) of FIG. 1) according to an embodiment may receive a drawing input (811) on content (801) and then select a creation menu (813) from a user. The first user interface (810) may illustrate an example of providing a first guide message (815) when the size of a drawing input (811) detected on content (801) is less than a first reference value (e.g., 1% of the total size of the content). The processor (120) may receive a first drawing input (831), a second drawing input (832), a third drawing input (833), and a fourth drawing input (834), and then select a creation menu (837) from the user. The processor (120) may set a masking area (835) including a first drawing input (831), a second drawing input (832), a third drawing input (833), and a fourth drawing input (834). The second user interface (830) may illustrate an example of providing a second guide message (839) when the masking area (835) exceeds a second criterion (e.g., 30% or 50% of the total content size).

[0120] Referring to FIG. 8B, the processor (120) may receive a drawing input (851) on a person image (801) and then receive a selection of a creation menu (857) from the user. The third user interface (850) may illustrate an example of providing a third guide message (859) when the location of the drawing input (851) detected on the content (801) is not a location where an object can be created. The processor (120) may set a masking area (853) including the drawing input (851), and if the masking area (853) includes the inside of the face of the person image (801), it may be determined that the location is not a location where an object can be created. This may be to prevent AI from regenerating the person's face and creating a face different from the original face when a drawing input is made inside a person's face. In this case, the processor (120) may inform the user to draw in a different location by providing the third guide message (859). Alternatively, the processor (120) may analyze whether the form of the drawing input corresponds to a violent or sexual form. If the form of the drawing input corresponds to a violent or sexual form, the processor (120) may provide a fourth user interface (870). The fourth user interface (870) may be an example that includes a fourth guide message (871) that guides that an object corresponding to the drawing input cannot be created.

[0121] FIG. 9 is a flowchart (900) illustrating a method for providing guidance for drawing input in an electronic device according to one embodiment.

[0122] Referring to FIG. 9, in operation 901, a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to an embodiment may receive a request for creating an object corresponding to a drawing input. The processor (120) may be requested to create an object corresponding to a drawing input by receiving a selection of a creation menu from a user.

[0123] In operation 903, the processor (120) may analyze the location of a drawing input. The drawing input may correspond to a picture drawn by the user on the content. The user may change (or set) input attribute values, such as the type, color, thickness, or transparency of the pen, to draw the picture he or she has in mind. The processor (120) may receive a drawing input from the user while the input attribute values ​​are set, and analyze the location of the received drawing input. For example, the processor (120) may analyze content including the drawing input to identify the location of the drawing input within the content.

[0124] In operation 905, the processor (120) may determine whether object creation is possible based on the analyzed result. The processor (120) may determine whether the location of the drawing input is a location where object creation is possible on the content. For example, if the subject in the content is a 'person', an object may not be created inside the person's face. If the location of the drawing input is capable of object creation, the processor (120) may perform operation 907, and if the location of the drawing input is not capable of object creation, the processor (120) may perform operation 906.

[0125] If the location of the drawing input is not a location where object creation is possible, in operation 906, the processor (120) may provide a first creation guide. If the location of the drawing input is not a location where object creation is possible on the content, the processor (120) may provide a guide for object creation (e.g., drawing is not possible at that location). If the location of the drawing input is within a person, the processor (120) may provide a guide message indicating that drawing is not possible within the person. The guide may include at least one of text, an image, a video, or audio. After performing operation 906, the processor (120) may perform operation 915.

[0126] If the location of the drawing input is such that object creation is possible, in operation 907, the processor (120) may analyze the size or shape of the drawing input. The processor (120) may analyze whether the size of the drawing input exceeds (or is less than) a specified size of the entire size of the content, or whether the shape of the drawing input corresponds to a violent or sexual form.

[0127] In operation 909, the processor (120) may determine whether object creation is possible based on the analyzed result. The processor (120) may determine whether the size or shape of the analyzed drawing input allows object creation. For example, if the size of the drawing input is less than a first criterion (e.g., 1% of the total content size), the processor (120) may not allow object creation. Alternatively, if the size of the drawing input exceeds a second criterion (e.g., 30%, 50% of the total content size), the processor (120) may not allow object creation. If the shape of the drawing input corresponds to a specified shape (e.g., violent or sexual), the processor (120) may not allow object creation. If the size or shape of the drawing input allows object creation, the processor (120) may perform operation 911, and if the size or shape of the drawing input does not allow object creation, the processor (120) may perform operation 910.

[0128] If the size or shape of the drawing input does not allow object creation, in operation 910, the processor (120) may provide a second creation guide. For example, if the size of the drawing input is less than a first reference value, the processor (120) may provide a second creation guide (e.g., "Please draw it bigger"). If the size of the drawing input exceeds a second reference value, the processor (120) may provide a second creation guide (e.g., "Please draw it smaller"). If the shape of the analyzed drawing input corresponds to a designated shape (e.g., violent or sexual), the processor (120) may provide a second creation guide (e.g., "You cannot draw that picture"). After performing operation 910, the processor (120) may perform operation 915.

[0129] If the size or shape of the drawing input allows object creation, in operation 911, the processor (120) may analyze the input attribute value of the drawing input. The input attribute value (or input attribute information) may include at least one of a pen type, a pen color, a pen thickness, or a pen transparency. The pen type may include, for example, at least one of a basic pen, a fountain pen, a colored pencil, a marker, a highlighter, a mosaic pen, a blur pen, and a text word art pen. The color of the pen may include 50, 100, 256 colors, etc. The pen thickness may be divided into 0 to N levels (e.g., N is 100, 200, etc.), and the transparency of the pen may be divided into 0 to K levels (e.g., K is 50, 100). The user may select a color that matches (or is suitable for) his / her drawing and draw the drawing. The processor (120) can recognize the type of pen, the color of the pen, the thickness of the pen, or the transparency of the pen as parameters, and utilize the input information as a prompt. For example, for the same drawing input, when the input attribute values ​​are fountain pen (e.g., pen type) and red (e.g., pen color), and when the input attribute values ​​are colored pencil (e.g., pen type) and blue (e.g., pen color), the objects may be different, or even the same objects may be created slightly differently. The processor (120) can identify (or extract) the input attribute values ​​of the drawing input.

[0130] In operation 913, the processor (120) may include the analyzed information in a prompt. The processor (120) may define an object corresponding to the drawing input based on the analysis result. The prompt may include the location, size, shape input attribute values ​​of the drawing input, and the defined object. If the user has pre-selected (or set) the style of the object, the style of the selected object may be further included in the prompt. Alternatively, the prompt may further include information of the content including the drawing input, for example, content color information, a subject (or object) within the content, content, content including the drawing input, information of an object within the content corresponding to the location where the drawing was input, and location information of the subject. In the past, the user had to directly input the information to be included in the prompt as text, but in the present disclosure, various information necessary for object creation can be easily included in the prompt through content analysis and analysis of the drawing input.

[0131] At step 915, the processor (120) may detect a new drawing input based on user input. After receiving the first generation guide or the second generation guide, the user may draw a new drawing on the content. The user may change the input attribute value or draw the new drawing by changing the location, size, or shape of the drawing. When the new drawing input is detected, the processor (120) may display the detected drawing input on the content.

[0132] FIGS. 10A and 10B are diagrams illustrating an example of providing result content in an electronic device according to one embodiment.

[0133] Referring to FIG. 10A, a processor (e.g., the processor 120 of FIG. 1) of an electronic device (e.g., the electronic device 101 of FIG. 1) according to an embodiment may receive a drawing input (1011) from a user on content (1001) (e.g., original content). A first user interface (1010) illustrates an example of receiving a drawing input (1011) on content (1001). When a creation menu (1013) is selected in the first user interface (1010), the processor (120) may create multiple objects having different shapes. The processor (120) may provide a second user interface (1020) to a fifth user interface (1050) including objects having different shapes in response to the drawing input (1011). The processor (120) may display all of the first content (1021) to the fourth content (1051) on a single screen for the drawing input. For example, the processor may display all of the result images on the second user interface (1020), or may provide the first content (1021) to the fourth content (1051) together with the original content on the first user interface (1010). The order in which the second user interface (1020) to the fifth user interface (1050) are displayed may differ from the drawing. The third user interface (1030) may be displayed first, and then the fifth user interface (1050) may be displayed according to a specified user input. The order in which the user interfaces are displayed may not be limited to the drawing. For example, the processor (120) may sequentially display objects that are similar to (more reflective of) the drawing input. The fourth content (1051) may be generated by reflecting a higher proportion of the creation by the AI ​​engine than the drawing input reflection ratio.

[0134] For example, the second user interface (1020) may include first content (1021) including a first object (1023) corresponding to a drawing input (1011). When a specified user input (e.g., a swipe) is detected on the first content (1021), the processor (120) may provide a third user interface (1030). The third user interface (1030) may include second content (1031) including a second object (1033) corresponding to the drawing input (1011). The second object (1033) may include an object that is the same as the first object (1023) (e.g., ice cream), but has a slightly different shape. The first content (1021) and the second content (1033) may be different only in the objects they include, and the background content (e.g., the original content (1001)) excluding the objects may be the same.

[0135] When a specified user input is detected on the second content (1031), the processor (120) may provide a fourth user interface (1040). The fourth user interface (1040) may include a third content (1041) that includes a third object (1043) corresponding to the drawing input (1011). The third object (1043) may be the same as the first object (1023) or the second object (1033) (e.g., ice cream), but may include an object with a slightly different shape. The first content (1021) to the third content (1041) may differ only in the objects they include, and the background content excluding the objects (e.g., the original content (1001)) may be the same.

[0136] When a specified user input is detected on the third content (1041), the processor (120) may provide a fifth user interface (1050). The fifth user interface (1050) may include a fourth content (1051) that includes a fourth object (1053) corresponding to the drawing input (1011). The fourth object (1053) may be the same as the first object (1023) to the third object (1043) (e.g., ice cream), but may include an object with a slightly different shape. The first content (1021) to the fourth content (1051) may differ only in the objects they include, and the background content excluding the objects (e.g., original content (1001)) may be the same.

[0137] When a second user input (e.g., long press) specified on the first content (1021) to the fourth content (1051) is detected, the processor (120) may provide a sixth user interface (1060). The sixth user interface (1060) may include content (1001) including a drawing input (1011) similar to the first user interface (1010). When a swipe gesture input in the opposite direction to the specified user input is detected on the first content (1021) to the fourth content (1051), the processor (120) may provide the first user interface (1010).

[0138] FIGS. 11A and 11B are diagrams illustrating an example of providing result content in an electronic device according to one embodiment.

[0139] Referring to FIG. 11A, a processor (e.g., a processor (120) of FIG. 1) of an electronic device (e.g., an electronic device (101) of FIG. 1) according to an embodiment may detect a drawing input (1111) on content (1101). The processor (120) may analyze a first original content (1110) including the content (1101) and the drawing input (1111) to generate a first result content (1130). The processor (120) may analyze the content (1101) to identify that it is underwater and that the subject (1113) is a person. The processor (120) may identify the location of the subject (1113) and identify that the drawing input (1111) is input to the left of the subject (1113). The drawing input (1111) may be a fish drawn in blue using a basic pen or colored pencil (e.g., a type of pen). The processor (120) may define the object as a 'dolphin' based on the position, size, shape, and input attribute values ​​of the drawing input (1111).

[0140] The processor (120) may include at least one of the following in the prompt: underwater, a subject (1113) (e.g., a person), a dolphin next to the subject, a location, a size, a shape, or an input attribute value of a drawing input (1111). The prompt may include content, content including a drawing input, information on an object (e.g., the subject (1113)) within the content corresponding to a location where a drawing is input, and location information of the subject (1113) within the content. The prompt may include a request to regenerate the content (1101) (e.g., the subject) by AI in conjunction with a defined object, and the AI ​​engine (e.g., the AI ​​model (270)) may provide content as a result of regenerating a subject that is surprised or smiling at an object (a dolphin) generated for the drawing input in response to the request. The processor (120) may receive a first result content (1130) in which an object (1131) corresponding to a drawing input (1111) is generated from an AI engine (e.g., a generative AI model (270) of FIG. 2) included in the electronic device (101) or an intelligent server (e.g., a server (108) of FIG. 1). The first result content (1130) may include (or add, generate) an object (1131) next to a subject (1113) in the original content (1101).

[0141] Referring to FIG. 11B, the processor (120) can detect a drawing input (1153) on content (1151) (e.g., second original content). The first user interface (1150) can include content (1151), a drawing input (1153), a drawing guide (1155), and a creation menu (1157). The processor (120) can analyze the content (1151) (e.g., original content) to identify a subject (1152) (e.g., a human upper body) sitting in front of a desk and a bookshelf behind the subject (1152). The processor (120) can identify the location of the subject (1152) and identify that the drawing input (1153) is input in front of the subject (1152). The drawing input (1153) can be a cat drawn in white with a basic pen (e.g., a type of pen). The processor (120) can define an object as a 'cat' based on the position, size, shape, and input attribute values ​​of the drawing input (1153).

[0142] When the creation menu (1157) of the first user interface (1150) is selected, the processor (120) may include at least one of the following in the prompt: a desk, a subject (1152) (e.g., a person), a cat in front of the subject, a position, a size, a shape, or an input attribute value of a drawing input (1153). The prompt may include a request to recreate the subject by AI in conjunction with the defined object, and the AI ​​engine (e.g., the AI ​​model (270)) may provide a result image of a subject being recreated by a person stroking the object (cat) created for the drawing input by the request.

[0143] The processor (120) may provide second result content (1171) in which an object (1173) corresponding to a drawing input (1153) is generated through a generative AI model (270) included in the electronic device (101) or a server (108). The second result content (1171) may include (or add, generate) an object (1173) in front of a subject (1152) in the original content (1151). The second user interface (1170) may display the second result content (1171) including the object (1173) in a large size as a representative content, and may include a first thumbnail image (1175) and a second thumbnail image (1177). The first thumbnail image (1175) may be a thumbnail of the second result content (1171). The second thumbnail image (1177) may be a thumbnail of the third result content. The second result content (1171) and the third result content may differ only in the generated objects, but the background content (e.g., the original content (1151)) may be identical.

[0144] FIGS. 12A and 12B are drawings illustrating an example of creating and providing an object corresponding to a drawing input in an electronic device according to one embodiment.

[0145] Referring to FIG. 12A, a processor (e.g., a processor (120) of FIG. 1) of an electronic device (e.g., an electronic device (101) of FIG. 1) according to an embodiment may display a first user interface (1210) including a first original content (1201) on a display (e.g., a display module (160) of FIG. 1). The first original content (1201) may be content with a night background including a Christmas tree. The processor (120) may receive a drawing input from a user while displaying the first user interface (1210). The second user interface (1220) may be a first drawing input (1221) displayed on the first original content (1201). When the create menu is selected in the second user interface (1220), the processor (120) can analyze the first original content (1201) and the first drawing input (1221) to provide a third user interface (1230) including the first result content (1233).

[0146] The processor (120) can identify the first drawing input (1221) as an object composed of points or lines and not in the form of a closed curve. Since a thin line is not a closed curve, it cannot be seen as an object, and thus it may be difficult to determine what kind of drawing it is based on the drawing shape. The processor (120) can determine that the drawn line is light based on the pen type (e.g., Blur pen) of the first drawing input (1221) and the fact that the first original content (1201) is a night background, and can generate a first object (1231) that expresses the color used in lighting as light. The processor (120) can include in the prompt a drawing input drawn in white with a blur pen on a night background as light.

[0147] Referring to FIG. 12B, the processor (120) may display a fourth user interface (1250) including second original content (1251) on the display module (160). The original content (1251) may be content with a daytime background in which the sun is reflected on the sea and sparkles. The processor (120) may receive a drawing input from the user while displaying the fourth user interface (1250). The fifth user interface (1260) may be a second drawing input (1261) displayed on the second original content (1251). When a create menu is selected in the fifth user interface (1260), the processor (120) may analyze the second original content (1251) and the second drawing input (1261) to provide a sixth user interface (1270) including second result content (1273). The processor (120) can identify the second drawing input (1261) as an object composed of points or lines rather than a closed curve. Based on the pen type (e.g., highlighter) of the second drawing input (1261) and the fact that the second original content (1251) is a sea with the sun reflected on it in the daytime background, the processor (120) can determine that the drawn point is light and can generate a second object (1271) that expresses the color used in lighting as light. The processor (120) can include in the prompt a drawing input drawn in white with a highlighter on a daytime background as light.

[0148] FIG. 13 is a diagram illustrating an example of generating an object corresponding to a drawing input in an electronic device according to one embodiment.

[0149] Referring to FIG. 13, a processor (e.g., a processor (120) of FIG. 1) of an electronic device (e.g., an electronic device (101) of FIG. 1) according to an embodiment may receive a drawing input (1311) from a user to content (1301). A first user interface (1310) represents an example of displaying a drawing input (1311) on content (1301). When object creation is requested, the processor (120) may analyze the content (1301) and the drawing input (1311) to provide a second user interface (1320). The processor (120) may analyze the drawing input (1311) to identify it as text, and may analyze an input attribute value (e.g., fountain pen, Cali pen) of the drawing input (1311). The second user interface (1320) may include result content (1323) containing an object (1321) corresponding to the drawing input (1311). The processor (120) may determine that the user's object creation request is to convert the text of the drawing input (1311) into a pretty font. The processor (120) may analyze the content (1301) and the drawing input (1311) to include information such as "great!", "pretty font conversion," and "calligraphy" in the prompt.

[0150] FIGS. 14A and 14B are drawings illustrating an example of creating and providing an object corresponding to a drawing input in an electronic device according to one embodiment.

[0151] Referring to FIG. 14A, a processor (e.g., a processor (120) of FIG. 1) of an electronic device (e.g., an electronic device (101) of FIG. 1) according to an embodiment may receive a first drawing input (1411) on a first original content (1401). The first user interface (1410) may display the first drawing input (1411) on the first original content (1401). When a create menu (1413) is selected, the processor (120) may analyze the first original content (1401) and the first drawing input (1411) to provide a second user interface (1430). The processor (120) may identify that the first original content (1401) is a portrait including a human face, and may analyze the first drawing input (1411) in the form of glasses at the eye location of the human face. The processor (120) can generate a prompt by analyzing the location of the subject within the first original content (1401), the location, size, shape, and input attribute values ​​of the first drawing input (1411) within the subject. The color, transparency, and thickness used when inputting the drawing can be used as a prompt. For example, the basic prompt may include information such as female, real photo, and additional information may be added as a prompt for additional information such as blue glasses on the left and red glasses on the right, sunglasses with translucent inside, and thin thickness. The AI ​​engine may reflect the pen characteristic information to generate a result (e.g., an object) corresponding to the first drawing input (1411). The second user interface (1430) may include first result content (1431) including a first object (1433) corresponding to the first drawing input (1411).

[0152] Referring to FIG. 14B, the processor (120) may receive a second drawing input (1451) on the second original content (1453). The third user interface (1450) may display the second drawing input (1451) on the second original content (1453). When object creation is requested, the processor (120) may analyze the second original content (1453) and the second drawing input (1451) to provide a fourth user interface (1470). The processor (120) may identify a tree from the second original content (1453) and analyze the second drawing input (1451) in which a round circle with a transparency of 80 is drawn between the trees in pink (or light pink, dark pink). The processor (120) may define the object as a 'flower' based on the position, size, shape, and input attribute values ​​of the drawing input (1451). The processor (120) may generate a prompt by analyzing the location of a subject (e.g., a tree) within the second original content (1453), the location, size, shape, and input attribute values ​​of a second drawing input (1451) within the subject. The fourth user interface (1470) may include second result content (1473) including a second object (1471) corresponding to the second drawing input (1451).

[0153] FIGS. 15A and 15B are drawings illustrating an example of generating and providing an object corresponding to a drawing input in an electronic device according to one embodiment.

[0154] Referring to FIG. 15A, a processor (e.g., the processor (120) of FIG. 1) of an electronic device (e.g., the electronic device (101) of FIG. 1) according to an embodiment may display a first user interface (1510) in which a first drawing input (1511) is displayed on original content (1501) on a display (e.g., the display module (160) of FIG. 1). When a creation menu (1513) is selected in the first user interface (1510), the processor (120) may analyze the original content (1501) and the first drawing input (1511) to provide a second user interface (1520). The outline of the first drawing input (1511) may be defined as a basic color in white or black. When the outline of the first drawing input (1511) is drawn in white or black, the color of the outline does not have a significant effect on object creation and may be mainly used for viewing the shape of the object. If the first drawing input (1511) is drawn in one color, the internal color of the drawing input (1511) can also be filled with the same color. If the first drawing input (1511) is drawn in blue for the upper body of the subject included in the content, the second user interface (1520) can include the first result content (1521) including the first object (1523) (e.g., blue top, blue shirt) corresponding to the first drawing input (1511).

[0155] The processor (120) can display a third user interface (1530) on which a second drawing input (1533) is displayed in content (1531) on the display module (160). When a creation menu (1535) is selected in the third user interface (1530), the processor (120) can analyze the content (1531) and the second drawing input (1533) to provide a fourth user interface (1540). The outline of the second drawing input (1533) can be colored yellow, and the interior of the second drawing input (1533) can be colored red. The processor (120) can generate a second object by distinguishing the outline color and the interior color of the second drawing input (1533). The fourth user interface (1540) may include a second result content (1541) including a second object (1543) corresponding to the second drawing input (1533), when the upper body of the subject included in the content is outlined in yellow and the interior in red.

[0156] Referring to FIG. 15B, the processor (120) may display a fifth user interface (1550) on which a third drawing input (1553) is displayed on a second original content (1551) on the display module (160). When a creation menu (1555) is selected on the fifth user interface (1550), the processor (120) may analyze the second original content (1551) and the third drawing input (1553) to provide a sixth user interface (1560) or a seventh user interface (1570). The processor (120) may express the third drawing input (1553) as an outline with a thick line, and, if expressed without internal division, may generate a shape with the color of the line itself. The sixth user interface (1560) may include a third result content (1561) including a third object (1563) corresponding to the third drawing input (1553). The seventh user interface (1570) may include fourth result content (1571) including a fourth object (1573) corresponding to the third drawing input (1553). The third object (1563) and the fourth object (1573) may be the same rainbow, but may have slightly different shapes.

[0157] FIGS. 16A and 16B are drawings illustrating an example of generating and providing an object corresponding to a drawing input in an electronic device according to one embodiment.

[0158] Referring to FIG. 16A, a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to an embodiment may detect a drawing input (1611) in content (1601). A first user interface (1610) may display a drawing input (1611) in the content (1601). When a create menu (1613) is selected in the first user interface (1610), the processor (120) may provide a second user interface (1630). The processor (120) may analyze the content (1601) and the drawing input (1611), include the analysis result in a prompt, and transmit the prompt to an intelligent server (e.g., server (108) of FIG. 1). The processor (120) can receive result content (1631) including an object (1633) corresponding to a drawing input (1611) from the server (108) and provide it to a second user interface (1630).

[0159] Referring to FIG. 16B, as shown in the first drawing symbol (1650), the processor (120) can extract an area corresponding to the drawing input (1611) from the content (1631) as a masking area (1651). The processor (120) can transmit the masking area (1651) to the server (108). Alternatively, the processor (120) can analyze the masking area (1651), include the analysis result in a prompt, and transmit the prompt to the server (108). As shown in the second drawing symbol (1670), the processor (120) can receive an object (1633) corresponding to the drawing input (1611) from the server (108). The processor (120) can add the object (1633) corresponding to the drawing input (1611) to the original content to generate the result content (1691). The generated result content can be provided as a third user interface (1690). The third user interface (1690) can include result content (1691) including an object (1633) corresponding to the drawing input (1611).

[0160] FIGS. 17A to 17C are drawings illustrating an example of providing a sketch conversion function in an electronic device according to one embodiment.

[0161] Referring to FIG. 17A, a processor (e.g., the processor (120) of FIG. 1) of an electronic device (e.g., the electronic device (101) of FIG. 1) according to an embodiment may display a first user interface (1710) on a display (e.g., the display module (160) of FIG. 1). The first user interface (1710) may receive an input for selecting a visual object (1711) supporting a generative AI function at the bottom of the screen while content (1701) including a subject (e.g., a snowman) is displayed. When the visual object (1711) supporting the generative AI function is selected, the display may switch to a second user interface (1720). The second user interface (1720) may display a visual object (1721) supporting sketch transformation on the content (1701). When a visual object (1721) supporting sketch transformation is displayed, a third user interface (1730) or a fourth user interface (1740) capable of receiving drawing input from a user may be switched. The third user interface (1730) may include a visual object (1731) for setting pen input property values ​​on the content (1701). When the visual object (1731) for setting pen input property values ​​is selected, a pop-up window (1732) for setting pen thickness and color may be displayed.

[0162] Referring to FIG. 17B, the fourth user interface (1740) may receive a green drawing input (1741) and a purple drawing input (1742) on the content (1701). The fourth user interface (1740) may additionally include an input window in which a user can input additional information to be requested to the AI ​​engine for object creation in the form of text. For example, text to be input as additional information may be “I drew a scarf and a button-up shirt,” “Draw it with a Frozen background,” or “Draw it so that it feels warm.” When a selection input for the create button (1743) is received, the fifth user interface (1750) may be switched. The fifth user interface (1750) may display a guide message (1751) guiding that an object corresponding to the drawing input is being created. The guide message may include text and visual objects. For example, the text may include at least one of the following: "Adding a special touch with artistic flair...", "Reborn as a great piece...", or "Breathing warm air into the sketch...". The text and visual objects may be provided with animated effects.

[0163] Alternatively, before displaying the fifth user interface (1750), a screen may be further provided that includes an interface for defining an object based on the drawing input analysis result, requesting confirmation (Yes, No) as to whether the defined object is the object the user is trying to draw, and receiving confirmation from the user regarding the request. If the defined object is not the object the user is trying to draw, a text input prompt for receiving additional information from the user may be additionally provided. The sixth user interface (1760) may include a first result content (1761) that includes only a first object (e.g., three purple buttons) (1762) corresponding to the purple drawing input. When a save as file button (1763) is selected in the sixth user interface (1760), the first result content (1761) may be saved.

[0164] Referring to FIG. 17c, when a designated gesture input (swipe gesture) is detected in the sixth user interface (1760), a switch may be made to the seventh user interface (1770). The seventh user interface (1770) may include a second result content (1771) including a second object (1772) corresponding to a green drawing input and a third object (1773) corresponding to a purple drawing input. The eighth user interface (1780) may include a third result content (1781) including a fourth object (1782) corresponding to a green drawing input and a fifth object (1783) corresponding to a purple drawing input. The ninth user interface (1790) may include a fourth result content (1791) including a sixth object (1792) corresponding to a green drawing input and a seventh object (1793) corresponding to a purple drawing input.

[0165] The sixth user interface (1760) to the ninth user interface (1790) may display at least one result image (1761 to 1791) including an object generated in response to a drawing input. According to one embodiment, when the processor (120) receives a user input (e.g., a long press) for selecting an object generated in response to a drawing input from the result image, the processor (120) provides an interface (e.g., an indicator, a visual cue, a menu) that can perform a function (e.g., resizing, changing a position, saving as a sticker, sharing) related to the generated object, and can resize, change a position, save as a sticker, and / or share the object according to the user input.

[0166] FIG. 18 is a diagram illustrating an example of storing result content in an electronic device according to one embodiment.

[0167] Referring to FIG. 18, a processor (e.g., the processor (120) of FIG. 1) of an electronic device (e.g., the electronic device (101) of FIG. 1) according to an embodiment may store at least one tag (keyword) related to a generated object as metadata of the image in a memory (e.g., the memory (130) of FIG. 1) when storing result content (e.g., an image, a video). The processor (120) may search for result content based on at least one keyword (tag) stored in the metadata. According to an embodiment, the generated object may be classified by category and stored as separate content.

[0168] Upon receiving a swipe gesture input in the opposite direction to the designated gesture in the sixth user interface (1760) to the ninth user interface (1790), the processor (120) may switch to a user interface (1810) displaying original content (1701) including a drawing input (1741, 1742). When a drawing edit button (1811) is selected in the user interface (1810), editing of the drawing input may be possible.

[0169] An operating method of an electronic device (101) according to an embodiment of the present disclosure may include an operation of detecting a drawing input on content displayed on a display (160) of the electronic device, an operation of displaying the detected drawing input on the content, an operation of receiving a request for generating an object corresponding to the detected drawing input, an operation of analyzing the content and the detected drawing input in response to the request, an operation of generating a prompt based on at least one of information of the content, a location of the drawing input, or an input attribute value of the drawing input as a result of the analysis, and an operation of generating an object corresponding to the detected drawing input on the content based on the generated prompt.

[0170] The above analyzing operation may include an operation of defining an object corresponding to the drawing input based on at least one of the information of the content, the location of the drawing input, or the input attribute value of the drawing input as the analyzed result.

[0171] The above-defining operation may include an operation of further analyzing the size or shape of the detected drawing input, and an operation of defining an object corresponding to the drawing input based on the size or shape of the further analyzed drawing input.

[0172] The input attribute value of the above drawing input may include at least one of the pen type, color, thickness, or transparency.

[0173] The method may further include an action of including a style of a specified object in response to the drawing input in the prompt, or an action of including a style of an object selected by the user in the prompt when a change in the style of the specified object is requested from the user.

[0174] The method may further include an operation of analyzing a location of the detected drawing input on the content in response to the request, an operation of determining whether the location of the detected drawing input is a location where an object can be created on the content, and an operation of providing a guide for object creation if the location of the detected drawing input is not a location where an object can be created on the content.

[0175] The method may further include an operation of analyzing the size of the detected drawing input in response to the request, an operation of determining whether the size of the detected drawing input exceeds a specified size, and an operation of providing a guide for object creation when the size of the detected drawing input exceeds the specified size.

[0176] The method may further include an action of setting an area including the detected drawing input in the content as a masking area, an action of transmitting the generated prompt and the masking area to an intelligent server, an action of receiving object information including an object generated in response to the detected drawing input from the intelligent server, and an action of generating and providing content including the generated object in the content based on the received object information.

[0177] The method may include an operation of creating a first object corresponding to the detected drawing input, an operation of creating a second object different from the first object corresponding to the detected drawing input, and an operation of displaying a second content including the second object when a specified user input is detected while displaying a first content including the first object.

[0178] The various embodiments of the present invention disclosed in this specification and drawings are merely specific examples presented to facilitate easy explanation of the technical content of the present invention and aid understanding thereof, and are not intended to limit the scope of the present invention. Therefore, the scope of the present invention should be interpreted to include all modifications or variations derived based on the technical concept of the present invention, in addition to the embodiments disclosed herein.

Claims

1. In an electronic device (101), Communication module (190); Display (160), Memory (130) for storing instructions; and A processor (120) is included, and the instructions, when executed by the processor, cause the electronic device to: Detecting drawing input on the content displayed on the above display, Display the above detected drawing input in the above content, Receive a request to create an object corresponding to the above detected drawing input, In response to the above request, analyze the above content and the above detected drawing input, Generate a prompt based on at least one of the information of the content, the location of the drawing input, or the input attribute value of the drawing input based on the analyzed result, An electronic device that generates an object corresponding to the detected drawing input on the content based on the generated prompt.

2. In the first paragraph, when the instructions are executed by the processor, the electronic device, An electronic device that defines an object corresponding to the drawing input based on at least one of the information of the content, the location of the drawing input, or the input attribute value of the drawing input based on the analyzed result.

3. In the second paragraph, when the instructions are executed by the processor, the electronic device, Further analyze the size or shape of the above detected drawing input, An electronic device that defines an object corresponding to the drawing input based on the size or shape of the additionally analyzed drawing input.

4. In paragraph 1, The input attribute values ​​of the above drawing input are: An electronic device comprising at least one of a pen type, color, thickness, or transparency.

5. In the first paragraph, when the instructions are executed by the processor, the electronic device, Include the style of the specified object in the above prompt in response to the above drawing input, or An electronic device that includes the style of an object selected by the user in the prompt when a request for a change in the style of the object specified above is received from the user.

6. In the first paragraph, when the instructions are executed by the processor, the electronic device, In response to the above request, analyze the location of the detected drawing input on the above content, Determine whether the location of the above-detected drawing input is a location where object creation is possible on the above content, An electronic device that provides a guide for object creation when the location of the detected drawing input is not a location where object creation is possible on the content.

7. In the first paragraph, when the instructions are executed by the processor, the electronic device, In response to the above request, analyze the size of the detected drawing input, Determine whether the size of the above-detected drawing input exceeds the specified size, An electronic device that provides a guide for object creation when the size of the detected drawing input exceeds a specified size.

8. In the first paragraph, when the instructions are executed by the processor, the electronic device, In the above content, the area containing the detected drawing input is set as a masking area, Transmit the generated prompt and the masking area to the intelligent server, Receive object information including an object generated in response to the detected drawing input from the intelligent server, An electronic device that generates and provides content including the generated object based on the received object information.

9. In the first paragraph, when the instructions are executed by the processor, the electronic device, Transmit the analysis information of the above-detected drawing input and the above-generated prompt to the intelligent server, Receive object information including an object generated in response to the detected drawing input from the intelligent server, An electronic device that generates and provides content including the generated object based on the received object information.

10. In the first paragraph, when the instructions are executed by the processor, the electronic device, Create a first object corresponding to the above detected drawing input, Generate a second object different from the first object corresponding to the detected drawing input, An electronic device that, while displaying first content including the first object, displays second content including the second object when a specified user input is detected.

11. In the 10th paragraph, when the instructions are executed by the processor, the electronic device, An electronic device that displays the first content and the second content as thumbnail images while displaying the first content including the first object.

12. In the operating method of the electronic device (101), An action of detecting a drawing input over content displayed on a display (160) of the electronic device; An action of displaying the detected drawing input on the content; An action of receiving a request to create an object corresponding to the above detected drawing input; An action of analyzing the content and the detected drawing input in response to the request; An operation of generating a prompt based on at least one of the information of the content, the location of the drawing input, or the input attribute value of the drawing input based on the analyzed result; and A method comprising an action of generating an object corresponding to the detected drawing input on the content based on the generated prompt.

13. In paragraph 12, the analyzing operation is: A method comprising an action of defining an object corresponding to the drawing input based on at least one of the information of the content, the location of the drawing input, or the input attribute value of the drawing input as a result of the analysis.

14. In the 13th paragraph, the defined action is: An operation of further analyzing the size or shape of the above-detected drawing input; and A method comprising an action of defining an object corresponding to the drawing input based on the size or shape of the drawing input further analyzed.

15. In paragraph 12, An action that includes the style of the specified object in the above prompt in response to the above drawing input, or A method further comprising an action of including the style of an object selected by the user in the prompt when a request for a change in the style of the object specified above is received from the user.

Images