Electronic device and image editing method for electronic device

Abstract

This electronic device may comprise: a memory for storing instructions; a display; and a processor. The instructions, when executed by the processor, may cause the electronic device to: determine a keyword corresponding to a first user input on the basis of reception of the first user input; obtain at least one image through crawling on the basis of the determined keyword; determine at least one graphical object corresponding to the at least one image; receive a second user input related to the at least one graphical object; and generate an artificial intelligence image using a machine learning model on the basis of any one of the second user input or information about the at least one graphical object.

Description

Electronic devices and methods for editing images on electronic devices

[0001] This article is about electronic devices and how to edit images on electronic devices.

[0002] Artificial intelligence (AI) technology is rapidly developing. In particular, generative AI is rapidly growing alongside this advancement. Generative AI refers to AI technology that utilizes machine learning and deep learning to generate similar content based on existing content, such as text, audio, and / or images. For example, generative AI can generate new content based on previously learned data, and can also be applied to actions such as improving image quality or editing videos.

[0003] AI content (e.g., images, videos) generated using generative AI may produce results partially, rather than entirely, generated by AI. For example, at least partially generated AI content may contain both real and fake data.

[0004] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above-described matters constitute prior art related to the present disclosure.

[0005] When a machine learning model generates a new object image from an image containing an object or a blank screen (e.g., a canvas), creating the image from scratch can be relatively time-consuming. Furthermore, the image generated by the machine learning model may not match the user's intent.

[0006] Furthermore, when generating images solely based on generative models without crawling, the dependence on trained data makes it difficult for users to obtain the desired images. The electronic device described in this document can provide guidance to help users create the desired image when creating a new object image.

[0007] The problem to be solved in this disclosure is not limited to the problem mentioned above, and may be expanded in various ways without departing from the spirit and scope of this disclosure.

[0008] According to one embodiment, an electronic device may include a memory storing instructions, a display, and a processor. The instructions, when executed by the processor, may cause the electronic device to determine a keyword corresponding to the first user input based on receiving a first user input, acquire at least one image through crawling based on the determined keyword, determine at least one graphical object corresponding to the at least one image, receive a second user input related to the at least one graphical object, and generate an artificial intelligence image using a machine learning model based on either the second user input or information about the at least one graphical object.

[0009] According to one embodiment, the operating method may include an operation of determining a keyword corresponding to the first user input based on receiving a first user input, an operation of obtaining at least one image through crawling based on the determined keyword, an operation of determining at least one graphical object corresponding to the at least one image, an operation of receiving a second user input related to the at least one graphical object, and an operation of generating an artificial intelligence image using a machine learning model based on either the second user input or information about the at least one graphical object.

[0010] An electronic device according to this document can generate an image that meets the user's intention by using edge-based drawing and a generative model when drawing a new object image or adding an object image to an existing image.

[0011] An electronic device according to this document can provide a guide function when a user wants to create a new object image, thereby helping the user to easily and quickly create an intended image.

[0012] An electronic device according to this document can provide an environment in which a user can intervene by detecting an edge image based on a crawling image, thereby helping the user obtain a desired image.

[0013] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly understood by a person having ordinary skill in the art to which the present disclosure belongs from the description below.

[0014] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0015] FIG. 2 is a block diagram of an electronic device according to various embodiments.

[0016] Figure 3 classifies the configuration of electronic devices according to various embodiments according to function.

[0017] FIGS. 4A and 4B illustrate one embodiment of a method for editing an image of an electronic device.

[0018] FIGS. 5A and 5B illustrate another embodiment of an image editing method of an electronic device.

[0019] FIG. 6 is a flowchart illustrating an image editing method of an electronic device according to various embodiments.

[0020] FIG. 7 is a flowchart illustrating an image editing method of an electronic device according to various embodiments.

[0021] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with the electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (104) or the server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0022] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0023] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0024] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0025] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0026] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0027] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0028] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0029] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0030] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0031] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0032] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0033] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0034] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0035] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0036] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0037] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0038] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0039] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0040] According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0041] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0042] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0043] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0044] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0045] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0046] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0047] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0048] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0049] FIG. 2 is a block diagram of an electronic device according to various embodiments.

[0050] Referring to FIG. 2, the electronic device (200) may include a processor (210), a memory (220), a display (230), a communication circuit (240), and a sensor (250), and may implement embodiments of the present document even if at least some of the illustrated configurations are omitted and / or replaced. The electronic device (200) may further include at least some of the configurations and / or functions of the electronic device (101) of FIG. 1.

[0051] Some of the components (e.g., processor (210), memory (220), communication circuit (240)) of the electronic device (200) shown in FIG. 2 and / or other components not shown may be positioned inside a housing (not shown) of the electronic device (200), and some components (e.g., display (230)) may have at least a portion thereof exposed to the outside of the housing.

[0052] According to one embodiment, the display (230) can display content provided from the processor (210).

[0053] A display (230) may be placed on the front of the electronic device (200), and the display (230) may include a rollable display. When the electronic device (200) is fully unfolded, an application execution screen may be displayed on the entire electronic device (200). However, the shape of the display (230) is not limited thereto.

[0054] According to one embodiment, the display (230) may be configured as a touch screen that detects touch and / or proximity touch (or hovering) input using a part of the user's body (e.g., a finger) or an input device (e.g., a stylus pen).

[0055] According to one embodiment, the electronic device (200) may include at least one sensor (250) that detects contact or proximity. For example, the electronic device (200) may include a type of near-field sensor that detects proximity or contact of an object, such as a near field communication (NFC), radio frequency identification (RFID), Bluetooth (or Bluetooth low energy), or touch sensor.

[0056] According to one embodiment, at least one sensor (250) may be positioned in the bezel area on the top, bottom, left, and right sides of the display (230) and / or the corner area of ​​the housing. Accordingly, when selecting an area to display main content or a background image, the user may be enabled to make a touch or proximity input to a sensor (250) adjacent to the intended area. In various embodiments, the electronic device (200) may position one to five sensors (250) around the display (230).

[0057] According to one embodiment, the communication circuit (240) may provide a wired or wireless communication interface with an external device. For example, the communication circuit (240) may include a high definition multimedia interface (HDMI) or a universal serial bus (USB) interface as examples of a wired communication interface (e.g., interface (177) of FIG. 1). In addition, the communication circuit (240) may include a wireless communication circuit (e.g., wireless communication circuit (192) of FIG. 1) that supports a short-range communication circuit with an external device. The wireless communication circuit may support a short-range wireless communication method (e.g., Wi-Fi, Bluetooth, BLE (Bluetooth low energy)) and may include independent hardware and / or software configurations for supporting each wireless communication method.

[0058] According to one embodiment, the memory (220) may include volatile memory and non-volatile memory, and may temporarily or permanently store data. The memory (220) may include at least some of the configuration and / or functions of the memory (130) of FIG. 1, and may store the program (140) of FIG. 1.

[0059] The memory (220) can store instructions that can be performed by the processor (210). Such instructions can include control commands such as arithmetic and logical operations, data movement, and input / output that can be recognized by the processor (210).

[0060] According to one embodiment, the processor (210) is a configuration capable of performing calculations or data processing related to control and / or communication of each component of the electronic device (200), and may be composed of one or more processors (210). The processor (210) may include at least some of the configurations and / or functions of the processor (120) of FIG. 1. The processor (210) may be operatively, functionally, and / or electrically connected to each component of the electronic device (200), including the display (230), the memory (220), and the sensor (250).

[0061] According to one embodiment, there will be no limitation to the computational and data processing functions that the processor (210) can implement on the electronic device (200), but below, a function of providing an image that matches the user's intention using edge-based drawing and a machine learning model will be described.

[0062] Figure 3 classifies the configuration of electronic devices according to various embodiments according to function.

[0063] In FIG. 3, the electronic device (200) may include components and machine learning models that perform various functions. The components illustrated in FIG. 3 may be physically separated and arranged on the electronic device (200) or may be included in a single processor (e.g., the processor (210) of FIG. 2).

[0064] According to one embodiment, the electronic device (200) can determine an object word input by a user using the object input unit (302) to be generated under the control of the processor (210). According to one embodiment, the electronic device (200) can receive a first user input and determine a keyword corresponding to the user input. The first user input can include either a text input or a voice input. The keyword can include, for example, a word such as vase or cup. In FIGS. 4A and 4B , the keyword is assumed to be vase, but this is only an example and the keyword is not limited thereto. In FIGS. 5A and 5B , the keyword is assumed to be cup, but this is only an example and the keyword is not limited thereto.

[0065] According to one embodiment, the electronic device (200) may provide an interface that allows a user to input an object they wish to add in natural language. The electronic device (200) may provide an interface for object input based on the selection of an object creation option. The electronic device (200) may display a user interface (UI) in the form of a chat input window, allowing the user to input the object word they wish to create in natural language.

[0066] According to one embodiment, the electronic device (200) can crawl images corresponding to keywords using the crawling unit (304) under the control of the processor (210). Crawling may refer to a process of regularly visiting web pages and extracting information. Alternatively, the electronic device (200) may receive images corresponding to keywords from an external server (e.g., server (108) of FIG. 1). The electronic device (200) can collect a large amount of information existing on the web through crawling. The electronic device (200) can collect images corresponding to keywords using the crawling unit (304).

[0067] According to one embodiment, the electronic device (200) may perform edge detection on collected images using the edge detection unit (306) under the control of the processor (210). Alternatively, the electronic device (200) may collect images corresponding to a keyword using the crawling unit (304) and perform edge detection on all collected images using the edge detection unit (306). The electronic device (200) may collect at least one image corresponding to the keyword and display all collected images to request user input. The electronic device (200) may determine one of the images based on the user input and perform edge detection on only the selected image. The electronic device (200) may perform edge detection on only the selected image to obtain a first image.

[0068] Alternatively, the electronic device (200) may perform edge detection on all collected images. The electronic device (200) may display all images on which edge detection has been performed and display an interface requesting user input. Based on the user input, the electronic device (200) may select one of the multiple edge images and determine it to be the first image.

[0069] An edge image according to various embodiments of the present invention may refer to an image that emphasizes the boundary line or the part where a change occurs in an object in an image. Edge detection may refer to an operation of creating an edge image from an actual image. Specifically, edge detection may refer to an operation of finding a part in an image where at least one of color, brightness, and texture changes rapidly. Edge detection may be utilized in various fields including object recognition, feature extraction, and image segmentation. The electronic device (200) may use an edge image to significantly reduce and express information of an original image.

[0070] According to one embodiment, the electronic device (200) may perform online image data crawling based on an input keyword (e.g., "vase"). Once crawling of images corresponding to the input word is completed, the electronic device (200) may perform edge detection on the crawled images. The electronic device (200) may set a threshold to control an edge detection rate during the edge detection process. For example, if the electronic device (200) attempts to provide an image in which relatively more edge information is detected, the image generation time may relatively increase. Furthermore, if relatively more edge information is detected, the area in which a user can intervene on the edge image may relatively decrease. The more edge information is detected, the more detailed the image is, and the less edge information is detected, the more abstract the image is. The electronic device (200) may set the threshold relatively high to control that relatively less edge information is detected. The threshold may be expressed as a percentage (%) or as an absolute value between 0 and 1. The electronic device (200) can control the degree to which edge information is detected by adjusting the size of the threshold value.

[0071] According to one embodiment, the electronic device (200) can be controlled to detect a minimum number of edges based on parameters related to edge detection. A higher value of a parameter related to edge detection (e.g., threshold) can lead to a more abstract image, resulting in relatively less edge information being detected. A lower value of a parameter related to edge detection (e.g., threshold) can lead to a relatively closer image to the original, resulting in relatively more edge information being detected.

[0072] According to one embodiment, the electronic device (200) can receive a user's drawing input using the drawing input receiving unit (308) under the control of the processor (210). The drawing input may mean an image input by the user.

[0073] According to one embodiment, the electronic device (200) may display an interface to enable a user to add a second user input on a first image, and may add the second user input to an area of ​​the first image or an area surrounding the first image based on the user input on the interface. The second user input may include a drawing input.

[0074] Alternatively, the electronic device (200) may receive a second user input in the first image area or the area surrounding the first image without displaying an interface. The first image may refer to an edge-detected image selected by the user. Alternatively, the first image may refer to an image of an area in which an original image and an edge-detected image are overlaid. The electronic device (200) may add a user input by displaying an interface on the first image, or may receive a user input in the first image area or the area surrounding the first image without displaying the interface.

[0075] According to one embodiment, the electronic device (200) may provide a user interface for selecting a plurality of detected edge images. The electronic device (200) may receive a user input for one of the plurality of edge images through the interface. The electronic device (200) may receive a user input for an edge image that most closely matches the shape of an object that the user wishes to create through the interface. The electronic device (200) may display a user interface (drawing UI) for drawing based on an edge image (e.g., a first image) determined based on the user input. The electronic device (200) may recognize an input for at least one of an input using a user's finger, an input using an electronic pen, or an input using a mouse pointer. The electronic device (200) may additionally generate an object image desired by the user on the first image based on the user input for the drawing UI. In addition, the electronic device (200) may also modify the detected edge image based on the user input for the drawing UI. The electronic device (200) can provide the image as input to a machine learning model when the image is completed based on user input to the drawing UI.

[0076] According to one embodiment, the electronic device (200) can generate an object image using an object image generation model (310) under the control of the processor (210).

[0077] In one embodiment, the electronic device (200) may generate a prompt requesting the machine learning model to generate an image based on a first image and a second user input. In the machine learning model, the prompt may represent an input for a user's question or request. The machine learning model may generate a corresponding response based on the prompt. The information provided in the prompt may play a significant role in shaping the machine learning model's response.

[0078] The machine learning model may include an object image generation model (310). As another example, the machine learning model may include an object placement control model (320). The object image generation model (310) may perform the role of generating an image using a generative artificial intelligence model or a machine learning model, and the object placement control model (320) may perform the role of determining the position to place the generated image on the background screen.

[0079] According to one embodiment, the electronic device (200) may provide either an edge image transmitted from the drawing input receiving unit (308) or an edge image with a drawing added thereto as an input to the object image generation model (310). The object image generation model (310) may generate a completed object image based on the edge information and output the result of applying coloring and lighting effects. The object image generation model (310) may be included in the electronic device (200) in an on-device form or may be located on an external server. The image output by the machine learning model may be stored on the server. The electronic device (200) may designate at least one of a keyword input by the user, an address of a web page including a crawled image, and a prompt used for image output as metadata and store it on the server together with the image output by the machine learning model.

[0080] According to one embodiment, the electronic device (200) can determine a location on the display where a second image is to be displayed based on a user input. The electronic device (200) can obtain information about the color and brightness of the background screen on which the second image is displayed, and change at least one of the color or brightness of the second image based on the information about the color and brightness of the background screen. The background screen may refer to the background displayed in the original image, or may refer to a newly added background. The electronic device (200) may use the background of the original image as is, or may add a new background based on the user input. The second image may refer to a result image output by the object image generation model (310). The size of the second image may be the same as that of the original image. Alternatively, the size of the second image may have an image size corresponding to a portion of an area corresponding to the original image. The area corresponding to the original image may refer to an area including an edge image and a drawing input area.

[0081] The electronic device (200) can use the lighting control unit (322) to measure the color and brightness of the second image and background screen.

[0082] According to one embodiment, the electronic device (200) may generate a prompt requesting the generation of a sound corresponding to a first image and a second user input together with the image. Here, the first image may refer to an edge image selected by the user. The second user input may refer to a user drawing input on the first image. According to one embodiment, the electronic device (200) may move the first image or change the size or color of the first image based on the second user input. The electronic device (200) may rotate the first image or flip it left and right based on the second user input.

[0083] The electronic device (200) may provide the generated prompt as input to the object image generation model (310) or the machine learning model, and receive a second image and sound information corresponding to the second image. The second image may include an image output by the machine learning model based on the first image and the second user input. The sound information may include sound information related to the second image. For example, if the second image includes a musical instrument (e.g., a guitar), the sound information related to the second image may include a guitar sound. The musical instrument (e.g., a guitar) is only an example, and the sound information is not limited thereto. The electronic device (200) may play a sound based on a user input (e.g., a touch input, a gesture input, or a voice input) for the second image.

[0084] According to one embodiment, the electronic device (200) can determine the placement of an object image generated using an object placement control model (320) under the control of the processor (210).

[0085] The object image generation model (310) and the object placement control model (320) may each exist in the form of separate machine learning models. That is, the object image generation model (310) may be included in a first machine learning model, and the object placement control model (320) may be included in a second machine learning model. Alternatively, the object image generation model (310) and the object placement control model (320) may be included in a single machine learning model and perform learning with different data.

[0086] According to one embodiment, the electronic device (200) may perform an operation of arranging a final object image (e.g., a second image) generated from the object image generation model (310) to match the surrounding background screen using the object placement control model (320). That the final object image (e.g., the second image) matches the background screen may mean that the difference between the color and brightness of the final object image and the color and brightness of the background screen is less than a specified level. For example, the electronic device (200) may change the brightness of the second image to be relatively lower when the brightness of the background screen is a first level (e.g., 30 lux) while the difference between the brightness of the final generated second image (e.g., 100 lux) and the first level exceeds a specified level. The mentioned brightness (e.g., 100 lux) is only an example and may vary depending on the settings.

[0087] Alternatively, the electronic device (200) may change the position at which the second image is placed so that the area where the placed object overlaps the second image is less than a specified level when another object is placed on the background screen. Alternatively, the electronic device (200) may move the other object when another object is placed on the background screen and place the second image in the center of the background screen.

[0088] According to one embodiment, a color can be expressed as at least one of an RGB value, HSL (hue, saturation, lightness), or HSV (hue, saturation, value). The electronic device (200) can use an RGB value, which is a method of expressing a color by combining the intensities of the three primary colors of light, red, green, and blue. The electronic device (200) can express the colors of images and the colors of the background screen as values ​​from 0 to 255. For example, the electronic device (200) can determine that (255, 0, 0) indicates red, (0, 255, 0) indicates green, and (0, 0, 255) indicates blue. The electronic device (200) can express various colors by combining RGB values.

[0089] Similar colors may mean that the difference in RGB values ​​is within a specified range (e.g., 20). The specified range (e.g., 20) is an example and may vary depending on the settings. HSL (hue, saturation, lightness) or HSV (hue, saturation, value) may refer to a method of expressing color. Hue is expressed as a value from 0 to 360 degrees, and can represent various colors by rotating along the color wheel. Saturation is the vividness of a color and can be expressed as a value from 0 to 100%. Lightness (or value) is the brightness of a color and can be expressed as a value from 0 to 100%. The electronic device (200) can display colors more intuitively than RGB by using HSL and HSV. The electronic device (200) can express similar or similar colors by using HSL and HSV.

[0090] Similar or similar colors may mean that the difference in hue values ​​is within a specified range (e.g., 10 degrees). The specified range (e.g., 10 degrees) is only an example and may vary depending on the settings. The color of the second image being distinct from the color of the background screen may mean that the difference between the RGB values ​​of the second image and the RGB values ​​of the background screen exceeds a specified level (e.g., 30). The specified level (e.g., 30) is only an example and may vary depending on the settings. Although described here based on RGB values, the electronic device (200) may also compare HSL (hue, saturation, lightness) or HSV (hue, saturation, value) to determine whether colors are similar.

[0091] According to one embodiment, when a user creates a new object on a blank screen, the electronic device (200) can place the created object on the blank screen without using the object placement control model (320). On the other hand, when a user creates a new object on an existing image, the electronic device (200) can adjust the placement of the existing image and the new object using the object placement control model (320). When the positioning of objects is completed based on user input (e.g., touch input or drag input), the electronic device (200) can adjust the brightness and color of the objects to output a final image in which editing is completed.

[0092] FIGS. 4A and 4B illustrate one embodiment of a method for editing an image of an electronic device.

[0093] In Figure 402 of Figure 4a, an electronic device (e.g., the electronic device (200) of Figure 2) can determine a background screen under the control of a processor (e.g., the processor (210) of Figure 2). The background screen may be a blank screen or an image containing an object.

[0094] In Figure 404 of FIG. 4a, the electronic device (200) can receive user input regarding a location (405) where an object image is to be generated, and display the location (405) where the object image is to be generated on a background screen.

[0095] In Figure 406 of FIG. 4A, the electronic device (200) may provide an interface through which a user can input a word. The electronic device (200) may receive a word corresponding to an object to be added to an existing image or blank canvas. When a user inputs a word, the electronic device (200) may use an inference function to recommend related words. By recommending related words, the electronic device (200) may enhance the user's input convenience.

[0096] In Figure 408 of Figure 4a, the electronic device (200) can perform crawling based on words entered by the user and display the performed images on the display. The electronic device (200) can receive the words entered by the user and proceed with crawling.

[0097] In one embodiment, the electronic device (200) may perform crawling by inputting more specific keywords based on keywords entered by the user. A specific keyword may refer to a keyword that includes modifiers or adjectives in addition to the word entered by the user. For example, the user's keyword may be specified based on words frequently entered by the user on the device's virtual keyboard.

[0098] As shown in 406 of Figure 4a, if the user inputs a word like "vase," the device can perform crawling by changing the keyword to "transparent vase," a word frequently entered by the user. This is merely an example, and the text input by the user and the keywords that specify it may vary depending on the settings.

[0099] Alternatively, the electronic device (200) may analyze the atmosphere of the background image and change it into a keyword corresponding to the background image to perform crawling. The electronic device (200) may analyze the atmosphere of the background image based on any one of the color, brightness, or surrounding objects of the background image. When performing crawling by changing a keyword, the electronic device (200) may perform crawling without notifying the user of the keyword change. Alternatively, the electronic device (200) may perform a process of confirming to the user through the UI whether to perform crawling with the changed keyword. When displaying the crawling result on the screen, the electronic device (200) may display the changed keyword on the screen together.

[0100] In Figure 410 of FIG. 4A, the electronic device (200) can perform edge detection on crawled images and display the edge images on a display. The electronic device (200) can perform edge detection on images acquired through crawling. The electronic device (200) can adjust the detection level of edge information by adjusting a preset threshold. Images for which edge detection has been completed can be organized into a single bundle. Images for which edge detection has been completed can be provided to the user in the form of a slide.

[0101] In Figure 412 of FIG. 4b, the electronic device (200) can receive a user input for a plurality of edge images on which edge detection has been performed. Based on the user input, the electronic device (200) can determine an edge image similar to an object intended by the user among the plurality of edge detection images.

[0102] In Figure 414 of Figure 4b, the electronic device (200) can display an edge image selected by the user on the background screen. The location where the selected edge image is displayed on the background screen may refer to the location (405) indicated in Figure 404.

[0103] In Figure 416 of FIG. 4b, the electronic device (200) may display a drawing user interface (UI) (426) to enable a user to add drawing input on an edge image selected by the user. The drawing user interface (426) may include, for example, a color palette for coloring.

[0104] According to one embodiment, the electronic device (200) can display a result according to a drawing input (430) input through the drawing user interface (426). The electronic device (200) can display an additional image based on the user input on the drawing UI (426). The user input on the drawing UI (426) can be input using the user's body (e.g., hand). Alternatively, the user input on the drawing UI (426) can be input using an external input device including an electronic pen or an input tablet. The drawing UI can include a pen, an eraser, a blur effect, and a palette for coloring. Alternatively, the electronic device (200) can receive a second user input on the first image without displaying the drawing user interface (UI) (426). The drawing user interface (UI) (426) may be a component that assists the user's input and may not be essential.

[0105] In Figure 418 of FIG. 4B, the electronic device (200) may provide the image generated in Figure 416 as input to a machine learning model. The image generated in Figure 416 may refer to an edge image to which a user's drawing input has been added. The electronic device (200) may output an image with coloring and lighting effects applied using the machine learning model. Additionally, the electronic device (200) may generate sound information related to the image using the machine learning model.

[0106] In Figure 420, the electronic device (200) can display an image with applied coloring and lighting effects output from a machine learning model. The electronic device (200) can play a sound based on sound information based on user input (e.g., touch input, gesture input, or voice input) for the displayed image.

[0107] In one embodiment, the electronic device (200) can output a sound when a user input is detected on a completed object. For example, if the object is a musical instrument, the electronic device (200) can output a musical instrument sound, and if the object is a cup made of iron, the electronic device (200) can output a sound corresponding to the material. The electronic device (200) can utilize the output sound to expand the user experience and provide information about what kind of object the generated image represents.

[0108] According to one embodiment, the electronic device (200) can change information about the color and brightness (illumination) of the generated object. The electronic device (200) can analyze the background screen and the generated object to adjust the position where the generated object is placed.

[0109] FIGS. 5A and 5B illustrate another embodiment of an image editing method of an electronic device.

[0110] In Figure 502 of Figure 5a, an electronic device (e.g., the electronic device (200) of Figure 2) can receive a keyword under the control of a processor (e.g., the processor (210) of Figure 2). The electronic device (200) can determine the keyword based on a user input. The user input can include at least one of a touch input on a keyboard or a voice input.

[0111] In Figure 504 of FIG. 5a, the electronic device (200) can perform crawling for the determined keyword.

[0112] In Figure 506 of FIG. 5A, the electronic device (200) may perform crawling and display at least some of the retrieved images on the display. The electronic device (200) may determine a single cup image based on a user input for any one of the displayed images.

[0113] In Figure 508 of FIG. 5A, the electronic device (200) can perform edge detection on a single selected cup image. The electronic device (200) can perform edge detection on a cup image selected by a user and acquire an edge image. The electronic device (200) can display the acquired edge image (e.g., a first image) on a display.

[0114] In Figure 510 of FIG. 5B, the electronic device (200) can additionally receive a drawing input (530) on the edge image or around the edge image. The drawing input (530) may be input by the user's body (e.g., hand) or by an external device (e.g., stylus pen). The electronic device (200) can separately display a user interface to assist the drawing input (530).

[0115] In Figure 512 of FIG. 5B, the electronic device (200) can generate an image using a machine learning model. The electronic device (200) can generate a prompt requesting image generation based on the edge image and the added drawing input, and provide the generated prompt to the machine learning model. In the machine learning model, the prompt can mean an input for a user's question or request. The machine learning model can generate an appropriate response based on the prompt. The information provided as the prompt can play an important role in forming the response of the machine learning model. For example, the prompt can be generated in the following form: "Check the location where indicators (e.g., images or objects) are displayed on the background screen, and change the color of the indicator so that each indicator can be distinguished from the background screen." For example, the object can mean a cup, and the image can mean smoke (or steam) on the cup.

[0116] In Figure 514 of FIG. 5b, the electronic device (200) can display an image output from a machine learning model on a display.

[0117] FIG. 6 is a flowchart illustrating an image editing method of an electronic device according to various embodiments.

[0118] The operations described through FIG. 6 can be implemented based on instructions that can be stored in a computer recording medium or memory (e.g., memory (130) of FIG. 1). The illustrated method (600) can be executed by the electronic device described above through FIGS. 1 to 5B (e.g., electronic device (101) of FIG. 1, electronic device (200) of FIG. 2), and the technical features described above will be omitted below. The order of each operation of FIG. 6 can be changed, some operations can be omitted, and some operations can be performed simultaneously.

[0119] In operation 610, the electronic device (200) may determine a keyword corresponding to the first user input under the control of a processor (e.g., the processor (210) of FIG. 2). The electronic device (200) may provide a search UI to receive a word for an object image that the user wishes to add.

[0120] In operation 620, the electronic device (200) may perform crawling based on keywords and acquire at least one image. At this time, the electronic device (200) may determine at least one image graphic object through edge detection for at least one image. Here, at least one graphic object may be referred to as a first image. The electronic device (200) may perform crawling based on the received words (keywords). The electronic device (200) may limit the number of images to be searched through crawling based on preset criteria. The electronic device (200) may perform edge detection for images for which crawling has been completed.

[0121] According to one embodiment, the electronic device (200) can control the detection of a minimum number of edges based on parameters related to edge detection. As the value of the parameter related to edge detection (e.g., threshold) increases, the image becomes more abstract, and thus relatively less edge information can be detected. As the value of the parameter related to edge detection (e.g., threshold) decreases, the image becomes relatively closer to the original, and thus relatively more edge information can be detected. The electronic device (200) can display images obtained by performing edge detection to the user using various types of UI (e.g., slide). The electronic device (200) can determine at least one edge image based on a user input.

[0122] In operation 630, the electronic device (200) may receive a second user input related to at least one graphic object. Furthermore, the electronic device (200) may display an interface to enable the user to add the second user input on or around the at least one graphic object, and may add the second user input. Alternatively, the electronic device (200) may receive the user input on the at least one graphic object without displaying the interface. The interface may not be essential as it is a configuration to assist the user in inputting or drawing.

[0123] According to one embodiment, the electronic device (200) may display a drawing UI for inputting a drawing based on the completion of edge image selection by the user. The electronic device (200) may display additional images based on user input on the drawing UI. The user input on the drawing UI may be input using the user's body (e.g., hand). Alternatively, the user input on the drawing UI may be input using an external input device including an electronic pen or an input tablet. The drawing UI may include a pen, an eraser, a blur effect, and a palette for coloring.

[0124] At operation 640, the electronic device (200) may request the machine learning model to generate an image based on a second user input or information about at least one graphic object.

[0125] According to one embodiment, the electronic device (200) may provide an edge image generated by a second user input as input to a machine learning model. The machine learning model may output an object image with applied coloring and lighting information based on the edge image generated by the second user input. The electronic device (200) may display the image with applied coloring and lighting effects output from the machine learning model on a display. The electronic device (200) may play a sound based on sound information based on a user input (e.g., a touch input, a gesture input, or a voice input) for the displayed image. The electronic device (200) may use the output sound to expand the user experience and provide information about what object the generated image represents.

[0126] According to one embodiment, the electronic device (200) can arrange the final object image (e.g., the second image) generated by the machine learning model to match the surrounding background screen. The final object image (e.g., the second image) matching the background screen may mean that the difference between the color and brightness of the final object image and the color and brightness of the background screen is less than a specified level. The electronic device (200) can perform an analysis of the background screen using the machine learning model. The electronic device (200) can receive information on the coordinates at which the completed object image is arranged, analyze the background screen, and compare it with the display coordinates of the object within the background screen. The electronic device (200) can determine the coordinates at which the completed object image is arranged by considering the coordinates of the object within the background screen.

[0127] FIG. 7 is a flowchart illustrating an image editing method of an electronic device according to various embodiments.

[0128] The operations described through FIG. 7 can be implemented based on instructions that can be stored in a computer recording medium or memory (e.g., memory (130) of FIG. 1). The illustrated method (700) can be executed by the electronic device described above through FIGS. 1 to 5B (e.g., electronic device (101) of FIG. 1, electronic device (200) of FIG. 2), and the technical features described above will be omitted below. The order of each operation of FIG. 7 can be changed, some operations can be omitted, and some operations can be performed simultaneously.

[0129] In operation 702, the electronic device (200) may determine either an image or a blank canvas under the control of a processor (e.g., processor (210) of FIG. 2).

[0130] In operation 704, the electronic device (200) may determine a keyword based on a user input. In one embodiment, the electronic device (200) may receive a first user input and determine a keyword corresponding to the user input. The first user input may include, for example, either a text input or a voice input.

[0131] In operation 706, the electronic device (200) may perform crawling and edge detection based on keywords. The electronic device (200) may perform crawling to obtain at least one image based on the determined keywords. The electronic device (200) may determine at least one image corresponding to the crawling. The electronic device (200) may perform edge detection on at least one image corresponding to the crawling to obtain a first image.

[0132] According to one embodiment, the electronic device (200) can control the number of images corresponding to crawling to be less than a preset level. The electronic device (200) can perform edge detection on at least one image for which crawling has been completed, and can provide an interface for receiving user input for any one of the images for which edge detection has been performed. The electronic device (200) can determine a first image based on a user input to the interface. The first image can refer to an edge image selected by the user.

[0133] According to one embodiment, the electronic device (200) can control the number of images corresponding to crawling to be less than a preset level. The electronic device (200) can receive a user input for one of the images corresponding to crawling and perform edge detection on the image to which the user input has been applied to determine the first image.

[0134] In operation 708, the electronic device (200) may perform a drawing operation on the edge image. The electronic device (200) may display an interface (e.g., a drawing UI) to enable adding a second user input on the first image, and may add the second user input on the first image based on the user input on the interface. The second user input may include at least one of a touch input, a gesture input, or a voice input for the interface. The types of the second user input are merely examples and are not limited thereto.

[0135] In operation 710, the electronic device (200) may input an image on which a drawing operation has been performed and a prompt to a machine learning model or an artificial intelligence learning model. The electronic device (200) may generate a prompt requesting the machine learning model to generate an image based on the first image and the second user input. The electronic device (200) may provide the generated prompt as an input to the machine learning model, and may receive a second image output from the machine learning model and display it on a display.

[0136] According to one embodiment, the electronic device (200) may generate a prompt based on at least one of the first user input or the keyword, and the first image and the second user input. Here, the prompt may refer to a command requesting image generation in a machine learning model. The electronic device (200) may provide the generated prompt as input to the machine learning model to request image output.

[0137] At operation 720, the electronic device (200) can determine whether the background is a blank screen or an image with an object present.

[0138] In operation 722, the electronic device (200) can display the image output from the machine learning model as is based on the background screen being a blank screen.

[0139] In operation 724, the electronic device (200) can determine a location on the background screen where an image output from a machine learning model is to be placed based on the fact that the electronic device (200) is not a blank screen but an image in which an object exists. The electronic device (200) can determine a location on the display where a second image is to be displayed based on a user input. The electronic device (200) can obtain information about the color and brightness of the background screen on which the second image is displayed, and can change at least one of the color or brightness of the second image based on the information about the color and brightness of the background screen. The electronic device (200) can analyze the background screen and the generated object to adjust a location where the generated object is to be placed.

[0140] According to one embodiment, the electronic device (200) can arrange the final object image (e.g., the second image) generated by the machine learning model to match the surrounding background screen. The final object image (e.g., the second image) matching the background screen may mean that the difference between the color and brightness of the final object image and the color and brightness of the background screen is less than a specified level. The electronic device (200) can perform an analysis of the background screen using the machine learning model. The electronic device (200) can receive information on the coordinates at which the completed object image is arranged, analyze the background screen, and compare it with the display coordinates of the object within the background screen. The electronic device (200) can determine the coordinates at which the completed object image is arranged by considering the coordinates of the object within the background screen.

[0141] In operation 726, the electronic device (200) may display an image output from the machine learning model on the background screen based on the location to be placed being determined.

[0142] According to one embodiment, the electronic device (200) may generate an image in which the background image and the generated object image are synthesized or merged based on the location where the image is to be placed. The electronic device (200) may store the generated image in a memory (e.g., the memory (220) of FIG. 2).

[0143] According to one embodiment, the electronic device can perform edge detection on at least one image to determine at least one graphical object.

[0144] According to one embodiment, an electronic device may generate a prompt requesting image generation based on a keyword corresponding to a first user input, at least one graphic object, and a second user input, provide the generated prompt as an input to a machine learning model, and receive a second image output from the machine learning model and display the second image on a display. The electronic device may determine a location on the display where the second image is to be displayed based on the user input, obtain information about a color and brightness of a background screen on which the second image is displayed, and change at least one of the color and brightness of the second image based on the information about the color and brightness of the background screen.

[0145] According to one embodiment, the electronic device controls the number of images corresponding to crawling to be less than a preset level, and performs edge detection on at least one image for which crawling has been completed. The electronic device provides an interface capable of receiving user input for any one of the images for which edge detection has been performed, and determines at least one graphic object based on the user input to the interface.

[0146] According to one embodiment, the electronic device can control the number of images corresponding to crawling to be less than a preset level, determine one image based on a user input for the corresponding image, and perform edge detection on the determined image to determine at least one graphic object.

[0147] According to one embodiment, the electronic device can be controlled to detect a minimum number of edges based on parameters related to edge detection. As the value of the edge detection parameter increases, the image becomes more abstract and thus the degree to which it is expressed as an edge increases. As the value decreases, the image becomes closer to the original and thus the degree to which it is expressed as an edge decreases.

[0148] The embodiments of this document disclosed in this specification and drawings are merely specific examples to easily explain the technical contents according to the embodiments of this document and to help understand the embodiments of this document, and are not intended to limit the scope of the embodiments of this document. Therefore, the scope of one embodiment of this document should be interpreted to include all changes or modified forms derived based on the technical idea of ​​one embodiment of this document, in addition to the embodiments disclosed herein.

Claims

1. In electronic devices, Memory that stores instructions; display; Contains at least one processor, The above instructions, when executed by the at least one processor, cause the electronic device to Determine a keyword corresponding to the first user input based on receiving the first user input, Obtain at least one image through crawling based on the above-determined keywords, Determine at least one graphical object corresponding to at least one image, receiving a second user input related to at least one graphic object; An electronic device that controls the generation of an artificial intelligence image using a machine learning model based on either the second user input or information about the at least one graphic object.

2. In paragraph 1, The above electronic device An electronic device for determining at least one graphical object by performing edge detection on at least one image.

3. In paragraph 1, The above electronic device Generate a prompt requesting image generation based on a keyword corresponding to the first user input, the at least one graphic object, and the second user input; Provide the generated prompt as input to the machine learning model, Receive a second image output from the machine learning model and display it on the display, determining where the second image is to be displayed on the display based on user input; Obtain information about the color and brightness of the background screen on which the second image is displayed, An electronic device that changes at least one of the color or brightness of the second image based on information about the color and brightness of the background screen.

4. In paragraph 1, The above electronic device Control the number of images corresponding to crawling to be less than a preset level, For at least one image for which crawling has been completed, edge detection is performed, and an interface is provided for receiving user input for any one of the images for which edge detection has been performed. An electronic device that determines at least one graphical object based on user input to the interface.

5. In paragraph 1, The above electronic device Control the number of images corresponding to crawling to be less than a preset level, Determine one image based on user input for the corresponding image, An electronic device for determining at least one graphic object by performing edge detection on a determined image.

6. In paragraph 1, The above electronic device Controls the detection of a minimum number of edges based on parameters related to edge detection. The parameters related to the above edge detection are As the value increases, the image becomes more abstract and the degree to which it is expressed as an edge increases. An electronic device in which the lower the value, the closer the image is to the original, and the less the edges are expressed.

7. In paragraph 1, The above electronic device Generate a prompt requesting the creation of an image along with at least one graphic object and a sound corresponding to the second user input, Providing the generated prompt as input to the machine learning model to receive a second image and sound information corresponding to the second image, The second image above is An image generated by the machine learning model based on at least one graphic object and the second user input, The above sound information is An electronic device comprising sound information related to the second image.

8. In paragraph 7, The above electronic device Based on the user input for the second image, music based on the sound information is output to the speaker, User input for the second image above is An electronic device comprising at least one of gesture, voice input or touch input.

9. In paragraph 1, The first user input for determining the above keyword is An electronic device that includes either text input (touch input) or voice input.

10. In paragraph 1, The above electronic device When creating a prompt, At least one of the first user input or the keyword, Generate a prompt requesting the machine learning model to generate an image based on at least one graphic object and the second user input, An electronic device that requests an image output by providing the generated prompt as input to the machine learning model.

11. In terms of operation method, An operation of determining a keyword corresponding to a first user input based on receiving the first user input; An action of obtaining at least one image through crawling based on the above-determined keyword; An action of determining at least one graphical object corresponding to at least one image; An operation of receiving a second user input related to at least one graphic object; and A method comprising generating an artificial intelligence image using a machine learning model based on either the second user input or information about the at least one graphic object.

12. In paragraph 11, A method further comprising the action of determining the at least one graphical object by performing edge detection on the at least one image.

13. In paragraph 11, An action of generating a prompt requesting image generation based on a keyword corresponding to the first user input, the at least one graphic object, and the second user input; The act of providing the generated prompt as input to the machine learning model; An operation of receiving a second image output from the machine learning model and displaying it on a display; An action of determining a location on the display where the second image is to be displayed based on user input; An operation of obtaining information about the color and brightness of the background screen on which the second image is displayed; A method further comprising an action of changing at least one of a color or brightness of the second image based on information about a color and brightness of the background screen.

14. In paragraph 11, An action to control the number of images corresponding to crawling to be less than a preset level; An action to perform edge detection on at least one image for which crawling has been completed; An operation that provides an interface for receiving user input for any one of the images on which edge detection has been performed; and A method further comprising determining at least one graphical object based on user input to the interface.

15. In paragraph 11, An action to control the number of images corresponding to crawling to be less than a preset level; An action to determine an image based on user input for the corresponding image; and A method further comprising the action of determining at least one graphic object by performing edge detection on the determined image.

Images