Electronic device, method, and recording medium for supporting image editing

Abstract

An embodiment of the present disclosure provides an electronic device, a method, and a recording medium for supporting image editing. The electronic device according to an embodiment may crop a portion corresponding to an object in an image on the basis of an input for selecting the portion corresponding to the object in the image while the image is displayed on a display. The electronic device may move the cropped portion on the basis of an input for moving the cropped portion from the original position in the image. The electronic device may set and display, as a masking area, an area corresponding to the original position of the cropped portion in the image on the basis of the movement of the cropped portion. The electronic device may provide a first handler related to editing of the cropped portion through the cropped portion on the basis of an input for selecting the cropped portion. The electronic device may provide a second handler related to editing of the cropped portion through the masking area on the basis of an input for selecting the masking area. The electronic device may edit an image in response to an input received on the basis of the first handler or the second handler. The electronic device may generate an instruction to perform in-painting and / or out-painting on the basis of the edited image. The electronic device may obtain a result image in relation to the instruction and display the result image through a display.

Description

Electronic device, method and recording medium supporting image editing

[0001] Embodiments of the present disclosure provide an electronic device, an operating method thereof, and a recording medium that support image editing based on artificial intelligence (AI) (e.g., generative AI).

[0002] With the advancement of digital technology, various types of electronic devices, such as smartphones, digital cameras, and / or wearable devices, are becoming widely used. The hardware and / or software components of these electronic devices are continuously being developed to support and enhance their functionality.

[0003] For example, portable electronic devices (hereinafter referred to as "electronic devices"), such as smartphones, can now be equipped with a variety of functions. Electronic devices include touchscreen-based displays that allow users to easily access various functions, and can display screens for various applications through these displays.

[0004] Recently, with the rapid development of big data and deep learning technologies, artificial intelligence (AI) has been applied to electronic devices. It is also being applied to intelligent personal services that analyze specific data and integrate and utilize information from various fields tailored to the user. For example, users can control electronic devices through voice conversations, and a deep learning-based knowledge base enables them to search for, query, and respond to specific information. Recently, generative AI has been implemented as AI technology evolves. Generative AI can refer to AI technology that generates similar content using existing content, such as text, audio, and / or images. For example, generative AI can refer to AI technology that can generate content (e.g., text, audio, images, and / or video) that responds to a given input.

[0005] Meanwhile, the electronic device may display an image through a display and provide a function for editing the image based on user input regarding the displayed image. In one embodiment, the user may expand the image displayed on the display or draw a new object within the image based on user input.

[0006] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0007] In one embodiment of the present disclosure, an electronic device, a method of operation thereof, and a recording medium are provided that support image editing based on generative AI (e.g., image expansion, background generation, object generation, separation, and / or addition within an image).

[0008] In one embodiment of the present disclosure, an electronic device, a method of operation thereof, and a recording medium are provided for editing an image based on interaction with a user, and reconstructing (or regenerating) and providing the image on a server or on-device based on the edited image.

[0009] In one embodiment of the present disclosure, an electronic device, a method of operation thereof, and a recording medium are provided that support changing the position, size, tilt, angle, horizontality, and / or verticality of one or more objects within an image, removing an object by separating it from an image, or adding a new object within an image.

[0010] In one embodiment of the present disclosure, an electronic device, an operating method thereof, and a recording medium are provided that generate a background around an object (e.g., a subject) attached to an edge of an image so that the object can be placed in a desired area within the image, and that supports editing an image (e.g., a landscape photograph, a portrait photograph, a background photograph, an object photograph, an animal photograph, and / or a food photograph) at a desired angle and composition by a user.

[0011] The technical problems to be achieved in this document are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0012] According to an embodiment of the present disclosure, an electronic device supporting image editing based on artificial intelligence (AI) (e.g., generative AI) is provided. The electronic device may include a display, communication circuitry, a memory storing one or more computer programs, and at least one processor communicatively coupled to the display, the communication circuitry, and the memory, the processor including processing circuitry. The one or more computer programs may include instructions (e.g., computer-executable instructions) that, when individually or collectively executed by the one or more processors, cause the electronic device to perform the following:

[0013] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display an image on the display. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to receive a first input selecting a portion corresponding to an object in the image. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to crop a portion corresponding to the object in the image based on the first input. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to detect a second input moving the cropped portion from an original location within the image. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display an area corresponding to an original position of the cropped portion in the image as a masking area based on movement of the cropped portion. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to receive a third input selecting one of the cropped portion or the masking area. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to provide a first handler associated with editing of the cropped portion through the cropped portion when the third input corresponds to selecting the cropped portion.The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to provide a second handler associated with editing the cropped portion through the masking area, if the third input corresponds to selecting the masking area. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to edit the image in response to a fourth input received based on the first handler or the second handler. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to generate an instruction that performs inpainting and / or outpainting based on the edited image, based on a fifth input. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to obtain a result image in relation to the instructions. The above instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to display the resulting image through the display.

[0014] An operating method of an electronic device according to an embodiment of the present disclosure may include an operation of displaying an image on a display. The operating method may include an operation of receiving a first input for selecting a portion corresponding to an object in the image. The operating method may include an operation of cropping a portion corresponding to the object in the image based on the first input. The operating method may include an operation of detecting a second input for moving the cropped portion from an original location in the image. The operating method may include an operation for setting and displaying an area corresponding to the original location of the cropped portion in the image as a masking area based on the movement of the cropped portion. The operating method may include an operation of receiving a third input for selecting either the cropped portion or the masking area. The operating method may include an operation of providing a first handler related to editing of the cropped portion through the cropped portion when the third input corresponds to selecting the cropped portion. The method of operation may include an operation of providing a second handler related to editing of the cropped portion through the masking area when the third input corresponds to selecting the masking area. The method of operation may include an operation of editing the image in response to a fourth input received based on the first handler or the second handler. The method of operation may include an operation of generating an instruction to perform inpainting and / or outpainting based on the edited image based on a fifth input. The method of operation may include an operation of obtaining a result image in relation to the instruction.The above method of operation may include an operation of displaying the result image through the display.

[0015] In order to solve the above-described problem, various embodiments of the present disclosure may include a computer-readable recording medium having recorded thereon a program for executing the method on a processor.

[0016] According to one embodiment, a non-transitory computer-readable recording medium (or computer program product) storing one or more programs is described. According to one embodiment, one or more programs are configured to: display an image on a display; receive a first input for selecting a portion of the image corresponding to an object; crop a portion of the image corresponding to the object based on the first input; detect a second input for moving the cropped portion from an original location within the image; set and display an area of ​​the image corresponding to the original location of the cropped portion as a masking area based on the movement of the cropped portion; receive a third input for selecting either the cropped portion or the masking area; provide a first handler related to editing of the cropped portion through the cropped portion when the third input corresponds to selecting the cropped portion; provide a second handler related to editing of the cropped portion through the masking area when the third input corresponds to selecting the masking area; in response to a fourth input received based on the first handler or the second handler, The instruction may include an action of editing the image, an action of generating an instruction to perform inpainting and / or outpainting based on the edited image based on a fifth input, an action of obtaining a result image in relation to the instruction, and an action of displaying the result image through the display.

[0017] Further scope of the applicability of the present disclosure will become apparent from the detailed description below. However, since various modifications and variations within the spirit and scope of the present disclosure will readily become apparent to those skilled in the art, it should be understood that the detailed description and specific examples, such as preferred embodiments of the present disclosure, are given by way of example only.

[0018] According to an embodiment of the present disclosure, an electronic device, an operating method thereof, and a recording medium thereof, it is possible to support changing the position, size, inclination, angle, horizontality, and / or verticality of one or more objects within an image, separating and removing an object within an image, or adding a new object within an image. For example, a user may be supported to separate at least one object (e.g., a subject) from an image and finely adjust the desired position and / or size as well as the angle / horizontal / verticality using a touch gesture or a dialer, thereby providing a new user experience (UX) that allows the user to create a desired composition of a photograph.

[0019] In one embodiment, a background may be generated around an object (e.g., a subject) attached to an edge of an image, allowing the object to be positioned in a desired area within the image, and supporting editing of images (e.g., landscape photos, portrait photos, background photos, object photos, animal photos) at a desired angle and composition by a user. For example, a new user experience may be provided by providing a naturally extended background that allows the user to edit the image at a desired angle and composition.

[0020] In addition, various effects may be directly or indirectly realized through this document. The effects obtained through this disclosure are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art to which this disclosure pertains, based on the description below.

[0021] In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar components.

[0022] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0023] FIG. 2 is a block diagram illustrating an integrated intelligence system according to one embodiment.

[0024] FIG. 3 is a diagram schematically illustrating the configuration of an electronic device according to one embodiment of the present disclosure.

[0025] FIG. 4 is a flowchart illustrating a method of operating an electronic device according to one embodiment of the present disclosure.

[0026] FIG. 5 is a diagram illustrating an example of a user interface that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0027] FIGS. 6 to 16 are drawings illustrating examples of image editing operations in an electronic device according to one embodiment of the present disclosure.

[0028] FIGS. 17 and 18 are diagrams illustrating examples of a user interface during image generation in an electronic device according to one embodiment of the present disclosure.

[0029] FIGS. 19 and 20 are diagrams illustrating examples of providing a result image according to image editing in an electronic device according to one embodiment of the present disclosure.

[0030] FIG. 21 is a diagram illustrating an example of providing a result image according to image editing in an electronic device according to one embodiment of the present disclosure.

[0031] FIGS. 22 to 26 are drawings illustrating examples of image editing operations in an electronic device according to one embodiment of the present disclosure.

[0032] FIG. 27 is a flowchart illustrating a method of operating an electronic device according to one embodiment of the present disclosure.

[0033] FIGS. 28A and 28B are flowcharts illustrating a method of operating an electronic device according to one embodiment of the present disclosure.

[0034] FIG. 29 is a flowchart illustrating a method of operating an electronic device according to one embodiment of the present disclosure.

[0035] FIG. 30 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0036] FIG. 31 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0037] FIG. 32 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0038] FIG. 33 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0039] FIG. 34 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0040] FIG. 35 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0041] FIG. 36 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0042] FIG. 37 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0043] FIG. 38 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0044] FIGS. 39A and 39B are diagrams illustrating examples of operations supporting image editing in an electronic device according to one embodiment of the present disclosure.

[0045] FIG. 40 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0046] FIG. 41 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0047] FIG. 42 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0048] FIG. 43 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0049] FIG. 44 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0050] FIG. 45 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0051] FIG. 46 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0052] FIG. 47 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0053] FIGS. 48A and 48B are diagrams illustrating examples of operations supporting image editing in an electronic device according to one embodiment of the present disclosure.

[0054] FIGS. 49A and 49B are diagrams illustrating examples of operations supporting image editing in an electronic device according to one embodiment of the present disclosure.

[0055] FIG. 50 is a diagram illustrating an example of an operation supporting image editing in an electronic device according to one embodiment of the present disclosure.

[0056] FIG. 51 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0057] FIG. 52 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0058] FIG. 53 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0059] FIGS. 54A and 54B are diagrams illustrating examples of operations supporting image editing in an electronic device according to one embodiment of the present disclosure.

[0060] FIG. 55 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0061] FIG. 56 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0062] FIG. 57 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0063] FIG. 58 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0064] FIG. 59 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0065] FIG. 60 is a diagram illustrating an example of an operation supporting image editing in an electronic device according to one embodiment of the present disclosure.

[0066] FIG. 61 is a flowchart illustrating a method of operating an electronic device according to one embodiment of the present disclosure.

[0067] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0068] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to various embodiments.

[0069] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0070] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit (CPU) or an application processor (AP)) or an auxiliary processor (123) (e.g., a graphic processing unit (GPU), a neural processing unit (NPU), an image signal processor (ISP), a sensor hub processor, or a communication processor (CP)) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0071] The auxiliary processor (123) may control at least a part of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0072] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0073] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (OS) (142), middleware (144), or an application (146).

[0074] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0075] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0076] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0077] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0078] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0079] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, or an audio interface.

[0080] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0081] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0082] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0083] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least a part of a power management integrated circuit (PMIC).

[0084] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0085] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0086] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). NR access technology can support high-speed transmission of high-capacity data (eMBB, enhanced mobile broadband), minimizing terminal power and connecting multiple terminals (mMTC, massive machine type communications), or high reliability and low latency communications (URLLC, ultra-reliable and low-latency communications). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0087] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas by, for example, the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0088] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

[0089] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0090] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0091] FIG. 2 is a block diagram illustrating an integrated intelligence system according to one embodiment.

[0092] Referring to FIG. 2, an integrated intelligent system of one embodiment may include an electronic device (201) (e.g., the electronic device (101) of FIG. 1), an intelligent server (300), and a service server (399).

[0093] According to the illustrated embodiment, the electronic device (201) may include a communication interface (210), an input / output (I / O) interface (220), a processor (230), and / or a memory (240). The components listed above may be operatively or electrically connected to each other. For example, the electronic device (201) may include at least some of the components of the electronic device (101) of FIG. 1.

[0094] The communication interface (210) can be connected to an external device (e.g., an intelligent server (300) and / or a service server (399)) via a network (299) (e.g., any network including a cellular network and / or a wireless local area network (WLAN)) to transmit and receive data. For example, the communication interface (210) can correspond to the CP and / or communication circuit of FIG. 1. The I / O interface (220) can receive user input, process received user input, and / or output a result processed by the processor (230) using an input / output device (not shown) (e.g., a microphone, a speaker, and / or a display (e.g., a display of FIG. 1).

[0095] The processor (230) may be operatively or electrically connected to the communication interface (210), the I / O interface (220), and / or the memory (240) (e.g., the memory of FIG. 1) to perform a designated operation. For example, the processor (230) may correspond to the processor (120) of FIG. 1. The processor (230) may execute a program (or one or more instructions) stored in the memory (240) to perform a designated operation. For example, the processor (230) may receive a user's voice input (e.g., a user's speech) through the I / O interface (220) or from an external electronic device. The processor (230) may transmit the voice input received through the communication interface (210) to the intelligent server (300). For example, the processor (230) may include one or more processors.

[0096] The processor (230) may receive a result corresponding to the voice input from the intelligent server (300). For example, the processor (230) may receive a plan corresponding to the voice input and / or a result calculated using the plan from the intelligent server (300). For example, the plan may include, but is not limited to, information regarding a plurality of sequential operations to be executed by the first electronic device (201) and / or another electronic device in relation to the voice input. The processor (230) may receive a request from the intelligent server (300) to obtain information (e.g., entities, slots, and / or parameters) necessary to generate a plan corresponding to the voice input. The processor (230) may transmit the necessary information to the intelligent server (300) in response to the request.

[0097] The processor (230) may visually, tactilely, and / or audibly output the results of executing the operations specified according to the plan via the I / O interface (220). For example, the processor (230) may sequentially display the execution results of multiple operations on the display. As an example, the processor (230) may display only the execution results of executing multiple operations (e.g., the execution results of one of the multiple operations or the last operation) on the display.

[0098] The processor (230) can recognize voice input. For example, the processor (230) can execute an intelligent app (or a voice recognition app) to process the voice input in response to a specified voice input (e.g., "Wake up!"). The processor (230) can provide a voice recognition service through the intelligent app. The processor (230) can transmit the voice input to the intelligent server (300) through the intelligent app and receive a result corresponding to the voice input from the intelligent server (300).

[0099] An intelligent server (300) of one embodiment can receive a user's voice input from an electronic device (201) via a network (299). The intelligent server (300) can convert audio data corresponding to the received voice input into text data. The intelligent server (300) can generate at least one plan for performing a task corresponding to the user's voice input based on the text data. The intelligent server (300) can transmit the generated plan or a result according to the generated plan to the electronic device (201) via the network (299).

[0100] An intelligent server (300) of one embodiment may include a front end (310), a natural language platform (320), a capsule database (330), an execution engine (340), and / or an end user interface (350).

[0101] The front end (310) can receive a voice input received by the electronic device (201) from the electronic device (201). The front end (310) can transmit a response corresponding to the voice input to the electronic device (201).

[0102] The natural language platform (320) may include an automatic speech recognition (ASR) module (321), a natural language understanding (NLU) module (323), a planner module (325), a natural language generator (NLG) module (327), and / or a text-to-speech (TTS) module (329).

[0103] The automatic speech recognition module (321) can convert the voice input received from the electronic device (201) into text data. The natural language understanding module (323) can identify the user's intent and / or parameters (e.g., entities and / or slots) based on the text data of the voice input. The user's intent corresponds to the voice input and may include information indicating an action (or function) that the user wishes to perform using the device. The slot may be detailed information related to the user's intent. The slot may be acquired based on a domain corresponding to the utterance. The slot may be variable information required to perform the action. In one embodiment, the variable information constituting the slot may include a named entity.

[0104] The planner module (325) can generate a plan using the intent and / or parameters determined by the natural language understanding module (323). For example, the planner module (325) can determine at least one domain necessary to perform a task based on the determined intent. The planner module (325) can determine a plurality of operations included in each of the at least one domain determined based on the intent. The domain may correspond to a category (or service) associated with an operation (or function) that the user wishes to perform using the device. The domain may be classified according to a service (e.g., an app) related to the text. The domain may be related to the user's intent corresponding to the text. The domain may be classified according to, for example, the type of application that received the voice input and / or the type of service to be provided based on the voice input, but is not limited thereto. In one example, the determination of the domain may be performed by another module (e.g., the natural language understanding module (323)). The planner module (325) can determine parameters required to execute a plurality of determined actions or result values ​​output by the execution of the plurality of actions. The parameters and result values ​​can be defined as concepts of a specified format (or class). For example, the plan can include a plurality of actions and / or a plurality of concepts determined by the user's intention. The planner module (325) can determine the relationship between the plurality of actions and / or the plurality of concepts in a step-by-step (or hierarchical) manner. For example, the planner module (325) can identify the execution order of the plurality of actions (e.g., the plurality of actions determined based on the user's intention) based on the plurality of concepts (e.g., parameters required to execute the plurality of actions and results output by the execution of the plurality of actions). The planner module (325) can generate a plan including association information (e.g., ontology) between the plurality of actions and the plurality of concepts.The planner module (325) can create a plan using information (e.g., at least one capsule) stored in a capsule database (330) in which a set of relationships between concepts and actions is stored.

[0105] The planner module (325) can generate a plan based on an artificial intelligence (AI) system. For example, the AI ​​system can include one or more electronic devices and / or one or more processing circuits to execute a rule-based system, a neural network-based system (e.g., a feedforward neural network (FNN), a recurrent neural network (RNN)), or a combination thereof. The AI ​​system described above is exemplary, and the AI ​​system can be an AI system based on any machine learning-based model. The planner module (325) can select a plan corresponding to a user request from a set of predefined plans, or generate a plan in real time in response to a user request.

[0106] The natural language generation module (327) can convert specified information into text format. The information converted into text format may be in the form of natural language speech. The text-to-speech conversion module (329) can convert information in text format into information in speech format.

[0107] The capsule database (330) can store information on the relationship between multiple concepts and actions corresponding to multiple domains (e.g., applications). The capsule database (330) can store at least one capsule (e.g., capsule (331) and / or capsule (333)) in the form of a concept action network (CAN). For example, the capsule database (330) can store actions for processing tasks corresponding to a user's voice input and parameters required for the actions in the form of a CAN. A capsule can include multiple action objects (or action information) and / or concept objects (or concept information) included in a plan. For example, capsules (331, 333) can be created for each domain and stored in the capsule database (330), but are not limited thereto.

[0108] The execution engine (340) can produce results using the generated plan. The end user interface (350) can transmit the produced results to the electronic device (201).

[0109] According to one embodiment, some functions (e.g., the natural language platform (320)) or all functions of the intelligent server (300) may be implemented in the electronic device (201). For example, the electronic device (201) may execute one or more programs including the natural language platform (e.g., the natural language platform (320) of FIG. 3) separately from the intelligent server (300). For example, the electronic device (201) may directly perform at least some of the operations of the natural language platform (320) of the intelligent server (300) (e.g., the automatic speech recognition module (321), the natural language understanding module (323), the planner module (325), the natural language generation module (327), and / or the text-to-speech module (329)).

[0110] In one embodiment, a service server (399) may provide a service (e.g., food ordering or hotel reservation) designated to an electronic device (201). The service server (399) may be a server operated by a different operator than the intelligent server (300). The service server (399) may communicate with the intelligent server (300) and / or the electronic device (201) via a network (299). The service server (399) may communicate with the intelligent server (300) via a separate connection (not shown). The service server (399) may provide the intelligent server (300) with information for generating a plan corresponding to a voice input received in the electronic device (201) (e.g., operation information and / or concept information for providing a designated service). The provided information may be stored in a capsule database (330). The service server (399) may provide the intelligent server (300) with result information according to the plan received from the electronic device (201).

[0111] FIG. 3 is a diagram schematically illustrating the configuration of an electronic device according to one embodiment of the present disclosure.

[0112] Referring to FIG. 3, an electronic device (101) according to one embodiment of the present disclosure may include a display (490) (e.g., the display module (160) of FIG. 1 or the I / O interface (220) of FIG. 2), a memory (130) (e.g., the memory (130, 240) of FIG. 1 or 2), a communication circuit (495) (e.g., the communication module (190) or the communication interface (210) of FIG. 1), and / or a processor (120) (e.g., the processor (120, 230) of FIG. 1 or 2). According to one embodiment, the electronic device (101) may include all or at least a part of the components of the electronic device (101, 201) described in the description with reference to FIG. 1 or 2. For example, in various embodiments of the present document, some of the illustrated configurations may be omitted or replaced. The electronic device (101) may include at least some of the configurations and / or functions of the electronic device (101) of FIG. 1 and / or the electronic device (201) of FIG. 2. At least some of the respective configurations of the illustrated (or not illustrated) electronic devices (101, 201) may be operatively, functionally and / or electrically connected to each other.

[0113] According to one embodiment, the display (490) may include a configuration identical or similar to the display module (160) of FIG. 1. According to one embodiment, the display (490) may display various images provided from the processor (120). According to one embodiment, the display (490) may visually provide, under the control of the processor (120), an application being executed (e.g., the application (146) of FIG. 1) and various screens related to its use (e.g., a contents screen, an application execution screen, a menu screen, and / or a function execution screen).

[0114] According to one embodiment, the display (490) may be combined with a touch sensor, a pressure sensor capable of measuring the intensity of a touch, and / or a touch panel (e.g., a digitizer) that detects a magnetic stylus pen. According to one embodiment, the display (490) may detect a touch input, an air gesture input, and / or a hovering input (or a proximity input) by measuring a change in a signal (e.g., voltage, light intensity, resistance, electromagnetic signal, and / or charge) for a specific location of the display (490) based on the touch sensor, the pressure sensor, and / or the touch panel. For example, the display (490) may include a touchscreen that detects a touch and / or a proximity touch (or a hovering) input using a part of a user's body (e.g., a finger) or an input device (e.g., a stylus pen). The display (490) may include at least some of the configuration and / or functions of the display module (160) of FIG. 1 and / or the I / O interface (220) of FIG. 2.

[0115] In one embodiment, the display (490) may include, but is not limited to, a liquid crystal display (LCD), a light-emitting diode (LED), an organic light-emitting diode (OLED) display, and / or an active matrix OLED (AMOLED) display, a micro electro mechanical systems (MEMS) display, or an electronic paper display. In one embodiment, the display (490) may include a flexible display.

[0116] According to one embodiment, the memory (130) includes at least a portion of the configuration and / or function of the memory (130) of FIG. 1 and / or the memory (240) of FIG. 2, and may store software (e.g., the program (140) of FIG. 1). The memory (130) may store various applications (e.g., the application (146) of FIG. 1) and program modules (e.g., client modules) that support intelligent services.

[0117] According to one embodiment, the memory (130) may store various data used by at least one component (e.g., processor (120)) of the electronic device (101). In one embodiment, the data may include, for example, software (e.g., program (140) of FIG. 1), and input data or output data for commands related to the software.

[0118] According to one embodiment, the memory (130) may include volatile memory (e.g., volatile memory (132) of FIG. 1) or non-volatile memory (134) (e.g., non-volatile memory (134) of FIG. 1). According to one embodiment, the memory (130) may store instructions or data received from the processor (120) in the volatile memory (132), and may store result data of instructions or data stored in the volatile memory (132) being processed by the processor (120) in the non-volatile memory (134).

[0119] In one embodiment, the data stored in the memory (130) may include various data (e.g., training data, prompt data, context, and / or training models) for supporting the electronic device (101) in editing and generating (e.g., regenerating or reconstructing) artificial intelligence-based data (e.g., images). In one embodiment, the data stored in the memory (130) may include information regarding various settings for supporting the electronic device (101) in controlling operations for editing and / or generating artificial intelligence-based data (e.g., images).

[0120] In one embodiment, the data stored in the memory (130) may include various learning data and / or parameters acquired based on the user's learning through interaction with the user. In one embodiment, the data may include various schemas (or algorithms, models, networks, or functions) for supporting artificial intelligence-based image editing and / or creation operations.

[0121] In one embodiment, the fields in which artificial intelligence technology is applied may be diverse. For example, it may consist of technologies in the fields of linguistic understanding, visual understanding, inference / prediction, knowledge representation, and / or motion control. Linguistic understanding refers to technologies that recognize and apply / process human language / characters, and may include natural language processing, machine translation, conversational systems, question answering, and / or speech recognition / synthesis. Visual understanding refers to technologies that recognize and process objects similar to human vision, and may include object recognition, object tracking, image search, person recognition, scene understanding, spatial understanding, and / or image enhancement. Inference / prediction refers to technologies that logically infer and predict by judging information, and may include knowledge / probability-based inference, optimized prediction, preference-based planning, and / or recommendation. Knowledge representation refers to technologies that automatically process human experience information into knowledge data, and may include knowledge construction (e.g., data generation / classification) and / or knowledge management (e.g., data utilization). Motion control is a technology for controlling the movement of an electronic device (101), and may include motion control and / or manipulation control (e.g., action control).

[0122] In one embodiment, a scheme for supporting artificial intelligence-based image editing and / or creation operations in an electronic device (101) may include a neural network. In one embodiment, the neural network may include a neural network model based on at least one of an artificial neural network (ANN), a convolution neural network (CNN), a region with convolution neural network (R-CNN), a region proposal network (RPN), a recurrent neural network (RNN), a stacking-based deep neural network (S-DNN), a state-space dynamic neural network (S-SDNN), a deconvolution network, a deep belief network (DBN), a restricted Boltzman machine (RBM), a long short-term memory (LSTM) network, a classification network, a plain residual network, a dense network, a hierarchical pyramid network, and / or a fully convolutional network. According to one embodiment, the type of the neural network model is not limited to the examples described above.

[0123] According to one embodiment, the memory (130) may store instructions that, when individually and / or collectively executed by the processor (120), cause the electronic device (101) to perform operations. For example, the instructions may be stored as software (e.g., the program (140) of FIG. 1) on the memory (130) and executable by the processor (120). For example, the instructions may include control commands such as arithmetic and logical operations, data movement, and / or input / output that may be recognized by the processor (120). According to one embodiment, the software may include various applications (e.g., application (146) of FIG. 1) that may provide various functions (or services) (e.g., routine functions, call functions, message functions, messenger functions, e-mail functions, SNS (social networking service) functions, search functions, media (e.g., video and / or music) playback functions, game functions, and / or wireless communication functions) on the electronic device (101).

[0124] According to one embodiment, the communication circuit (495) may support the establishment of a designated wireless communication channel (e.g., short-range communication such as Bluetooth communication and / or BLE communication) and the performance of communication through the established wireless communication channel. For example, the communication circuit (495) may perform designated communication (e.g., Bluetooth communication and / or BLE communication) with an external device. According to one embodiment, the communication circuit (495) may support wireless communication with an external device using cellular wireless communication (e.g., 4G LTE, 5G NR) and / or short-range wireless communication (e.g., Wi-Fi). For example, the electronic device (101) may communicate with an external server (e.g., a generative artificial intelligence server) that provides an artificial intelligence-based function (or service) through a network using the communication circuit (495). According to one embodiment, the communication circuit (495) may transmit data (e.g., images and / or prompts) generated in the electronic device (101) to the external server and receive data (e.g., images) transmitted from the external server. According to one embodiment, the communication circuit (495) may include at least some of the configuration and / or functionality of the communication module (190) of FIG. 1 and / or the communication interface (210) of FIG. 2.

[0125] According to one embodiment, the processor (120) may perform an application layer processing function requested by a user of the electronic device (101). According to one embodiment, the processor (120) may provide control and commands of functions for various blocks of the electronic device (101). According to one embodiment, the processor (120) may perform operations or data processing related to control and / or communication of each component of the electronic device (101). For example, the processor (120) may include at least some of the configurations and / or functions of the processor (120) of FIG. 1. According to one embodiment, the processor (120) may be operatively connected to the components of the electronic device (101). According to one embodiment, the processor (120) may load commands or data received from other components of the electronic device (101) into the memory (130), process the commands or data stored in the memory (130), and store result data.

[0126] In one embodiment, the processor (120) may include at least one processor including processing circuitry and / or executable program elements. In one embodiment, the processor (120) may control (or process) the overall operations related to supporting artificial intelligence-based image editing and / or reconstruction based on the processing circuitry and / or executable program elements.

[0127] According to one embodiment, the processor (120) may display an image on a display (490) (e.g., the display module (160) of FIG. 1). According to one embodiment, the processor (120) may receive a first input for selecting a portion corresponding to an object in the image. According to one embodiment, the processor (120) may crop a portion corresponding to the object in the image based on the first input.

[0128] In one embodiment, the processor (120) may detect a second input that moves the cropped portion from its original location within the image. In one embodiment, the processor (120) may set and display an area corresponding to the original location of the cropped portion within the image as a masking area based on the movement of the cropped portion.

[0129] In one embodiment, the processor (120) may receive a third input that selects either a cropped portion or a masked area. In one embodiment, the processor (120) may determine the area in which the third input is received (e.g., the cropped portion or the masked area).

[0130] In one embodiment, when the third input corresponds to selecting a cropped portion, the processor (120) may provide a first handler related to editing the cropped portion through the cropped portion. In one embodiment, when the third input corresponds to selecting a masking area, the processor (120) may provide a second handler related to editing the cropped portion through the masking area.

[0131] According to one embodiment, the processor (120) can edit the image in response to a fourth input received based on the first handler or the second handler.

[0132] In one embodiment, the processor (120) may generate instructions (or prompts) to perform inpainting and / or outpainting on the edited image based on the fifth input. In one embodiment, the processor (120) may obtain a result image in relation to the instructions. In one embodiment, the processor (120) may display the result image via the display (490).

[0133] According to one embodiment, the processor (120) may display an image on the display (490) (e.g., the display module (160) of FIG. 1). According to one embodiment, the processor (120) may detect a user input for performing editing of an image being displayed on the display (490). According to one embodiment, the processor (120) may determine the type of image editing based on the user input. According to one embodiment, based on determining that the image editing is a first editing, the processor (120) may perform a first editing based on at least one object based on the user input. According to one embodiment, based on determining that the image editing is a second editing, the processor (120) may perform a second editing based on the image based on the user input. According to one embodiment, the processor (120) may detect an image generation request. According to one embodiment, based on the image generation request, the processor (120) may perform an image generation process based on an image edited by the first editing and / or the second editing. According to one embodiment, the processor (120) may display an image generated (e.g., regenerated or reprocessed) based on the image generation process on the display (490).

[0134] According to one embodiment, the processor (120) may display an image on the display (490) (e.g., the display module (160) of FIG. 1). According to one embodiment, the processor (120) may detect a first input for selecting at least one object from the image. According to one embodiment, the processor (120) may isolate at least one object from the image based on the first input. According to one embodiment, the processor (120) may set an area corresponding to at least one object in the image as a masking area and display the area. According to one embodiment, the processor (120) may set a link between at least one object and the masking area. According to one embodiment, the processor (120) may detect a second input based on at least one object or masking area. According to one embodiment, the processor (120) may edit at least one object based on the second input. In one embodiment, the processor (120) may perform an image generation process based on generative AI based on a third input. In one embodiment, the processor (120) may obtain a result image in which at least one object and a masking area are reconstructed based on the image generation process. In one embodiment, the processor (120) may display the result image through a display.

[0135] According to one embodiment, the detailed operation of the processor (120) of the electronic device (101) is described with reference to the drawings described below.

[0136] According to one embodiment, the processors (120) can operate individually and / or collectively.

[0137] According to one embodiment, the processor (120) may be an application processor (AP). According to one embodiment, the processor (120) may be a system semiconductor that is responsible for the operation and multimedia driving functions of the electronic device (101). According to one embodiment, the processor (120) may be configured in the form of a system-on-chip (SoC), and may include a technology-intensive semiconductor chip that integrates multiple semiconductor technologies into one and implements system blocks into a single chip.

[0138] According to one embodiment, the system blocks of the processor (120) may include a graphics processing unit (GPU) (410), an image signal processor (ISP) (420), a central processing unit (CPU) (430), a neural processing unit (NPU) (440), a digital signal processor (DSP) (450), a modem (460), a connectivity (470), and / or a security (480) block, as illustrated in FIG. 3.

[0139] In one embodiment, the GPU (410) may be responsible for graphics processing. In one embodiment, the GPU (410) may receive commands from the CPU (430) and perform graphics processing to express the shape, position, color, shading, movement, and / or texture of objects (or objects) on the display.

[0140] In one embodiment, the ISP (420) may be responsible for image processing and correction of images and videos. In one embodiment, the ISP (420) may correct raw data (e.g., raw data) transmitted from an image sensor of a camera (e.g., the camera module (180) of FIG. 1) to generate an image in a form more preferred by the user. In one embodiment, the ISP (420) may perform post-processing, such as adjusting partial brightness of an image and emphasizing detailed parts. For example, the ISP (420) may independently perform a process of tuning and correcting the image quality of an image acquired through a camera to generate a result preferred by the user.

[0141] According to one embodiment, the ISP (420) may support artificial intelligence (AI)-based image processing technology. According to one embodiment, the ISP (420) may support scene segmentation (e.g., image segmentation) technology that recognizes and / or classifies parts of a scene being captured in conjunction with the NPU (440). For example, the ISP (420) may include a function that applies different parameters to objects such as the sky, bushes, and / or skin and processes them. According to one embodiment, the ISP (420) may detect and display a human face during image capture using the AI ​​function, or adjust the brightness, focus, and / or color of the image using the coordinates and information of the face.

[0142] According to one embodiment, the CPU (430) can perform operations corresponding to the processor (120). According to one embodiment, the CPU (430) can decode a user's command, perform arithmetic and logical operations, and / or data processing operations. For example, the CPU (430) can be responsible for functions such as memory, interpretation, calculation, and control. According to one embodiment, the CPU (430) can control the overall function of the electronic device (101). For example, the CPU (430) can execute all software (e.g., application (146) of FIG. 1) of the electronic device (101) on an operating system (OS) and control hardware devices. According to one embodiment, the CPU (430) can execute an application and control the overall operation of the processor (120) to perform neural network-based tasks required according to the execution of the application.

[0143] According to one embodiment, the CPU (430) may store instructions or data in volatile memory (e.g., volatile memory (132) of FIG. 1) of the memory (130), process the instructions or data stored in the volatile memory, and store resultant data in nonvolatile memory (e.g., nonvolatile memory (134) of FIG. 1) of the memory (130) as at least part of data processing or calculation.

[0144] According to one embodiment, the CPU (430) may include a single processor core or multiple processor cores (multi-core). According to one embodiment, the CPU (430) may be a programmable processor that stores executable instructions (e.g., instructions capable of performing operations of the CPU (430)) and executes the instructions.

[0145] According to one embodiment, the CPU (430) can operate in a multi-domain environment. According to one embodiment, the CPU (430) can operate in a multi-domain environment including a domain of the normal world (e.g., a non-secure world, a framework, or a non-secure environment) and a domain of the secure world (e.g., a secure framework or a secure environment). In one embodiment, a domain of the secure world can include one or more domains (e.g., a trusted OS, a trust zone, and / or a virtualization framework).

[0146] According to one embodiment, the NPU (440) can perform processing optimized for artificial intelligence deep-learning algorithms. According to one embodiment, the NPU (440) is a processor optimized for deep-learning algorithm operations (e.g., artificial intelligence operations) and can process big data quickly and efficiently like a human neural network. For example, the NPU (440) can be mainly used for artificial intelligence operations. According to one embodiment, the NPU (440) can recognize objects, environments, and / or people in the background when taking a video through a camera and automatically adjust the focus, automatically switch the shooting mode of the camera module (180) to food mode when taking a picture of food, and / or perform processing to erase only unnecessary subjects from the captured results. According to one embodiment, the NPU (440) can perform processing to generate (e.g., regenerate or reconstruct) an image based on given information (e.g., an image and / or a prompt).

[0147] According to one embodiment, the electronic device (101) can support integrated machine learning processing by interacting with all processors such as the GPU (410), the ISP (420), the CPU (430), and the NPU (440).

[0148] In one embodiment, the DSP (450) may represent an integrated circuit that facilitates rapid processing of digital signals. In one embodiment, the DSP (450) may perform the function of converting analog signals into digital signals and performing high-speed processing.

[0149] According to one embodiment, the modem (460) may perform operations that enable the use of various communication functions in the electronic device (101). For example, the modem (460) may support communications such as telephone and data transmission and reception by exchanging signals with a base station. According to one embodiment, the modem (460) may include an integrated modem (e.g., a cellular modem, an LTE modem, a 5G modem, a 5G-Advanced modem, and a 6G modem) that supports communication technologies such as long term evolution (LTE) and 2G to 5G. According to one embodiment, the modem (460) may include an artificial intelligence modem that applies an artificial intelligence algorithm.

[0150] In one embodiment, the connectivity (470) may support wireless data transmission based on IEEE 802.11. In one embodiment, the connectivity (470) may support communication services based on IEEE 802.11 (e.g., Wi-Fi) and / or 802.15 (e.g., Bluetooth, ZigBee, UWB). For example, the connectivity (470) may support communication services targeting an unspecified number of people in a localized area, such as indoors, using an unlicensed band.

[0151] According to one embodiment, security (480) can provide an independent security execution environment between data or services stored in the electronic device (101). According to one embodiment, security (480) can perform an operation to prevent external hacking through software and hardware security during the process of user authentication when providing services such as biometrics, mobile identification, and / or payment of the electronic device (101). For example, security (480) can provide an independent security execution environment in device security for reinforcing the security of the electronic device (101) itself and in security services based on user information such as mobile identification, payment, and car keys in the electronic device (101).

[0152] According to one embodiment, the operations performed by the processor (120) may be implemented by executing instructions stored in a recording medium (or a computer program product or storage medium). For example, the recording medium may include a non-transitory computer-readable recording medium having recorded thereon a program for executing various operations performed by the processor (120).

[0153] The embodiments described in the present disclosure may be implemented in a computer-readable recording medium using software, hardware, or a combination thereof. In a hardware implementation, the operations described in one embodiment may be implemented using at least one of application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, and / or other electrical units for performing functions.

[0154] In one embodiment, a computer-readable recording medium (or computer program product) is provided, which records a program for causing an electronic device to perform (or execute) various operations.

[0155] The above operations include: an operation of displaying an image on a display; an operation of receiving a first input for selecting a portion corresponding to an object in the image; an operation of cropping a portion corresponding to the object in the image based on the first input; an operation of detecting a second input for moving the cropped portion from an original location in the image; an operation of setting and displaying an area corresponding to the original location of the cropped portion in the image as a masking area based on the movement of the cropped portion; an operation of receiving a third input for selecting either the cropped portion or the masking area; an operation of providing a first handler related to editing the cropped portion through the cropped portion when the third input corresponds to selecting the cropped portion; an operation of providing a second handler related to editing the cropped portion through the masking area when the third input corresponds to selecting the masking area; an operation of editing the image in response to a fourth input received based on the first handler or the second handler; an operation of inpainting and / or based on the edited image based on a fifth input. The operation may include generating an instruction (or prompt) to perform outpainting, obtaining a result image in relation to the instruction, and displaying the result image via a display.

[0156] The above operations may include: displaying an image on a display; detecting a user input for performing an edit of an image being displayed on the display; determining a type of image edit based on the user input; performing a first edit based on at least one object based on the user input based on determining that the image edit is a first edit; performing a second edit based on the image based on the user input based on determining that the image edit is a second edit; detecting an image generation request; performing an image generation process based on an image edited by the first edit and / or the second edit based on the image generation request; and displaying an image generated (e.g., regenerated or reprocessed) based on the image generation process on the display.

[0157] The above operations may include an operation of displaying an image on a display, an operation of detecting a first input for selecting at least one object in the image, an operation of separating at least one object from the image based on the first input, an operation of setting an area corresponding to at least one object in the image as a masking area and displaying the area, an operation of setting a link between at least one object and the masking area, an operation of detecting a second input based on at least one object or the masking area, an operation of editing at least one object based on the second input, an operation of performing an image generation process based on a generative AI based on a third input, an operation of obtaining a result image in which at least one object and the masking area are reconstructed based on the image generation process, and an operation of displaying the result image through the display.

[0158] An electronic device (e.g., an electronic device (101, 201) of FIGS. 1 to 3) according to one embodiment of the present disclosure may include a display (e.g., a display module (160) of FIG. 1 or a display (490) of FIG. 3), communication circuitry (e.g., a communication module (190) of FIG. 1, a communication interface (210) of FIG. 2 or a communication circuit (495) of FIG. 3), at least one processor including processing circuitry (e.g., a processor (120, 230) of FIGS. 1 to 3), and a memory (e.g., a memory (130, 240) of FIGS. 1 to 3)) that stores instructions. In one embodiment, the memory may store instructions that, when individually and / or collectively executed by at least one processor, cause the electronic device (101) to perform operations.

[0159] In one embodiment, the instructions, when executed individually or collectively by at least one processor, may cause the electronic device to display an image on the display. The instructions, when executed individually or collectively by at least one processor, may cause the electronic device to receive a first input selecting a portion of the image corresponding to an object. The instructions, when executed individually or collectively by at least one processor, may cause the electronic device to crop a portion of the image corresponding to the object based on the first input. The instructions, when executed individually or collectively by at least one processor, may cause the electronic device to detect a second input moving the cropped portion from an original location within the image. The instructions, when executed individually or collectively by at least one processor, may cause the electronic device to display an area corresponding to an original position of the cropped portion in the image as a masking area based on movement of the cropped portion. The instructions, when executed individually or collectively by at least one processor, may cause the electronic device to receive a third input selecting one of the cropped portion or the masking area. The instructions, when executed individually or collectively by at least one processor, may cause the electronic device to provide a first handler associated with editing of the cropped portion through the cropped portion if the third input corresponds to selecting the cropped portion.The instructions, when individually or collectively executed by at least one processor, may cause the electronic device to provide a second handler associated with editing the cropped portion through the masking area, if the third input corresponds to selecting the masking area. The instructions, when individually or collectively executed by at least one processor, may cause the electronic device to edit the image in response to a fourth input received based on the first handler or the second handler. The instructions, when individually or collectively executed by at least one processor, may cause the electronic device to generate an instruction that causes the electronic device to perform inpainting and / or outpainting based on the edited image, based on a fifth input. The instructions, when individually or collectively executed by at least one processor, may cause the electronic device to obtain a result image in relation to the instructions. The above instructions, when executed individually or collectively by at least one processor, may cause the electronic device to display the resulting image through the display.

[0160] According to one embodiment, the instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to establish a link between the cropped portion and the masking area.

[0161] According to one embodiment, the resulting image may include an image reconstructed by the inpainting and / or the outpainting performed based on the masking area in the edited image.

[0162] According to one embodiment, the resulting image may include an image reconstructed by the inpainting and / or the outpainting performed based on the cropped area in the edited image.

[0163] According to one embodiment, the resulting image may include an image reconstructed by the inpainting and / or the outpainting performed based on the cropped area and the masked area in the edited image.

[0164] According to one embodiment, the instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to highlight the masking area corresponding to the cropped portion when the third input is detected through the cropped portion.

[0165] According to one embodiment, the instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to recognize a link established between the cropped portion and the masking area based on an input selecting the masking area. The instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to determine a cropped portion corresponding to the masking area based on the link.

[0166] According to one embodiment, the instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to select and display a cropped portion corresponding to the masked area if the cropped portion corresponding to the masked area exists on the image.

[0167] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to provide a first handler based on the cropped portion, if a cropped portion corresponding to the masked area exists on the image. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to provide a second handler based on the masked area, if a cropped portion corresponding to the masked area is removed from the image.

[0168] According to one embodiment, the instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to restore and display a cropped portion corresponding to the masking area in the masking area when an input specified through the second handler is detected.

[0169] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to recognize an area corresponding to the cropped portion in the image when cropping the portion corresponding to the object. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to set the masking area based on the recognized area. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to mask the masking area with a grid pattern.

[0170] According to one embodiment, the instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to cause the cropped portion and the masked area to inter-operate based on a link between the cropped portion and the masked area.

[0171] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to determine an object adjacent to the first cropped portion based on a specified input. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to crop a portion of the image corresponding to the adjacent object. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to set the cropped portions as a single group object.

[0172] According to one embodiment, the instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to move a cropped portion corresponding to the adjacent object into the handler based on an extension of the handler provided through the first cropped portion.

[0173] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to determine, based on a specified input, an object adjacent to a masking area corresponding to the first cropped portion. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to crop a portion of the image corresponding to the adjacent object. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to set the cropped portions as a single group object.

[0174] According to one embodiment, the instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to provide a function of comparing the image with the resulting image when providing the resulting image.

[0175] Hereinafter, an operating method of an electronic device (e.g., electronic devices 101 and 201 of FIGS. 1 to 3) (hereinafter, electronic device 101) according to various embodiments will be described in detail. Operations performed in the electronic device (101) according to various embodiments may be executed by a processor (e.g., processor 120 and 230 of FIGS. 1 to 3) including various processing circuitry and / or executable program elements of the electronic device (101). According to one embodiment, the operations performed in the electronic device (101) may be stored as instructions in a memory (130) and individually and / or collectively executed by the processor (120 and 230).

[0176] FIG. 4 is a flowchart illustrating a method of operating an electronic device according to one embodiment of the present disclosure.

[0177] According to one embodiment, FIG. 4 may illustrate an example of a method for supporting image editing and / or creation (e.g., regeneration or reprocessing) in an electronic device (101) according to one embodiment.

[0178] A method for supporting image editing and / or creation in an electronic device (101) according to one embodiment of the present disclosure may be performed, for example, according to a flowchart illustrated in FIG. 4. The flowchart illustrated in FIG. 4 is an example according to one embodiment of the operation of the electronic device (101), and the order of at least some operations may be changed or performed in parallel, performed as independent operations, or at least some other operations may be performed complementarily to at least some operations. According to one embodiment of the present disclosure, operations 401 to 417 may be performed in at least one processor of the electronic device (101) (e.g., processors 120 and 230 of FIGS. 1 to 3 ).

[0179] As illustrated in FIG. 4, an operation method performed by an electronic device (101) according to one embodiment may include an operation of displaying an image (401), an operation of detecting a user input for executing editing of an image (403), an operation of determining a type of image editing (405), an operation of performing a first editing based on at least one object based on the user input based on determining a first editing (407), an operation of performing a second editing based on the image based on the user input based on determining a second editing (409), an operation of detecting an image generation request based on an image edited by the first editing and / or the second editing (411), an operation of performing an image generation process (413), an operation of completing image generation (415), and an operation of displaying the generated image (417).

[0180] Referring to FIG. 4, in operation 401, the processor (120) of the electronic device (101) may display an image. According to one embodiment, the processor (120) may receive an input related to the execution of an application (e.g., a gallery application, a photography application, and / or an image editing application) from a user. According to one embodiment, the processor (120) may execute the application and display the execution screen of the application on the display in response to the input related to the execution of the application. According to one embodiment, the user may perform an operation of selecting and displaying an image to be edited while the execution screen of the application is displayed. According to one embodiment, the processor (120) may control the display to display the image selected by the user on the execution screen of the application.

[0181] In operation 403, the processor (120) may detect (e.g., receive) a user input for performing editing of an image. In one embodiment, the processor (120) may receive a designated user input for editing the displayed image while the image is displayed. In one embodiment, the designated user input may include a first input that is directly input, such as a designated touch gesture on the image, and / or a second input that is input based on an image editing tool.

[0182] In operation 405, the processor (120) may determine the type of image editing based on user input. In one embodiment, the type of image editing may include, but is not limited to, first editing and second editing, and may include various image editing methods. For example, the image editing may include various image editing methods, and the processor (120) may determine the type of image editing based on the user request and process the determined editing method.

[0183] In one embodiment, a first edit (or a first editing operation) of an image may include an operation that supports cropping one or more objects within the image, editing using the cropped objects, and generating an image based on the edited objects (e.g., inpainting and / or outpainting). For example, the first edit may include an editing operation that changes the position, size, tilt, angle, horizontality, and / or verticality of a cropped object (e.g., a subject within the image) based on user input, or that removes a cropped object. In one embodiment, the first edit may include an editing operation that adds a new object within the image. In one embodiment, the first edit may include an editing operation that restores an object removed from the image. According to one embodiment, a user may crop at least one object within the image to separate it from the image, move the cropped object to a desired location, resize the cropped object, and / or adjust the tilt of the cropped object to create a composition of the photograph desired by the user.

[0184] In one embodiment, the second editing (or second editing operation) of the image may include an operation that supports extending a background (or background area) to the outside of the image and creating an image (e.g., inpainting and / or outpainting) based on the extended area to the outside of the image. For example, the second editing may include an editing operation that rotates the image (e.g., changes its tilt) based on user input and, in response to the rotation of the image, extends an area for the background to the outside of the image. In one embodiment, the second editing may include an editing operation that creates new objects and / or backgrounds (e.g., inpainting and / or outpainting) associated with objects and / or backgrounds at the edge of the image through the extended area. According to one embodiment, the user may rotate the image left or right by a designated input (e.g., two-finger touch-based rotation) on the image, or by scrolling (or dragging) an editing tool (e.g., an image rotation tool or a dialer) left or right to rotate the image left or right, thereby creating a desired composition for the photo. For example, the second edit may include an edit that generates an image by background expansion of the image, and may support background expansion based on various specified inputs related to background expansion, such as inputs related to changing the tilt of the image (e.g., image rotation).

[0185] According to one embodiment, a user may selectively (or individually) and / or sequentially perform a first edit and / or a second edit with respect to an image. According to one embodiment, the processor (120) may determine, based on the user input, whether the user input is an input related to a first edit of the image (e.g., a first touch gesture and / or an input related to selecting a first edit tool related to performing the first edit) or whether the user input is an input related to a second edit of the image (e.g., a second touch gesture and / or an input related to selecting a second edit tool related to performing the second edit). According to one embodiment, examples of performing the first edit and / or the second edit and performing image editing based thereon are described below.

[0186] In operation 407, the processor (120) may perform a first edit based on at least one object based on user input, based on determining the first edit. According to one embodiment, an example of performing image editing based on the first edit is described below.

[0187] In operation 409, the processor (120) may perform a second image-based editing based on the user input, based on determining the second editing. An example of performing image editing based on the second editing is described below, according to one embodiment.

[0188] In operation 411, the processor (120) may detect a request for image generation based on an image edited by the first editing and / or the second editing. According to one embodiment, the user may edit at least one object and / or image in the image based on the first editing and / or the second editing, and complete the image editing. In one embodiment, the completion of the image editing may be performed by a designated command input. For example, the user may request the completion of the image editing (or the image generation) based at least on a designated object (e.g., a software button provided on a user interface including an image), a designated voice command, and / or a designated motion gesture (e.g., shaking the electronic device (101)) for the completion of the image editing (or the image generation (or re-generation)). According to one embodiment, the processor (120) may determine the entry of an operation for generating a new image based on the edited image based on detecting a designated command input while performing the image editing based on the interaction with the user.

[0189] In operation 413, the processor (120) may perform an image generation process. According to one embodiment, the processor (120) may perform an image generation process related to generating a new image based on an image edited by the first editing and / or the second editing, based on detection of an image generation request.

[0190] In one embodiment, the image generation process may include an operation to perform inpainting and / or outpainting based on the edited image. For example, the processor (120) may generate instructions (or prompts) to perform inpainting and / or outpainting based on the edited image. In one embodiment, the processor (120) may provide the generative AI with instructions to execute the image generation process based on the generated instructions. In one embodiment, the generated instructions may be provided to the generative AI on-device and / or to the generative AI on a server.

[0191] In one embodiment, the image generation process may include generating (e.g., regenerating or reprocessing) an edited image into a new image based on artificial intelligence (AI) (e.g., generative AI). In one embodiment, the image generation process may be provided on-device and / or server-based. An example of performing the image generation process is described below, according to one embodiment.

[0192] In operation 415, the processor (120) may complete image generation. In one embodiment, the processor (120) may complete image generation based on completion of the image generation process. For example, the processor (120) may obtain a result image (or a new image) in relation to an instruction (or prompt). In one embodiment, the processor (120) may obtain a result image based on an image generation process (e.g., inpainting and / or outpainting based on an edited image) executed in relation to an instruction (or prompt) in an on-device AI. In one embodiment, the processor (120) may obtain (or receive) a result image generated based on an image generation process (e.g., inpainting and / or outpainting based on an edited image) executed in relation to an instruction (or prompt) in a server AI from a server. In one embodiment, the processor (120) may determine completion of image generation based on the generation and / or acquisition of the result image.

[0193] In operation 417, the processor (120) may display the acquired image (e.g., the result image). In one embodiment, the processor (120) may control the display to display the acquired result image in relation to the instruction (or prompt). In one embodiment, the processor (120) may display the acquired image on the execution screen through the display. In one embodiment, the processor (120) may control the display to display the acquired image (e.g., the result image) by replacing the previously displayed image (e.g., the original image or the edited image).

[0194] FIGS. 5 to 26 are drawings illustrating a user interface that supports image editing in an electronic device according to one embodiment and an example of an operation of image editing using the same.

[0195] According to one embodiment, FIGS. 5 to 26 may illustrate examples of operations that support image editing based on artificial intelligence in an electronic device (101) according to one embodiment. According to one embodiment, the artificial intelligence may include generative AI. Generative AI may refer to an AI technology that newly creates similar content using existing content such as text, audio, and / or images. For example, generative AI may refer to an AI technology that can generate content (e.g., text, audio, images, and / or videos) corresponding to an input based on a given input. According to one embodiment, the electronic device (101) may generate (e.g., reconstruct images) and provide images and / or objects corresponding to a user's image editing based on generative AI (e.g., on-device AI). According to one embodiment, the electronic device (101) may request generation (e.g., reconstruct images) corresponding to image editing from a server, and receive and provide an image generated based on the server's generative AI. According to one embodiment, the electronic device (101) may provide the generative AI with a target image requesting image generation (e.g., an edited image) and / or a prompt (or generative AI prompt) related to image generation (e.g., a question or instruction to be entered into the generative AI).

[0196] In one embodiment, image editing may include various edits related to transforming (or adjusting) an image, such as tilting, scaling, horizontally adjusting, vertically adjusting, isolating an object, moving an object, removing an object, and / or adding an object of the image and / or at least one object within the image.

[0197] According to one embodiment, as illustrated in FIGS. 5 to 26, the method of operation performed by the electronic device (101) according to one embodiment may include an operation of editing an image based on interaction with a user, and providing the image by regenerating (or reconstructing) the image on a server or on-device based on the edited image. According to one embodiment, the operation of editing the image may include an operation related to a first editing and an operation related to a second editing.

[0198] In one embodiment, a first edit (or a first editing operation) of an image may include an operation that supports cropping one or more objects within the image, editing using the cropped objects, and generating an image based on the edited objects (e.g., inpainting and / or outpainting). For example, the first edit may include an editing operation that changes the position, size, tilt, angle, horizontality, and / or verticality of a cropped object (e.g., a subject within the image) based on user input, or that removes a cropped object. In one embodiment, the first edit may include an editing operation that adds a new object within the image. In one embodiment, the first edit may include an editing operation that restores an object removed from the image. According to one embodiment, a user may crop at least one object within the image to separate it from the image, move the cropped object to a desired location, resize the cropped object, and / or adjust the tilt of the cropped object to create a composition of the photograph desired by the user.

[0199] In one embodiment, the second editing (or second editing operation) of the image may include an operation that supports extending a background (or background area) to the outside of the image and creating an image (e.g., inpainting and / or outpainting) based on the extended area to the outside of the image. For example, the second editing may include an editing operation that rotates the image (e.g., changes its tilt) based on user input and, in response to the rotation of the image, extends an area for the background to the outside of the image. In one embodiment, the second editing may include an editing operation that creates new objects and / or backgrounds (e.g., inpainting and / or outpainting) associated with objects and / or backgrounds at the edge of the image through the extended area. According to one embodiment, the user may rotate the image left or right by a designated input (e.g., two-finger touch-based rotation) on the image, or by scrolling (or dragging) an editing tool (e.g., an image rotation tool or a dialer) left or right to rotate the image left or right, thereby creating a desired composition for the photo. For example, the second edit may include an edit that generates an image by background expansion of the image, and may support background expansion based on various specified inputs related to background expansion, such as inputs related to changing the tilt of the image (e.g., image rotation).

[0200] FIG. 5 may illustrate an example of a user interface that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0201] In one embodiment, a user may use an electronic device (101) to run an application (e.g., a gallery application or an image editing application) and select an image to be edited. In one embodiment, the electronic device (101) may display a user interface (500) including the image (501) selected by the user on the display.

[0202] According to one embodiment, the user interface (500) may include a first area (510) in which an image (501) is displayed, a second area (520) in which a first object (e.g., an indicator, an editing tool, an icon, or a menu) for selecting an editing type (e.g., auto-correction, adjustment, filter, tone, and / or deco) of the image (501) is provided, a third area (530) in which a second object (e.g., an indicator editing tool, an icon, or a menu) related to editing the image (501) is provided, a fourth area (540) in which a third object (e.g., an indicator, an editing tool, an icon, or a menu) related to editing the image (501) is provided, a fifth area (550) in which a fourth object (or a dialer) related to editing the image (501) is provided, and / or a sixth area (560) in which a fifth object (or icon) related to executing artificial intelligence-based automatic editing is provided.

[0203] According to one embodiment, the second object of the third area (530) and / or the third object of the fourth area (540) may be provided with objects of different editing functions based on the type of editing according to the first object selected in the second area (520). For example, in the example of FIG. 5, the type of editing may be related to adjusting an image (501), and the second object and the third object may be provided with objects of menus (or editing functions) related to adjusting the image (501). For example, the second object may include objects related to tilting, horizontally, and / or vertically adjusting the image (501). For example, the third object may include objects related to left / right flipping, rotating, and / or scaling of the image (501).

[0204] In one embodiment, the fourth object of the fifth area (550) may include an object that supports scroll control of a menu (or editing function) related to an object selected in the third area (530). For example, when an object related to tilt control is selected in the third area (530), the fourth object may support adjusting the tilt angle of the image (501). For example, when an object related to horizontal or vertical control is selected in the third area (530), the fourth object may support adjusting the horizontal or vertical angle of the image (501).

[0205] In one embodiment, various objects in the first region (510), the second region (520), the third region (530), the fourth region (540), the fifth region (550), and / or the sixth region (560) may be replaced with information corresponding to the operation being performed (e.g., a graphic object such as an icon) while performing image editing.

[0206] For example, while an image (501) is displayed through the first region (510), if an object (e.g., an editing function or menu) is selected through any one of the second region (520) to the sixth region (560), the selected object may be switched to and provided as another object (e.g., a menu). For example, the fourth region (540) may provide a guide related to image editing performed by the user. For example, the second region (520) may provide a bottom menu related to image editing. For example, the third region (530) may provide information on the status of the edited image (e.g., information on the rotation status (e.g., rotation angle)) according to the image editing. For example, the second area (520) to the sixth area (560) may provide an object (e.g., a creation object (or button)) for requesting image creation (e.g., regeneration (or reconstruction) of the image (501)) based on the edited image according to the user's image editing performance (e.g., completion of image editing).

[0207] According to one embodiment, a user may have a need to change the composition of a photo to a desired composition when taking a photo or when storing an image in an electronic device (101) (e.g., an image taken by the electronic device (101) or an image downloaded from an external source). According to the present disclosure, the electronic device (101) supports freely changing the composition of an image (e.g., a taken photo) (e.g., adding / deleting an object in the image (501) and / or changing the tilt, horizontality and / or verticality of the image (501) or an object), thereby satisfying the needs of a user who wants to change the composition of a poorly taken image or an image whose composition the user wants to change, and providing a new user experience (UX) to the user.

[0208] In one embodiment, a user may perform a designated input to enter an editing mode (e.g., an AI mode or an AI editing mode) to support image editing while an image (501) is displayed. In one embodiment, the designated input may include a preset touch gesture input, voice command input, motion gesture input, or editing tool selection input to enter the editing mode.

[0209] According to one embodiment, the electronic device (101) may enter an edit mode based on a specified input detection while the image (501) is displayed. According to one embodiment, based on entering the edit mode, the electronic device (101) may display a user interface including the image (501) and various objects related to editing the image (501) (e.g., objects corresponding to indicators, icons, and / or text) on the display.

[0210] According to one embodiment, the user may enter an edit mode based on a designated input while an image (501) is displayed, as in the example of FIG. 5, and may provide an interface corresponding to a first edit operation or a second edit operation in the edit mode, respectively. According to one embodiment, the user may select an object through a designated input for selecting an object in the image (501) and perform a first edit operation for performing image editing based on the selected object. According to one embodiment, the user may perform a second edit operation for performing image editing based on background area expansion through a designated input for rotating the image (501) while the image (501) is displayed in the edit mode, or after performing the first edit operation, and subsequently to the first edit operation. According to one embodiment, the user may also perform the first edit operation after performing the second edit operation in the edit mode and subsequently to the second edit operation.

[0211] FIGS. 6 to 16 are drawings illustrating a user interface that supports image editing in an electronic device according to one embodiment of the present disclosure and an example of an operation of image editing using the same.

[0212] According to one embodiment, FIGS. 6 to 16 are diagrams illustrating examples of operations related to image editing in an electronic device (101). According to one embodiment, FIGS. 6 to 16 may illustrate examples of operations related to a first edit among image editing operations according to the present disclosure. In one embodiment, the first edit (or first edit operation) of an image may include operations that support cropping one or more objects within an image, editing using the cropped objects, and generating an image based on the edited objects (e.g., inpainting and / or outpainting).

[0213] According to one embodiment, as illustrated in FIGS. 6 to 16, the electronic device (101) may perform an editing operation (e.g., a first edit) related to at least one object within a displayed image (601) based on a user input (e.g., a first input on an image or a second input based on an editing tool). For example, as illustrated in FIGS. 6 to 16, the user may crop (or separate) at least one object within the image (601), move and / or remove the cropped object, and / or add a new object to create a new image (e.g., regenerate or reconstruct the image (601)).

[0214] In one embodiment, the image (601) in the examples of FIGS. 6 to 16 may be an image (e.g., an edited image having a background area expanded based on image rotation) on which an editing operation (e.g., a second editing) based on background expansion of the image (601) as illustrated in FIGS. 22 to 26 described below has been performed, and may represent an example of performing an editing operation (e.g., a first editing) related to at least one object in the edited image (e.g., a rotated image). The present disclosure is not limited thereto, and the first editing may be performed based on the original image (501) before the second editing (e.g., background area expansion based on image rotation) is performed, as in the example of FIG. 5. In one embodiment, the second editing may be performed continuously on the image edited according to the first editing.

[0215] According to one embodiment, the first edit (or first edit operation) of the image (601) may include the ability to adjust (e.g., edit) one or more objects (e.g., object (610), object (620)) within the image (601). For example, the electronic device (101) may support cropping (or isolating) an object (e.g., a subject within the image) within the image (601), changing the position, size, tilt, angle, horizontality, and / or verticality of the cropped object, removing the cropped object within the image (601), or adding a new object within the image (601) based on user interaction. For example, the electronic device (101) may support the user to create a desired composition of a photo by cropping (or separating) at least one object (610, 620) in an image (601) based on a user input, moving the object to a desired position, adjusting the size of the at least one object (610, 620), and / or adjusting the tilt / horizontal / vertical orientation of the at least one object (610, 620).

[0216] According to one embodiment, FIG. 6 may illustrate an example of a state in which an electronic device (101) displays an image (601) in an edit mode. For example, as discussed in the example of FIG. 5, the electronic device (101) may enter an edit mode based on a designated input for supporting image editing while the image (501) is displayed. According to one embodiment, the electronic device (101) may, in the edit mode, provide a guide object (603) (e.g., guide text) that guides a user on a method for editing an image. According to one embodiment, the electronic device (101) may, in the edit mode, provide an editing tool (605) (e.g., a dialer) that supports editing of the image (601).

[0217] According to one embodiment, the electronic device (101) may provide an object (600) (e.g., a creation object (or button)) for requesting image generation (e.g., regeneration (or reconstruction) of the image (601)) based on an image edited by a user in an edit mode (e.g., completion of image editing). For example, the object (600) for executing image generation may be provided based on an area corresponding to the second area (520) of the user interface (500) of FIG. 5. According to one embodiment, the object (600) for executing image generation may be displayed while image editing is being performed (or while entering the edit mode (or AI mode)) and may not be displayed when image editing is completed (e.g., acquisition of a result image based on completion of image generation).

[0218] According to one embodiment, a user may select at least one object within an image (601) through a user input while the image (601) is displayed, as illustrated in FIG. 6. For example, as illustrated in FIG. 7, a user may select (or designate) at least one target object (e.g., object (620)) for a first editing in the image (601). According to one embodiment, the electronic device (101) may select at least one object corresponding to the user input in the image (601) as a target object for image editing.

[0219] In one embodiment, the user input for selecting a target object may include an input based on at least one of various touch gesture-based inputs. For example, the user input may include a first gesture for tapping an object on the image (e.g., single tap, double tap, triple tap, or long press) as an input for designating selection of an object within the image (601), a second gesture for selecting an object in a square or circle shape on the image (601), a third gesture for more free and flexible selection of an object (e.g., drawing a closed curve, drawing a circle, drawing a lasso) by drawing a relatively circular shape without a designated shape (e.g., drawing a circle to contain the object within it), and / or a fourth gesture for selecting and moving an object within the image (601) (e.g., tap & move or long tap & move).

[0220] In one embodiment, the various user inputs related to image editing are not limited to input via the user's fingers, and may include input via various input devices (e.g., a stylus pen, an electronic pen, a mouse).

[0221] In one embodiment, when the electronic device (101) detects a user input (e.g., a first gesture, a second gesture, a third gesture, or a fourth gesture) for selecting an object (620) within an image (601), the electronic device (101) may determine the object (620) selected according to the user input as a target object for image editing. In one embodiment, the electronic device (101) may display an object area (625) based on (or centered on) the selected object (620). In one embodiment, the object area (625) may display a dotted line (e.g., a lasso selection) based on an outer edge (or boundary or boundary surface) of the object (620) to provide the user with a selected state of the object (620). According to one embodiment, the user may perform various editing operations, such as moving, deleting, copying, pasting, rotating (or adjusting an angle or a tilt), adding an object, horizontally adjusting, vertically adjusting, and / or resizing, based on the selected object (620).

[0222] In one embodiment, the selected object (620) can be separated (or cropped) from the image (601) (e.g., a background image) based on user input and edited separately (or independently) from the image (601). For example, the object (620) within the image (601) can be cropped by the user input to enable movement and / or editing. In one embodiment, the electronic device (101) can crop the object (620) to separate it from the image (601) based on the object area (625). For example, the electronic device (101) can separate the object (620) from a background image that includes other objects that are not selected in the image (601) based on a user input (e.g., a first user input) and edit the object (620) separately from the background image based on a user input (e.g., a second user input that is different from the first user input). According to one embodiment, editing operations using a cropped object (620) are illustrated in FIGS. 8 to 16.

[0223] According to one embodiment, as illustrated in FIG. 8, a user may move a cropped object (620) (e.g., a target object) in the example of FIG. 7. According to one embodiment, the electronic device (101) may move and display the cropped object (620) to a position corresponding to the user input based on a user input (e.g., a drag) based on the cropped object (620). According to one embodiment, the movement of the cropped object (620) may be performed simultaneously by selection and movement based on a fourth gesture (e.g., tap & move or long tap & move) according to the example of FIG. 7.

[0224] According to one embodiment, the electronic device (101) may display a masking area (630) based on the movement of the cropped object (620), by setting the area where the object (620) was originally located (e.g., the area where the object (620) was located before moving). For example, the electronic device (101) may display a masking (e.g., a grid pattern) on the area where the selected object was originally located. In one embodiment, the masking area (630) may be an area that indicates that an object (e.g., the object (620)) in the image (601) has been edited (e.g., moved, deleted (or removed)) at its original location (e.g., the masking area (630)) after the object (e.g., the object (620)) has been separated from the image (601). For example, the masking area (630) may be provided based on a specified graphic object (e.g., a grid pattern).

[0225] According to one embodiment, the electronic device (101) can establish a relationship between an object (620) (e.g., a cropped and moved object) and a masking area (630). In one embodiment, the relationship between the object (620) and the masking area (630) can represent a link (or connection) between the object (620) and the masking area (630) that allows the object (620) and the masking area (630) to inter-operate, for example, based on a user input based on the object (620) or the masking area (630). In one embodiment, the electronic device (101) can display a dotted line (e.g., a lasso selection) object area (625) based on an outer edge (or boundary or border) of the masking area (630) to provide the user with a state in which the masking area (630) is set.

[0226] According to one embodiment, the electronic device (101) may provide a handler (900) related to image editing, as illustrated in FIG. 9, based on at least one target object (e.g., object (620)) selected for editing in the example of FIG. 8. According to one embodiment, the user may perform at least one edit (e.g., editing such as resizing, adjusting an angle, and / or removing (or erasing)) related to the object (620) based on the handler (900) provided based on the object (620) (e.g., a cropped and moved object).

[0227] In one embodiment, the handler (900) may include a handler box (910) and at least one tool (920) (e.g., an editing tool or a function tool). In one embodiment, the handler box (910) may be provided as a rectangular box surrounding at least one target object (e.g., an object (620)). In one embodiment, the tool (920) may include objects (e.g., at least one function object (or tool object)) each corresponding to an option (or tool) related to editing of at least one target object (e.g., an object (620)). For example, the tool (920) may include a first function object (e.g., an undo object (or a revert object)) and a second function object (e.g., an eraser object or a trash can object).

[0228] In one embodiment, a first functional object (e.g., an undo (or revert) object) may include an object that supports a function of restoring (e.g., reverting) to a previous state. For example, an undo object may include an object that supports a function of canceling a previously executed editing command and reverting to a state prior to issuing the editing command (e.g., a state prior to execution) (e.g., a cancel function). In one embodiment, a second functional object (e.g., an eraser object or a trash can object) may include an object that supports a function of deleting (or removing) a target object.

[0229] In one embodiment, the tool (920) is not limited to the first function object and the second function object, and may include additional function objects depending on the settings of the electronic device (101) or may be provided based on at least one function object according to user definition. For example, the tool (920) may include a third function object (e.g., a redo object) that supports cancellation of an undo function, for example, a function that returns to a state in which a command was executed again, and a fourth function object (e.g., a reverse object) that supports reversal of an object.

[0230] In one embodiment, when a user wishes to restore an object (620) after editing the object (620) (e.g., to its original state (e.g., original position and / or original tilt)), the user may utilize a first functional object (e.g., an undo object). For example, the electronic device (101) may support returning the object (620) to its initial state (e.g., a state before image editing) based on a user input based on the first functional object.

[0231] According to one embodiment, FIGS. 10 to 12 may illustrate examples of an operation in which a user additionally selects (or designates) at least one target object (e.g., object (610)) for editing in an image (601). For example, based on FIGS. 7 to 9 , the user may perform an operation of selecting an additional object (610) for editing in the image (601) after editing an object (620).

[0232] According to one embodiment, the electronic device (101) may select an object (610) in the image (601) as a target object for editing based on a user input. In one embodiment, the user input for selecting the target object may include an input corresponding to a first gesture to a fourth gesture. For example, the user input for selecting the target object may include an input for drawing a relatively circular shape (e.g., a lasso (1100)) around the object (610) on the image (601), as illustrated in FIG. 11 (e.g., a third gesture input for drawing a circle (or a closed curve) to include the object (610) within it). According to one embodiment, the user may select the object more freely and flexibly by drawing a circle around the object (610).

[0233] In one embodiment, when the electronic device (101) detects a user input for selecting an object (610) within an image (601), the electronic device (101) may determine the object (610) selected according to the user input as a target object for image editing. In one embodiment, the electronic device (101) may display an object area (615) based on (or centered on) the selected object (610). In one embodiment, the object area (615) may display a dotted line (e.g., lasso selection) based on an outer edge (or boundary or boundary surface) of the object (610) to provide the user with a selected state of the object (610). According to one embodiment, the user may perform various editing operations, such as moving, deleting, copying, pasting, rotating (or adjusting angle or tilt), adding an object, horizontally adjusting, vertically adjusting, and / or resizing, based on the selected object (610).

[0234] In one embodiment, the selected object (610) can be separated (or cropped) from the image (601) (e.g., a background image) based on user input and edited separately (or independently) from the image (601). For example, the object (610) within the image (601) can be cropped by user input to enable movement and / or editing. In one embodiment, the electronic device (101) can crop the object (610) to separate it from the image (601) based on the object area (615). For example, the electronic device (101) can separate the object (610) from a background image that includes other objects (e.g., including the edited object (620)) in the image (601) that are not selected based on the user input, and can edit the object (610) separately from the background image based on the user input.

[0235] According to one embodiment, as illustrated in FIG. 12, a user may move and / or edit an object (610) (e.g., a target object) selected in the example of FIG. 11 using a handler (900). For example, the electronic device (101) and the user may perform an editing operation on the object (610) corresponding to the operations described in the description with reference to FIGS. 8 and 9.

[0236] For example, the electronic device (101) may display an area where the object (610) was originally located (e.g., an area where the object (610) was located before moving) as a masking area based on the movement of the object (610). For example, the electronic device (101) may provide a handler (900) related to image editing based on the object (610) (e.g., an object that has been cropped and moved). For example, a user may perform at least one edit (e.g., an edit such as resizing, adjusting an angle, and / or removing (or erasing)) related to the object (610) based on the handler (900) provided based on the object (610) (e.g., an object that has been cropped and moved).

[0237] According to one embodiment, FIG. 13 may illustrate an example in which a user performs an edit to remove (or delete or erase) the selected object (610) in the example of FIG. 12. For example, the user may remove the object (610) using an eraser (or trash can) object of the handler (900). According to one embodiment, the electronic device (101) may not display the object (610) in the image (601) based on the removal of the selected object (610), and may provide the handler (900) based on a masking area (1300) corresponding to the removed object (610). For example, the electronic device (101) may display a masking (e.g., a checkered pattern) on the area where the object (610) was originally located (e.g., the masking area (1300)).

[0238] According to one embodiment, the electronic device (101) may establish a relationship between the object (610) and the masking area (1300). In one embodiment, the relationship between the object (610) and the masking area (1300) may represent a link (or connection) between the object (610) and the masking area (1300) that allows the object (610) and the masking area (1300) to inter-operate, for example, based on a user input based on the object (610) or the masking area (1300). In one embodiment, the electronic device (101) may display a dotted line (e.g., a lasso selection) based on the outer edge (or boundary or boundary surface) of the masking area (1300) and display a handler (900) for executing a function according to the link.

[0239] In one embodiment, the handler (900) may include an undo object that supports a function to restore (e.g., revert) to a previous state. For example, the undo object may include an object that supports a function (e.g., a cancel function) to cancel a previously executed editing command and return to a state before the editing command was issued (e.g., a state before execution). According to one embodiment, when a user wants to restore (e.g., restore to an original position) an object (610) after editing (e.g., removing) the object (610) from a background image, the user may use the handler (900) (e.g., an undo object) of the masking area (1300) to restore to an initial state (e.g., a state before image editing (e.g., removing)). For example, the electronic device (101) may, in response to a user input using the undo object, restore the object (610) based on the masking area (1300) and provide the state before image editing, as in the example of FIG. 10.

[0240] According to one embodiment, FIG. 14 may illustrate an example of an operation of additionally selecting (or designating) an object (1410) for editing in an image (601) while the user has selected an object (610) as in the example of FIG. 10 or FIG. 11. For example, the user may perform an operation of selecting an additional object (1410) for editing in an image (601) while having selected an object (610) based on FIG. 10 or FIG. 11.

[0241] According to one embodiment, when an object (610) is selected and a portion connected to a previously selected object area is selected (e.g., an object-based tap or circle drawing input), the electronic device (101) may select and provide an object consecutively with the previous selection. For example, as illustrated in FIG. 15, the electronic device (101) may connect a first object area of ​​a previously selected object (610) and a second object area of ​​an additionally selected object (1410) to provide a single object area (1500). For example, the object area (1500) may be formed based on an outer edge (or boundary or boundary surface) of a bundle object (or group object) including a previously selected object (610) and an additionally selected object (1410). For example, the electronic device (101) can link the first object area and the second object area to provide the previously selected object (610) and the additionally selected object (1410) as one target object for editing (e.g., a bundled object or a group object).

[0242] According to one embodiment, a user may perform various edits, such as moving, deleting, copying, pasting, rotating (or angling or tilting), adding objects, horizontally adjusting, vertically adjusting, and / or resizing, based on a bundle object (or group of objects) (e.g., multiple objects (610, 1410)). According to one embodiment, the edits may be performed by separating (or cropping) the bundle object from the image (601) (e.g., a background image) and editing the bundle object separately (or independently) from the image (601).

[0243] According to one embodiment, a user may edit a selected bundle object (1600) (e.g., a target object) in relation to editing in the example of FIG. 15 using a handler (900). For example, the electronic device (101) and the user may perform editing on the bundle object (1600) by corresponding operations described in the description with reference to FIGS. 8 and 9. For example, the electronic device (101) may display an area where the bundle object (1600) was originally located (e.g., an area where the bundle object (1600) was located before moving) as a masking area based on the movement of the bundle object (1610). For example, the electronic device (101) may provide a handler (900) related to image editing based on the bundle object (1600). For example, a user may perform at least one edit (e.g., resizing, angling, and / or removing (or erasing)) related to a bundle object (1600) based on a handler (900) provided based on the bundle object (1600).

[0244] FIGS. 17 and 18 are diagrams illustrating examples of a user interface during image generation in an electronic device according to one embodiment of the present disclosure.

[0245] According to one embodiment, FIGS. 17 and 18 may illustrate examples of a user interface and its operation during image generation (e.g., regeneration or reconstruction) (e.g., during an image generation process). For example, FIGS. 17 and 18 may illustrate examples of a user interface during an image generation process. According to one embodiment, the image generation process may include operations for performing inpainting and / or outpainting based on an edited image. For example, the electronic device (101) may generate instructions (or prompts) for performing inpainting and / or outpainting based on the edited image. According to one embodiment, the electronic device (101) may provide the generative artificial intelligence to execute the image generation process based on the generated instructions. According to one embodiment, the generated instructions may be provided to the generative artificial intelligence of the on-device and / or the generative artificial intelligence of the server.

[0246] According to one embodiment, a user may perform image editing on an image (601) based on performing at least one operation illustrated in FIGS. 6 to 16 and request image generation based on the edited image. In one embodiment, the image generation request may be performed by a designated command input. For example, the user may request image generation (or completion of image editing) based at least on a designated object for image generation (or re-generation) (e.g., a software button provided on a user interface including an image (e.g., the generation object (600) of FIG. 6), a designated voice command, and / or a designated motion gesture (e.g., shaking the electronic device (101)). According to one embodiment, the electronic device (101) may determine an operation entry for generating a new image based on the edited image based on detecting a designated command input while performing image editing based on interaction with the user.

[0247] According to one embodiment, a user may perform a user input by selecting a software button (e.g., a creation object (600)) provided on a user interface as in the examples of FIGS. 6 to 16. According to one embodiment, the creation object (600) may be provided through a designated area while image editing is performed (or while entering an editing mode (or AI mode)). According to one embodiment, the electronic device (101) may receive a user input based on the creation object (600) while performing image editing based on interaction with the user.

[0248] According to one embodiment, the electronic device (101) may perform a process (e.g., an image generation process) related to generating a new image based on an edited image in response to receiving a user input based on a creation object (600). According to one embodiment, the image generation process may include an operation of regenerating (or reprocessing) an image based on the edited image based on artificial intelligence (AI) (e.g., generative AI). In one embodiment, the image generation process may be provided on-device and / or server-based.

[0249] According to one embodiment, as illustrated in the examples of FIGS. 17 and / or 18, the electronic device (101) may provide a user with a relevant interface while the image generation process is in progress. For example, the relevant interface may include a guide object (1700) indicating that the image generation process is in progress. For example, the relevant interface may be in the form of dimming a background including an image and providing a guide object (1700) on the dimmed background.

[0250] In one embodiment, the guide object (1700) may be provided based on an indicator (e.g., the guide object (1700)) indicating the progress status of the image generation process. In one embodiment, the indicator (e.g., the guide object (1700)) may include an indicator (or an icon, an item, an identifier, or an object) (e.g., a moving icon (or an animated icon) whose state changes) and / or text (e.g., a progress guide phrase). In one embodiment, the text (e.g., a progress guide phrase) may be provided with various contents changed along with changes in the indicator while the process progresses, as in the examples of FIGS. 17 and 18.

[0251] For example, the electronic device (101) may provide guide phrases that change in accordance with changes in an indicator (e.g., a moving icon). For example, guide phrases may be provided that change over time, such as “Filling the background...” in FIG. 17 and “Drawing a new landscape...” in FIG. 18. In one embodiment, the guide phrases are not limited thereto and may include various phrases that are predefined or automatically generated based on artificial intelligence. For example, the guide phrases may be provided in various ways, such as “Adding a missing area...” and “Coloring carefully...”

[0252] FIGS. 19 and 20 are diagrams illustrating examples of providing a result image according to image editing in an electronic device according to one embodiment of the present disclosure.

[0253] According to one embodiment, FIGS. 19 and 20 may illustrate examples of a user interface and its operation provided upon completion of an image generation process. According to one embodiment, the electronic device (101) may complete an image generation operation based on completion of the image generation process and provide a related interface.

[0254] According to one embodiment, the electronic device (101) may acquire a new image (e.g., a result image) in relation to an instruction (or prompt). According to one embodiment, the electronic device (101) may acquire (e.g., generate) a new image (e.g., a result image) based on an image generation process (e.g., inpainting and / or outpainting based on an edited image) executed in relation to an instruction (or prompt) in an on-device AI. According to one embodiment, the electronic device (101) may acquire (or receive) an image (e.g., a result image) generated from a server based on an image generation process (e.g., inpainting and / or outpainting based on an edited image) executed in relation to an instruction (or prompt) in a server AI. According to one embodiment, the electronic device (101) may determine (or determine) the completion of the image generation process based on the generation and / or acquisition of the new image (e.g., the result image).

[0255] According to one embodiment, the electronic device (101) can display the generated image (1910). According to one embodiment, the electronic device (101) can display the generated image (1910) by replacing an image (e.g., an original image) that was previously displayed on the display. According to one embodiment, the generated image (1910) can be, for example, an image in which a blank area (e.g., a masking area) generated by editing an object and an object are reconstructed. For example, the masking area can be generated (e.g., inpainted) and the edited object can be included in (e.g., combined) a background image.

[0256] According to one embodiment, the electronic device (101) may provide a comparison function by switching between a generated image (1910) (e.g., a result image) and an original image (1920) (e.g., image (601)), as illustrated in FIGS. 19 and 20 . For example, as illustrated in FIGS. 19 and 20 , the user interface may include an object (e.g., a toggle object) that supports display switching between the generated image (1910) and the original image (1920). According to one embodiment, an object for display switching (e.g., a toggle object) may include a first toggle object (1930) for switching from a generated image (1910) to an original image (1920) (e.g., original view), as in the example of FIG. 19, and a second toggle object (1940) for switching from a converted original image (1920) to a generated image (1910) (e.g., modified view), as in the example of FIG. 20.

[0257] According to one embodiment, a result image (e.g., a generated image (1910)) may be provided with N or more result images (e.g., multiple result images). For example, the electronic device (101) may generate multiple result images based on generative artificial intelligence and provide a preview (e.g., a thumbnail) corresponding to each of the multiple result images as a bottom menu of the result image (e.g., the generated image (1910)). According to one embodiment, when multiple result images are provided, a user may sequentially switch between the multiple result images by sliding the preview (e.g., inputting a flick or swipe gesture). For example, the electronic device (101) may sequentially switch between and display the multiple result images in response to a user's input for switching the preview.

[0258] In one embodiment, the user may select a desired image from among multiple result images or perform an action to regenerate the image using the selected image. For example, if the user is not satisfied with the displayed result image (e.g., the generated image (1910)), the user may perform an additional image generation using the generate button.

[0259] According to one embodiment, as illustrated in FIGS. 19 and 20 , the user interface may include a completion object (1950) (or completion button). For example, a user may perform a user input to check the generated image (1910) and select the completion object (1950) to complete image editing. According to one embodiment, the electronic device (101) may determine the end of the image editing operation (e.g., edit mode or AI mode) in response to the user input using the completion object (1950).

[0260] FIG. 21 is a diagram illustrating an example of providing a result image according to image editing in an electronic device according to one embodiment of the present disclosure.

[0261] According to one embodiment, FIG. 21 may illustrate an example of providing a result image according to image editing in an electronic device according to one embodiment of the present disclosure. According to one embodiment, the electronic device (101) may terminate the editing operation (e.g., release (or deactivate) the editing mode) based on a determination to terminate the image editing operation, and may provide the generated image (1910) by including it in a user interface (e.g., an application execution screen).

[0262] According to one embodiment, the user interface may include a storage object (2110) that supports the function of storing the generated image (1910) in internal memory (e.g., memory of the electronic device (101)) and / or external memory (e.g., cloud) and an indicator (2120) (e.g., artificial intelligence object) that indicates that the displayed image is an image generated based on artificial intelligence. According to one embodiment, the user interface may include the generated image (1910), the storage object (2110), and / or the indicator (2120) in the user interface (500) as described in the description section with reference to FIG. 5 .

[0263] Various embodiments related to object-based image editing operations according to one embodiment are described with reference to the drawings described below.

[0264] FIGS. 22 to 26 are drawings illustrating examples of image editing operations in an electronic device according to one embodiment of the present disclosure.

[0265] According to one embodiment, FIGS. 22 to 26 are diagrams illustrating examples of operations related to image editing in an electronic device (101). According to one embodiment, FIGS. 22 to 26 may illustrate examples of operations related to second editing among image editing operations according to the present disclosure. In one embodiment, the second editing (or second editing operation) of an image may include an operation that supports extending a background (or background area) to the periphery of the image and generating an image (e.g., inpainting and / or outpainting) based on the area extended to the periphery of the image.

[0266] According to one embodiment, as illustrated in FIGS. 22 to 26, the electronic device (101) may perform an editing operation (e.g., a second editing operation) based on background expansion of a displayed image (2201) based on a user input (e.g., a first input on an image or a second input based on an editing tool). For example, as illustrated in FIGS. 22 to 26, the user may rotate the image (2201) left or right with a designated input (e.g., a two-finger touch-based rotation) on the image (2201), or may rotate the image (2201) left or right by scrolling (or dragging) an editing tool (2210) (e.g., an image rotation tool or a dialer) left or right.

[0267] According to one embodiment, the second edit (or second edit operation) of the image (2201) includes an edit that generates an image by background expansion of the image (2201) (e.g., regenerating or reconstructing the image (2201)), and may support background expansion based on various designated inputs related to background expansion (e.g., inputs related to tilt change of the image (2201)). For example, the electronic device (101) may perform an edit (e.g., tilt adjustment) on the image (2201) to generate a new image (e.g., regenerating or reconstructing the image (2201)), as illustrated in FIGS. 22 to 26 .

[0268] In one embodiment, the image (2201) in the examples of FIGS. 22 to 26 may be an original image (501) before editing is performed, for example, as illustrated in FIG. 5. For example, the examples illustrated in FIGS. 22 to 26 may represent examples of performing image editing based on background expansion of the original image (501). The present disclosure is not limited thereto, and the second editing may be performed based on an image edited according to the first editing corresponding to what was described in the description referring to FIGS. 6 to 16. For example, the second editing may be performed continuously on an image edited according to the first editing.

[0269] According to one embodiment, the second editing (or second editing operation) of the image (2201) may include, for example, a function of generating a background (e.g., object re-creation) around an object (e.g., a subject) attached to an edge of the image (2201) (e.g., a landscape photo, a portrait photo, a background photo, a photograph of an object, or a photograph of an animal), so that even if the tilt of the image (2201) changes, the object within the image (2201) can be positioned in a desired area within the image (2201), and providing a naturally extended background so that the image (2201) can be edited by the user at a desired angle and composition.

[0270] According to one embodiment, FIG. 22 may illustrate an example of a state in which an electronic device (101) displays an image (2201) in an edit mode. For example, as discussed in the example of FIG. 5 , the electronic device (101) may enter an edit mode based on a designated input for supporting image editing while the image (501) is displayed. According to one embodiment, the electronic device (101) may, in the edit mode, provide a guide object (e.g., guide text) that guides the user on a method for editing an image. According to one embodiment, the electronic device (101) may, in the edit mode, provide an editing tool (2210) (e.g., a dialer) that supports editing of the image (2201).

[0271] According to one embodiment, the electronic device (101) may, in an edit mode, provide an object (600) (e.g., a creation object (or button)) for requesting image creation (e.g., regeneration (or reconstruction) of the image (2201)) based on an image edited by a user (e.g., completion of image editing). According to one embodiment, the electronic device (101) may, in an edit mode, provide an editing tool (2210) (e.g., an image rotation tool or a dialer) that supports image editing. In one embodiment, the editing tool (2210) may provide a user input and, in response to image editing based on the user input, provide a real-time change state of the image (2201) (e.g., information on a rotation state (e.g., tilt or angle information)).

[0272] According to one embodiment, as illustrated in FIGS. 22 to 24, a user may input a designated touch gesture (e.g., a touch gesture for rotating the image (2201) left or right based on a two-finger touch) on an image (2201) or input a touch gesture for manipulating an editing tool (2210) (e.g., an image rotation tool or a dialer) (e.g., a touch gesture for scrolling (or dragging) the dialer left or right). According to one embodiment, FIGS. 22 to 24 may illustrate an example of a state in which an image (2201) is rotated left by a user input.

[0273] According to one embodiment, the electronic device (101) may generate and display a masking area (2220) indicating an area where a background can be generated around the periphery of the image (2201) in response to a rotation of the image (2201) based on a user input. In one embodiment, the masking area (2220) may indicate an expandable area around the image (2201) (e.g., at least one of the upper / lower / left / right areas of the image (2201). For example, image editing may include an operation of generating an image by background expansion based on a change in the tilt of the image (2201). In one embodiment, the second edit may include editing that creates a background (e.g., recreates an object) around an object (e.g., a subject) attached to an edge of the image (2201), so that the object within the image (2201) can be positioned in a desired area within the image (2201) even if the tilt of the image (2201) changes, and provides a naturally extended background so that the image (2201) can have a desired angle and composition by the user.

[0274] According to one embodiment, when the electronic device (101) executes image editing (e.g., second editing) (e.g., background expansion function or background expansion mode), the electronic device (101) may provide a real-time change state of the masking area (2220) (or expandable area) (e.g., at least one area among the upper / lower / left / right areas of the image (2201)) around the image (2201) according to the rotation of the image (2201) based on a user input (e.g., the degree to which the masking area (2220) is displayed). For example, the masking area (2220) (or expandable area) may be defined as about n% of the image (2201) (e.g., n is a natural number), and the masking area (2220) may be larger than an area displayed on the display of the electronic device (101). According to one embodiment, the electronic device (101) may provide a state in which the tilt (e.g., rotation), ratio, translation, horizontality and / or verticality of the image (2201) can be adjusted while displaying the masking area (2220).

[0275] According to one embodiment, the electronic device (101) can utilize the outside area of ​​the image (2201) rather than inside a cropper (or image cropper) (not shown) as a background extension area. In one embodiment, the cropper (or image cropper) may be software (or an application or program) (e.g., a plug-in (e.g., a flutter plug-in)) that supports processing when an image needs to be cropped in a repetitive and stereotypical form. For example, in the conventional art, the background area of ​​the image is defined as an area expected to be extended inside the cropper, but in the present disclosure, the background extension area can be defined outside the image (2201) rather than inside the cropper.

[0276] According to one embodiment, the electronic device (101) may provide real-time change status of the image (2201) (e.g., change status according to progress of left rotation) based on the editing tool (2210) in response to rotation of the image (2201) based on user input. For example, the electronic device (101) may provide information (e.g., tilt or angle information) on the rotation status of the image (2201) (e.g., “-2.8” in FIG. 22, “-5.5” in FIG. 23, “-12.0” in FIG. 24, indicating the current position value of the dialer).

[0277] According to one embodiment, as illustrated in FIGS. 25 to 26, FIGS. 25 to 26 may represent examples of a state in which an image (2201) is rotated to the right by a user input. For example, a user may perform a user input to rotate the image (2201) to the right to confirm the opposite composition, thereby reversing the rotation direction (or tilt) of the image (2201), as in the examples of FIGS. 22 to 24, while or after rotating the image (2201) to the left.

[0278] According to one embodiment, as illustrated in FIGS. 25 and 26, the electronic device (101) and the user may perform an image editing operation on the image (2201) corresponding to the operations described in the description with reference to FIGS. 22 to 24. For example, the electronic device (101) may display a change in a masking area (2220) indicating an area where a background may be generated around the periphery of the image (2201) in response to a rotation of the image (2201) based on a user input, while providing a real-time change status of the image (2201) (e.g., a status according to the progress of a right rotation). For example, the electronic device (101) may provide information (e.g., tilt or angle information) on the rotation status of the image (2201) (e.g., “-5.7” in FIG. 25, “3.4” in FIG. 26, indicating a current position value of the dialer).

[0279] According to one embodiment, a user may perform image editing based on a second edit on an image (2201) based on at least one operation performed as illustrated in FIGS. 22 to 26, and request image generation based on the edited image. In one embodiment, the image generation request may be performed by a designated command input. For example, the user may request image generation (or completion of image editing) based at least on a designated object for image generation (or regeneration) (e.g., a generation object (600)), a designated voice command, and / or a designated motion gesture (e.g., shaking the electronic device (101)). According to one embodiment, the electronic device (101) may determine an operation entry for generating a new image based on the edited image based on detecting a designated command input while performing image editing based on interaction with the user.

[0280] In one embodiment, the electronic device (101) may acquire a new image (e.g., a result image) in response to an instruction (or prompt). In one embodiment, the electronic device (101) may perform a process (e.g., an image generation process) related to generating a new image (e.g., a result image) based on an edited image based on detecting a specified command input while performing image editing based on user interaction. In one embodiment, the electronic device (101) may perform the image generation process by performing an operation corresponding to that described in the description with reference to FIGS. 17 and 18 in response to user interaction.

[0281] In one embodiment, upon completion of the image generation process, the electronic device (101) may interact with the user to provide an action corresponding to that described in the description with reference to FIGS. 19 and 20 . For example, the electronic device (101) may provide a function for comparing the generated image with the original image by switching between the generated image and the original image.

[0282] According to one embodiment, the electronic device (101) may perform operations corresponding to FIGS. 19 and 20 upon completion of the image generation process, and then, upon completion of image editing, interact with the user to complete the image editing operation with operations corresponding to those described in the description with reference to FIG. 21. For example, the electronic device (101) may release (or deactivate) the editing mode and provide the generated image (e.g., the result image) by including it in the user interface (e.g., the execution screen of the application). According to one embodiment, the electronic device (101) may provide an image reconstructed with background expansion based on the result of performing the second editing operation, for example, based on an empty area (e.g., a masking area) generated by image rotation. For example, the masking area may be generated by generating (e.g., by outpainting). For example, the electronic device (101) may provide an image in which the outer background area of ​​the image is expanded, while the blank area (e.g., masking area) generated as the background area is based on the image and the background (e.g., a background filled with an object attached to the edge of the image) is generated.

[0283] Various embodiments related to image editing operations based on tilt adjustment of an image according to one embodiment are described with reference to the drawings described below.

[0284] FIG. 27 is a flowchart illustrating a method of operating an electronic device according to one embodiment of the present disclosure.

[0285] According to one embodiment, FIG. 27 may illustrate an example of a method for supporting image editing and / or creation (e.g., regeneration or reprocessing) in an electronic device (101) according to one embodiment. According to one embodiment, FIG. 27 may illustrate an example of a method for supporting a first editing (e.g., object-based image editing) in an electronic device (101) according to one embodiment.

[0286] A method for supporting image editing and / or creation in an electronic device (101) according to one embodiment of the present disclosure may be performed, for example, according to a flowchart illustrated in FIG. 27. The flowchart illustrated in FIG. 27 is an example according to one embodiment of the operation of the electronic device (101), and the order of at least some operations may be changed or performed in parallel, performed as independent operations, or at least some other operations may be performed complementarily to at least some operations. According to one embodiment of the present disclosure, operations 2701 to 2717 may be performed in at least one processor of the electronic device (101) (e.g., processors 120 and 230 of FIGS. 1 to 3 ).

[0287] According to one embodiment, the operations described in FIG. 27 may be heuristically performed in combination with the operations described in FIGS. 4 to 26, for example, or may be heuristically performed as a replacement for at least some of the operations described and combined with at least some other operations, or may be heuristically performed as a detailed operation of at least some of the operations described.

[0288] As illustrated in FIG. 27, an operation method performed by an electronic device (101) according to an embodiment may include an operation of displaying a first image (2701), an operation of detecting a first input for selecting at least one object from the first image (2703), an operation of separating at least one object from the first image based on the first input (2705), an operation of detecting a second input related to image editing based on the at least one object (2707), an operation of performing image editing based on the at least one object based on the second input (2709), an operation of detecting an image generation request (2711), an operation of performing an image generation process based on the edited image (2713), an operation of generating a second image based on the image generation process (2715), and an operation of displaying the second image (2717).

[0289] Referring to FIG. 27, in operation 2701, the processor (120) of the electronic device (101) may display a first image. According to one embodiment, the processor (120) may execute an application (e.g., a gallery application, a shooting application, and / or an image editing application) and display a first image selected by a user on the display.

[0290] In operation 2703, the processor (120) may detect a first input for selecting at least one object from the first image. In one embodiment, the processor (120) may detect an input for performing image editing based on the first image. In one embodiment, the processor (120) may detect a designated input for editing the first image while the first image is displayed. In one embodiment, the designated input may include various touch gestures for selecting an object on the first image.

[0291] In operation 2705, the processor (120) may separate at least one object from the first image based on the first input. In one embodiment, the processor (120) may separate the at least one selected object from a background image including other objects not selected in the first image based on the first input.

[0292] In operation 2707, the processor (120) may detect a second input related to image editing based on at least one object. In one embodiment, the second input may include various inputs for editing at least one object, such as moving, deleting, copying, pasting, rotating (or tilting), adding an object, and / or resizing the at least one object.

[0293] In operation 2709, the processor (120) may perform image editing based on at least one object based on a second input. In one embodiment, the processor (120) may edit (e.g., first edit) at least one object separated from the first image independently from the background image based on the second input. In one embodiment, the processor (120) may display the background image in a fixed manner and change at least one separated object based on the second input.

[0294] In operation 2711, the processor (120) may detect a request for image generation. In one embodiment, the user may edit at least one object in the first image based on the second input and complete the image editing. In one embodiment, the completion of the image editing may be performed by a designated command input. For example, the user may request the completion of the image editing (or the generation of the image) based at least on a designated object (e.g., a software button provided on a user interface), a designated voice command, and / or a designated motion gesture (e.g., shaking the electronic device (101)) for completing the image editing (or image generation (or re-generation)). In one embodiment, the processor (120) may determine that a request for generation of a new image based on the edited image is made based on detecting a designated command input while performing the image editing based on the interaction with the user.

[0295] In operation 2713, the processor (120) may perform an image generation process based on the edited image. In one embodiment, the processor (120) may perform an image generation process related to generating a second image (e.g., a new image) based on a first image edited by image editing based on a second input, based on detection of an image generation request. In one embodiment, the image generation process may include an operation of generating (e.g., regenerating or reprocessing) the edited image based on artificial intelligence (AI) (e.g., generative AI). In one embodiment, the image generation process may be performed on-device and / or server-based. In one embodiment, the processor (120) may provide a prompt (or generative AI prompt) (e.g., a question or instruction to be entered into the generative AI) to the generative AI for the image generation process, for the edited image and / or for image generation based on the edited image.

[0296] In operation 2715, the processor (120) may generate a second image based on an image generation process. In one embodiment, the processor (120) may generate the second image based on an image generation process based on on-device artificial intelligence. In one embodiment, the processor (120) may acquire (or receive) the second image from a server based on an image generation process based on server artificial intelligence. In one embodiment, the processor (120) may determine the completion of image generation based on the generation and / or acquisition of the second image.

[0297] In operation 2717, the processor (120) may display a second image. According to one embodiment, the processor (120) may display the second image on the execution screen of the application through the display. According to one embodiment, the processor (120) may replace the first image that was previously displayed with the second image and display it. According to one embodiment, the processor (120) may display the second image and provide a comparison function that compares the first image and the second image by switching between them based on a user input. According to one embodiment, the processor (120) may display the second image and provide a storage function that stores the second image in an internal memory (e.g., a memory (130) of the electronic device (101)) and / or an external memory (e.g., a cloud) based on a user input.

[0298] FIGS. 28A and 28B are flowcharts illustrating a method of operating an electronic device according to one embodiment of the present disclosure.

[0299] According to one embodiment, FIGS. 28A and 28B may illustrate examples of a method for supporting image editing and / or creation (e.g., regeneration or reprocessing) in an electronic device (101) according to one embodiment. According to one embodiment, FIGS. 28A and 28B may illustrate examples of a method for supporting a first editing (e.g., object-based image editing) in an electronic device (101) according to one embodiment.

[0300] A method for supporting image editing and / or creation in an electronic device (101) according to one embodiment of the present disclosure may be performed, for example, according to the flowcharts illustrated in FIGS. 28A and 28B. The flowcharts illustrated in FIGS. 28A and 28B are examples according to one embodiment of the operation of the electronic device (101), and the order of at least some operations may be changed or performed in parallel, performed as independent operations, or at least some other operations may be performed complementarily to at least some operations. According to one embodiment of the present disclosure, operations 2801 to 2833 may be performed in at least one processor of the electronic device (101) (e.g., processors 120 and 230 of FIGS. 1 to 3 ).

[0301] According to one embodiment, the operations described in FIGS. 28A and 28B may be heuristically performed in combination with the operations described in FIGS. 4 through 27, or heuristically performed as a replacement for at least some of the operations described and combined with at least some other operations, or heuristically performed as a detailed operation of at least some of the operations described.

[0302] As illustrated in FIGS. 28A and 28B, an operation method performed by an electronic device (101) according to an embodiment includes an operation of displaying an image (2801), an operation of detecting a user input for selecting an object from an image (2803), an operation of separating an object from an image (2805), an operation of setting a masking area based on an area corresponding to an object (2807), an operation of setting a link between an object and a masking area (2809), an operation of detecting a user input for selecting an object or a masking area (2811), an operation of providing a handler based on an object based on a user input for selecting an object (2813), an operation of selecting and displaying a masking area corresponding to an object (2815), an operation of editing an object based on a user input (2817), an operation of recognizing a link set based on a user input for selecting a masking area (2819), an operation of determining whether an object corresponding to a masking area exists (2821), an operation of providing a handler based on a masking area based on the absence of an object (2823), a user input for selecting an object based on a user input for selecting an object (2824), an operation of determining whether an object corresponding to a masking area exists (2825), an operation of determining whether an object exists based on a user input for selecting an object (2826), an operation of determining whether an object exists based on a user input for selecting an object (2827), an operation of determining whether an object exists based on a user input for selecting an object (2828), an operation of determining whether an object exists based on a user input for selecting an object (2829), an operation of determining whether an object exists based on a user input for selecting an object (2830), an operation of determining whether an object exists based on a user input for selecting an object (2831), an operation of determining whether an object exists based on a user input for selecting an object (2832), an operation of determining whether an object exists based on a user input for selecting an object (2833 It may include an operation of restoring an object corresponding to a masking area based on an input (2825), an operation of providing a handler based on an object corresponding to a masking area based on the existence of an object (2827), an operation of editing or restoring an object based on a user input (2829), an operation of detecting an image generation request (2831), and an operation of regenerating an image based on an image generation process (2833).

[0303] Referring to FIGS. 28A and 28B, in operation 2801, the processor (120) of the electronic device (101) may display an image. According to one embodiment, the processor (120) may execute an application (e.g., a gallery application, a shooting application, and / or an image editing application) and display an image selected by a user (e.g., a first image or an original image) on the display.

[0304] In operation 2803, the processor (120) may detect a user input for selecting an object in an image. In one embodiment, the processor (120) may detect an input for performing image editing based on the image. In one embodiment, the processor (120) may detect a designated input for selecting an image while the image is displayed. In one embodiment, the designated input may include various touch gestures for selecting an object in the image.

[0305] In operation 2805, the processor (120) may separate an object from an image. In one embodiment, the processor (120) may separate one or more objects selected based on a specified input from a background image that includes other objects not selected from the image.

[0306] In operation 2807, the processor (120) may set a masking area based on an area corresponding to an object. In one embodiment, the masking area may include an empty area generated by object editing. In one embodiment, the processor (120) may set and display an area where the selected object was originally located in the image (e.g., an area where the object was located before moving) as a masking area based on the separation of the selected object from the image. For example, the processor (120) may display a masking (e.g., a grid pattern) on the area where the selected object was originally located. In one embodiment, the masking area may be an area where, after separating an object within an image, a deletion mark (e.g., a masking) is made using a designated graphic object (e.g., a grid pattern) to indicate that the object is deleted (or removed) from its original location (e.g., a masking area).

[0307] In operation 2809, the processor (120) may establish a link between an object and a masking area. In one embodiment, the processor (120) may establish a relationship between a selected object and a masking area. In one embodiment, the relationship between the selected object and the masking area may indicate a link (or connection) between the object and the masking area such that the object and the masking area inter-operate, for example, based on a user input based on the selected object or masking area. In one embodiment, the processor (120) may display a designated graphical object (e.g., a dotted line (e.g., a lasso selection) or edge highlighting) based on the outer edge (or boundary or border) of the masking area to provide the user with a state in which the masking area is set.

[0308] In operation 2811, the processor (120) may detect a user input selecting an object or masking area.

[0309] In operation 2811, the processor (120) may provide a handler based on an input for selecting an object, and in operation 2813, based on the selected object. According to one embodiment, the processor (120) may provide a handler related to image editing in association with the object. According to one embodiment, the user may perform at least one edit (e.g., a first edit such as move, position, size, angle, and / or remove) related to the object based on the handler provided based on the object. In one embodiment, the handler may include a handler box and at least one tool (e.g., an editing tool or a function tool). In one embodiment, the handler box may be provided as a rectangular box surrounding at least one target object (e.g., a selected object). In one embodiment, the tool may include at least one function object (or tool object) each corresponding to an option (or tool) related to editing of at least one target object (e.g., a selected object). For example, a tool may include a first functional object (e.g., an undo object) and a second functional object (e.g., an eraser object or a trash can object).

[0310] In operation 2815, the processor (120) may select and display a masking area corresponding to an object. According to one embodiment, the processor (120) may determine a masking area corresponding to the selected object based on an input for selecting the object. According to one embodiment, the processor (120) may recognize a set link based on an input for selecting the object, and determine a masking area corresponding to the selected object based on the set link. According to one embodiment, the processor (120) may select and display the determined masking area. In one embodiment, the processor (120) may display a designated graphic object (e.g., edge dotted line or edge highlighting) based on an outer edge (or boundary or boundary surface) of the masking area to provide the user with a state in which the masking area is selected.

[0311] In operation 2817, the processor (120) may edit an object based on a user input. In one embodiment, the processor (120) may perform object editing (e.g., object-based image editing) within an image, such as moving, deleting, copying, pasting, rotating (or tilting), and / or resizing at least one object, or adding a new object, based on the user input. In one embodiment, the processor (120) may edit (e.g., first editing) an object separated from an image independently from a background image based on the user input. In one embodiment, the processor (120) may display the background image in a fixed manner and change the separated object based on the user input.

[0312] In operation 2811, the processor (120) may recognize an established link based on an input for selecting a masking area, in operation 2819. According to one embodiment, the processor (120) may recognize an established link between an object and a masking area based on an input for selecting a masking area. According to one embodiment, the processor (120) may select and display a selected masking area based on an input for selecting a masking area. In one embodiment, the processor (120) may display a designated graphic object (e.g., edge dotted line or edge highlighting) based on an outer edge (or boundary or boundary surface) of the masking area to provide a user with a state in which the masking area is selected.

[0313] In operation 2821, the processor (120) may determine whether an object corresponding to the masking area exists. According to one embodiment, the processor (120) may determine an object corresponding to the selected masking area based on the established link recognition. According to one embodiment, the processor (120) may recognize the established link based on an input for selecting the masking area, and may determine whether an object corresponding to the selected masking area exists on the image based on the established link.

[0314] In operation 2821, the processor (120) may provide a handler based on the absence of an object corresponding to the masking area on the image (e.g., based on detecting the absence of an object on the image), in operation 2823, based on the masking area. In one embodiment, the absence of an object corresponding to the masking area may include, for example, a case where the object corresponding to the masking area is removed by the user's object editing. According to one embodiment, when the object corresponding to the masking area on the image does not exist, the processor (120) may provide a handler related to image editing in association with the masking area. In one embodiment, the handler may include a handler box and at least one tool (e.g., an editing tool or a function tool). In one embodiment, the handler box may be provided as a rectangular box surrounding the target masking area (e.g., a selected masking area). In one embodiment, the tool may include a function object (e.g., an undo object) related to the restoration of the object corresponding to the masking area.

[0315] In operation 2825, the processor (120) may restore an object corresponding to the masked area based on a user input. In one embodiment, the user may perform at least one edit (e.g., restoration) related to the object based on a handler provided based on the masked area. In one embodiment, the processor (120) may detect the user input through a function object (e.g., an undo object) of the handler, and may restore and provide an object corresponding to the masked area (e.g., a removed object) to the masked area based on the user input based on the function object.

[0316] In operation 2821, the processor (120) may provide a handler based on the presence of an object corresponding to the masking area on the image (e.g., based on detecting the presence of an object on the image), in operation 2827, based on the object corresponding to the masking area. According to one embodiment, when the object corresponding to the masking area exists, the processor (120) may provide a handler related to image editing in association with the object corresponding to the masking area. In one embodiment, the handler may include a handler box and at least one tool (e.g., an editing tool or a function tool). In one embodiment, the handler box may be provided as a rectangular box surrounding a target object (e.g., an object corresponding to the selected masking area). In one embodiment, the tool may include a function object related to editing the target object (e.g., an eraser object or a trash can object) and a function object related to restoring the object (e.g., an undo object).

[0317] In operation 2829, the processor (120) may edit or restore an object based on a user input. In one embodiment, the user may perform at least one edit related to the object (e.g., a first edit such as a move, a position, a size, an angle, and / or a removal) based on a handler provided based on the object, or may cancel the editing of the object and restore it to the masked area. In one embodiment, the processor (120) may detect a user input based on the handler, and may edit and provide the object based on a function (e.g., an edit or a restore) according to the user input, or may restore and provide an object corresponding to the masked area (e.g., an edited object) to the masked area.

[0318] In operation 2831, the processor (120) may detect a request for image generation. According to one embodiment, the user may complete image editing based on a specified command input. In one embodiment, the specified command input may include an input for completing image editing (or image generation (or image regeneration)) based on a specified object (e.g., a software button (e.g., a creation object)), a specified voice, and / or a specified motion gesture (e.g., shaking the electronic device (101)) for completing image editing (or image generation (or image regeneration)). According to one embodiment, the processor (120) may determine that a request for new image generation (or image regeneration) based on an edited image is made based on detecting a specified command input while performing image editing based on interaction with the user.

[0319] In operation 2833, the processor (120) may regenerate an image based on an image generation process. In one embodiment, the processor (120) may perform an image generation process related to generating a new image (e.g., a second image) based on an edited image (e.g., a first image) based on detection of an image generation request. In one embodiment, the image generation process may include generating (e.g., regenerating or reprocessing) the first image into a second image based on artificial intelligence (AI) (e.g., generative AI). In one embodiment, the image generation process may be performed on-device and / or server-based.

[0320] In one embodiment, the processor (120) may provide a prompt (or generative AI prompt) (e.g., a question or instruction input to the generative AI) to the generative AI for the image generation process, for editing the image and / or for image generation based on the edited image. In one embodiment, the processor (120) may generate a second image based on the image generation process. In one embodiment, the processor (120) may provide the generated second image to the user. For example, the processor (120) may display the second image on a display.

[0321] According to one embodiment, the processor (120) may display a second image and provide a comparison function for comparing the first image and the second image by switching between them based on a user input. According to one embodiment, the processor (120) may display a second image and provide a storage function for storing the second image in an internal memory (e.g., a memory (130) of the electronic device (101)) and / or an external memory (e.g., a cloud) based on a user input.

[0322] FIG. 29 is a flowchart illustrating a method of operating an electronic device according to one embodiment of the present disclosure.

[0323] According to one embodiment, FIG. 29 may illustrate an example of a method for supporting image editing and / or creation (e.g., regeneration or reprocessing) in an electronic device (101) according to one embodiment. According to one embodiment, FIG. 29 may illustrate an example of a method for supporting second editing (e.g., image-based image editing) in an electronic device (101) according to one embodiment.

[0324] A method for supporting image editing and / or creation in an electronic device 101) according to one embodiment of the present disclosure may be performed, for example, according to a flowchart illustrated in FIG. 29. The flowchart illustrated in FIG. 29 is an example according to one embodiment of an operation of the electronic device (101), and the order of at least some operations may be changed or performed in parallel, performed as independent operations, or at least some other operations may be performed complementarily to at least some operations. According to one embodiment of the present disclosure, operations 2901 to 2917 may be performed in at least one processor (e.g., processors 120 and 230 of FIGS. 1 to 3 ) of the electronic device (101).

[0325] According to one embodiment, the operations described in FIG. 29 may be heuristically performed in combination with the operations described in FIGS. 4 through 28B, or heuristically performed as a replacement for at least some of the operations described and combined with at least some other operations, or heuristically performed as a detailed operation of at least some of the operations described.

[0326] As illustrated in FIG. 29, the method of operation performed by the electronic device (101) according to one embodiment may include an operation of displaying an image (2901), an operation of displaying an expanded area based on the periphery of the image (2903), an operation of rotating the image in a first direction based on a first input while the corner of the image touches the cropper (2905), an operation of increasing the display size of the image from a point in time when the expanded area touches the cropper during image rotation and displaying it, an operation of rotating the image in a state where the expanded area touches the cropper (2907), an operation of rotating the image in a state where the expanded area touches the cropper (2909), an operation of rotating the image in a second direction based on a second input (2911), an operation of decreasing the display size of the image from a point in time when the corner of the image touches the cropper during image rotation and displaying it, an operation of rotating the image in a state where the corner of the image touches the cropper (2915), and an operation of displaying the size of the image in an initial state when the state of the image reaches an initial inclination (2917).

[0327] Referring to FIG. 29, in operation 2901, the processor (120) of the electronic device (101) may display an image. According to one embodiment, the processor (120) may execute an application (e.g., a gallery application, a shooting application, and / or an image editing application) and display an image selected by a user (e.g., a first image or an original image) on the display.

[0328] In operation 2903, the processor (120) may display an expansion area based on the periphery of the image. In one embodiment, the processor (120) may detect an input for performing image editing based on the image. In one embodiment, the processor (120) may switch to an editing mode based on the image editing execution, and in the editing mode, may display an expansion area (e.g., a masking area) to be expanded based on the periphery of the image.

[0329] In operation 2905, the processor (120) may rotate the image in a first direction based on a first input, such that the corner of the image touches the cropper. In one embodiment, the user may perform the first input to adjust the tilt of the image (e.g., image rotation). For example, the user may perform a touch gesture input to rotate the image left or right based on a two-finger touch, a touch gesture input to operate an editing tool (e.g., an image rotation tool or a dialer) (e.g., a touch gesture to scroll (or drag) the dialer left or right), or a direct input of a tilt angle (e.g., a number). For example, the user may operate the image to rotate right by the first input.

[0330] In one embodiment, the processor (120) may adjust the tilt (e.g., image rotation) of the image based on the first input so that the corners of the image touch the cropper. In one embodiment, the tilt of the image may be adjusted so that at least two corners of the image come inside the cropper and the remaining two corners touch the cropper. In one embodiment, when the image is rotated according to the adjusted tilt, the processor (120) may also rotate an extended area (e.g., a masking area) associated with the image.

[0331] In operation 2907, the processor (120) may increase the display size of the image from the point in time when the expansion area touches the cropper during image rotation. In one embodiment, when the rotation angle of the image increases while the tilt of the image is adjusted while touching the cropper, the expansion area may touch the cropper. In one embodiment, the processor (120) may increase and display the display size of the image from the point in time when the expansion area touches the cropper. In one embodiment, the processor (120) may adjust the tilt and expansion area of ​​the image while rotating the expansion area of ​​the image while touching the cropper and increasing the sizes of the image and the expansion area.

[0332] In operation 2909, the processor (120) may rotate the image while the expansion region is in contact with the cropper. In one embodiment, the expansion region of the image may be rotated while in contact with the cropper, and the size of the image and the expansion region may be adjusted as they grow. For example, from the point where the image and the expansion region are in contact with the cropper at the same time, the tilt of the image (e.g., image rotation) may be adjusted as the image grows (e.g., as the image moves out of the cropper). In one embodiment, the processor (120) may maintain the expansion region in contact with the cropper during the operation where the image and the expansion region are adjusted as they grow.

[0333] In operation 2911, the processor (120) may rotate the image in a second direction based on a second input. In one embodiment, while the user is rotating the image based on the first input, the user may perform a second input to rotate the image in a second direction opposite to the first direction according to the first input. For example, the user may operate the second input to rotate the image to the left.

[0334] In operation 2913, the processor (120) may reduce the display size of the image from the point where the corner of the image touches the cropper during image rotation. In one embodiment, the corner of the image may touch the cropper when the rotation angle of the image becomes smaller while the image is being tilted away from the cropper. In one embodiment, the processor (120) may reduce the display size of the image and display it from the point where the corner of the image touches the cropper. In one embodiment, at least two corners of the image may come inside the cropper, and the remaining two corners may touch the cropper.

[0335] In operation 2915, the processor (120) may rotate the image so that the corner of the image touches the cropper. In one embodiment, the processor (120) may rotate the image so that the expanded area of ​​the image touches the cropper, and adjust the tilt and expanded area of ​​the image while reducing the size of the image and the expanded area.

[0336] In operation 2917, the processor (120) may display the size of the image as the initial state when the state of the image reaches the initial slope.

[0337] Below, examples of various operations that support image editing in electronic devices of various embodiments (e.g., electronic devices (101, 201) of FIGS. 1 to 3) (hereinafter, electronic device (101)) are described.

[0338] According to one embodiment, a user may have a need to change the composition of an image (e.g., a photo (or image) taken by the electronic device (101) or an image downloaded from an external source) stored in the electronic device (101) to a desired composition when taking a photo. According to one embodiment, the electronic device (101) supports freely changing the composition of an image (e.g., a taken photo), thereby satisfying the needs of a user who wants to change the composition of a poorly taken photo or an image whose composition the user wants to change.

[0339] According to one embodiment, FIGS. 30 to 60 may illustrate examples of operations that support image editing based on artificial intelligence (AI) in an electronic device (101) according to one embodiment. According to one embodiment, the AI ​​may include generative AI. Generative AI may refer to an AI technology that creates similar content using existing content such as text, audio, and / or images. For example, generative AI may refer to an AI technology that can generate content (e.g., text, audio, image, and / or video) corresponding to an input based on a given input.

[0340] According to one embodiment, the electronic device (101) may generate (e.g., reconstruct) images and / or objects in response to the user's image editing based on generative artificial intelligence (e.g., on-device AI) and provide the generated images and / or objects. According to one embodiment, the electronic device (101) may request a server to generate (e.g., reconstruct) images in response to the image editing, and may receive and provide images generated based on the server's generative artificial intelligence from the server.

[0341] According to one embodiment, as illustrated in FIGS. 30 to 60, the method of operation performed by the electronic device (101) according to one embodiment may edit an image based on interaction with a user, and generate (e.g., regenerate or reconstruct) and provide an image on a server or on-device based on the edited image. According to one embodiment, the operation of editing an image may include an operation related to a first editing and an operation related to a second editing.

[0342] According to one embodiment, the operations described with reference to FIGS. 30 to 60 may be heuristically performed in combination with the operations described in FIGS. 4 to 29, for example, or may be heuristically performed by replacing at least some of the operations described and combining at least some other operations, or may be heuristically performed as detailed operations of at least some of the operations described.

[0343] According to one embodiment, FIGS. 30 through 49B may illustrate various image editing operation examples based on a first editing according to one embodiment. For example, FIGS. 30 through 49B may illustrate operations for generating (e.g., regenerating or reconstructing) an image based on object editing in an image. According to one embodiment, FIGS. 50 through 60 may illustrate various image editing operation examples based on a second editing according to one embodiment. For example, FIGS. 50 through 60 may illustrate operations for generating (e.g., regenerating or reconstructing) an image based on background expansion in an image.

[0344] According to one embodiment, the first editing operation based on FIGS. 30 to 49B and the second editing operation based on FIGS. 50 to 60 may be performed individually, collectively, and / or sequentially in one operation.

[0345] FIG. 30 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0346] Referring to Figure 30, an example <3001> As illustrated, the electronic device (101) may display an image (3000) selected by a user on a display. According to one embodiment, the image (3000) may be provided through a designated user interface related to image editing. According to one embodiment, the electronic device (101) may enter an editing mode (e.g., an artificial intelligence mode or an artificial intelligence editing mode) based on the execution of image editing on the selected image (3000) and provide a user interface of the editing mode including the selected image (3000).

[0347] In one embodiment, the example <3003> As illustrated, the electronic device (101) can select an object (3010) from an image (3000).

[0348] In one embodiment, the example <3005> Wow, example <3007> As illustrated, the electronic device (101) may crop (e.g., separate from the image (3000)) and move an object (3010) selected in the image (3000) based on a designated input (e.g., a tap and hold gesture). According to one embodiment, the electronic device (101) may provide a guide phrase indicating that the object (3010) in the image (3000) may be moved with a designated input when the object (3010) in the image (3000) is selected. According to one embodiment, the electronic device (101) may select the object (3010) in response to a first input (e.g., an input for selecting a region portion of the object (3010) in the image (3000), crop a portion corresponding to the object (3010) in response to a second input (e.g., a tap and move gesture for selecting and moving the object (3010), and move the cropped portion (e.g., the object (3010)). According to one embodiment, the electronic device (101) may not provide a guide phrase if an object (3010) is selected by a specified input and then moved immediately.

[0349] According to one embodiment, when an input specified in a portion connected to a previously selected object area is received while an object (3010) is selected, the electronic device (101) may select and provide an object area in continuation with the previous selection. An example of this will be described below.

[0350] According to one embodiment, the electronic device (101) can provide clipping (e.g., object manipulation state) of an object (3010) (or subject). For example, the electronic device (101) can crop (or separate) the object (3010) from the background image (3000) based on a user's specified input (e.g., a tap-and-hold gesture) in the displayed image (3000). According to one embodiment, the electronic device (101) can fix the background image and allow smooth movement of the selected object (3010). For example, the electronic device (101) can fix the background image in the object manipulation state. For example, the electronic device (101) can fix the background image except for the selected object (3010) in the image (3000) and support object editing (e.g., moving, expanding, reducing, rotating, removing, creating) based on the separated object (3010) (e.g., the cropped portion).

[0351] According to one embodiment, the electronic device (101) may provide a handler (3020) having a function of controlling (e.g., editing) an object (3010) around an object (3010) separated by a specified input. For example, the electronic device (101) may crop a portion corresponding to the object (3010) in an image (3000) and provide the handler (3020) around the object (3010) based on the movement of the cropped portion from its original location within the image (3000). According to one embodiment, the user may perform an operation of editing (e.g., moving, expanding, reducing, rotating, removing, creating) the object (3010) based on the handler (3020).

[0352] According to one embodiment, the electronic device (101) may remove (e.g., make hidden) the handler (3020) of the object (3010) and not display it when an area other than the object (3010) (e.g., a background image area) is selected.

[0353] According to one embodiment, the electronic device (101) can separate an object (3010) and display a mask (e.g., a grid pattern) on an area (3030) (e.g., a masking area) where the separated object (3010) originally resided. According to one embodiment, in an object manipulation state, a background image can be provided in a fixed state, and the separated object (3010) can be edited independently of the background image.

[0354] FIG. 31 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0355] According to one embodiment, FIG. 30 may illustrate an example of a screen and its operation when a cropper (3130) is not provided. According to one embodiment, FIG. 31 may illustrate an example of a screen and its operation when a cropper (3130) is provided.

[0356] Referring to Figure 31, an example <3101> As illustrated, the electronic device (101) may display an image (3100) selected by a user on a display. According to one embodiment, the image (3100) may be provided through a designated user interface related to image editing. According to one embodiment, the electronic device (101) may enter an editing mode (e.g., an artificial intelligence mode or an artificial intelligence editing mode) based on the execution of image editing on the selected image (3100) and provide a user interface of the editing mode including the selected image (3100).

[0357] In one embodiment, the electronic device (101) may provide a cropper (3130) based on the image (3100) to indicate a state in which the edit mode has been entered based on the entry into the edit mode. For example, the cropper (3130) may indicate a controllable state of the image (3100) and may display a guide box that allows the image (3100) to be edited using the cropper (3130). In one embodiment, the cropper (3130) may or may not be displayed depending on the settings of the electronic device (101).

[0358] In one embodiment, the example <3103> As illustrated, the electronic device (101) can select and move an object (3110) in an image (3100).

[0359] According to one embodiment, the electronic device (101) can crop (e.g., separate from the image (3100)) and move an object (3110) selected from an image (3100) based on a specified input (e.g., a tap-and-hold gesture). According to one embodiment, the electronic device (101) can select the object (3110) in response to a first input (e.g., an input for selecting a region portion of the object (3110) in the image (3100), crop a portion corresponding to the object (3110) in response to a second input (e.g., a tap-and-hold gesture for selecting and moving the object (3110), and move the cropped portion (e.g., the object (3110)).

[0360] According to one embodiment, the electronic device (101) may crop (or separate) an object (3110) from a background image (3000) based on a user's specified input (e.g., a tap-and-hold gesture) in the image (3100). According to one embodiment, the electronic device (101) may remove (e.g., hide) a cropper (3130) displayed on the image (3100) and not display it based on the specified input. For example, the electronic device (101) may remove the display of the cropper (3130) and fix the background image so that the selected object (3110) can move smoothly. For example, as shown in <3105> As illustrated, the electronic device (101) can fix the background image except for the selected object (3110) in the image (3100) and support object editing (e.g., moving, expanding, reducing, rotating, removing, creating) based on the separated object (3110) (e.g., cropped portion).

[0361] According to one embodiment, the electronic device (101) may provide a handler (3120) having a function of controlling (e.g., editing) an object (3110) around an object (3110) separated by a specified input. For example, the electronic device (101) may crop a portion corresponding to the object (3110) in an image (3100) and provide the handler (3120) around the object (3110) based on the movement of the cropped portion from its original location within the image (3100). According to one embodiment, the user may perform an operation of editing (e.g., moving, expanding, reducing, rotating, removing, creating) the object (3110) based on the handler (3120).

[0362] According to one embodiment, the electronic device (101) may, for example, select an area other than the object (3110) (e.g., a background image area). <3107> As illustrated, the handler (3120) of the object (3110) can be removed (e.g., hidden) so that it is not displayed, and the cropper (3130) can be displayed while selecting a background image.

[0363] According to one embodiment, the electronic device (101) can separate an object (3110) and display a mask (e.g., a grid pattern) on an area (3140) (e.g., a masking area) where the separated object (3110) was originally located. According to one embodiment, in an object manipulation state, a background image can be provided in a fixed manner, and the separated object (3110) can be edited independently of the background image.

[0364] FIG. 32 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0365] As illustrated in FIG. 32, FIG. 32 may illustrate an example of adjusting the tilt, straighten, vertical, and / or horizontal of an image and / or a selected object within the image using an image editing tool (or toolbar) (e.g., a cropper (3200) and / or a dialer (3210)).

[0366] According to one embodiment, the electronic device (101) may provide a cropper (3200) based on an image when an image (e.g., a background image) is selected (e.g., an image is focused). According to one embodiment, the electronic device (101) may provide a dialer (3210) in a bottom menu that can adjust tilt, straighten, horizontal, and / or vertical when an image is selected (e.g., an image is focused). For example, a user may edit an image and / or a selected object within the image using the cropper (3200) and / or the dialer (3210).

[0367] According to one embodiment, when reconstructing the appearance of an image, the electronic device (101) may provide a cropper (3200) for controlling the image to the image, and may support image editing using the cropper (3200). According to one embodiment, when reconstructing the appearance of an image, the electronic device (101) may provide a dialer (3210) for controlling the image as a bottom menu, and may support image editing using the dialer (3210).

[0368] According to one embodiment, when an area other than a separated object (e.g., a background image area) is selected, the electronic device (101) can select a background image (e.g., background focus) and support editing of the background image based on the cropper (3200) and / or the dialer (3210) of the bottom menu. According to one embodiment, when the cropper (3200) is displayed on an image (e.g., a background image), the user can edit the image based on the cropper (3200) and / or the dialer (3210) of the bottom menu.

[0369] FIG. 33 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0370] According to one embodiment, FIG. 33 is an example <3301> , example <3303> , example <3311> and / or examples <3313> As illustrated, an example of adjusting the tilt, straightness, verticality, and / or horizontality of an object (3310) separated from an image (3300) using an editing tool (or toolbar) may be presented.

[0371] According to one embodiment, the electronic device (101) may provide a handler (3320) based on the object (3310) when the object (3310) is selected (e.g., object focus) in the image (3300). According to one embodiment, the electronic device (101) may provide a dialer (3330) in the bottom menu that can adjust tilt, straighten, horizontal, and / or vertical when the object (3310) is selected. For example, a user may edit the object (3310) using the handler (3320) and / or the dialer (3330).

[0372] According to one embodiment, when reconstructing the appearance of an object (3310), the electronic device (101) may provide a handler (3320) for controlling a separated object (3310) around the focused object (3310) (e.g., a separated object) and support object editing using the handler (3320). According to one embodiment, when reconstructing the appearance of an object, the electronic device (101) may provide a dialer (3330) for controlling a separated object (3310) as a bottom menu and support object editing using the dialer (3330). According to one embodiment, when the handler (3320) is displayed on the separated object (3310), the user may edit (e.g., tilt, straighten, vertically, and / or horizontally adjust) the object (3310) based on the handler (3320) and / or the dialer (3330) of the bottom menu.

[0373] FIG. 34 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0374] According to one embodiment, FIG. 34 is an example <3401> , example <3403> and examples <3405> As illustrated in , an example of user input for selecting an object (3410) (e.g., an object to be edited) in an image may be represented. For example, an example <3401> , example <3403> and examples <3405> As illustrated, a user may select (or designate) at least one target object (e.g., object (3410)) for image editing in an image. In one embodiment, the electronic device (101) may select at least one object corresponding to a user input in the image as a target object for image editing.

[0375] In one embodiment, the various user inputs related to image editing to select a target object are not limited to input via the user's finger, and may include input via various input devices (e.g., a stylus pen, a stylus, a mouse). In one embodiment, the user input may include input based on at least one of various touch gesture-based inputs.

[0376] For example, the user input may include a first gesture (e.g., single tap, double tap, triple tap, or long press) for tapping the object (3410) on the image as an input for specifying selection of an object (3410) within the image, a second gesture for selecting the object (3410) in a square or circle on the image, a third gesture (e.g., drawing a closed curve, drawing a circle, drawing a lasso) for more free and flexible selection of the object (3410) by drawing a relatively circular shape without a specified shape (e.g., drawing a circle to include the object (3410) within it), and / or a fourth gesture (e.g., tap & move or long tap & move) for selecting and moving the object (3410) within the image.

[0377] In one embodiment, FIG. 34 may illustrate an example of object (3410) selection based on a third gesture.

[0378] According to one embodiment, a user may perform a user input (3420) to draw a periphery of an object (3410) on an image (e.g., an input to draw a relatively circular shape to include the periphery of the object (3410). According to one embodiment, the electronic device (101) may select at least one object (3410) within a range drawn according to the user input (3420) on the image (e.g., within the circle) as a target object for image editing. According to one embodiment, the electronic device (101) may provide a handler (3430) in association with the object (3410) based on the selection of the object (3410). In one embodiment, the handler (3430) may include a handler box and / or at least one functional object.

[0379] In one embodiment, the electronic device (101) may display an object area based on (or centered on) the selected object (3410). In one embodiment, the object area may display a dotted line (e.g., a lasso selection) based on the outer edge (or boundary or boundary surface) of the object (3410) to provide the user with a selected state of the object (3410).

[0380] According to one embodiment, the user can perform various edits, such as moving, deleting, copying, pasting, rotating (or adjusting the angle or tilt), adding objects, horizontally adjusting, vertically adjusting, and / or resizing, based on the selected object (3410). According to one embodiment, the selected object (3410) can be separated (or cropped) from an image (e.g., a background area) based on a user input and edited separately (or independently) from the image. For example, the object (3410) within the image can be cropped by a user input (3420) to enable movement and / or editing. In one embodiment, the electronic device (101) can crop (or separate) a portion corresponding to the object (3410) from the image. For example, the electronic device (101) can crop an object (3410) from a background image that includes other objects that are not selected based on user input (3420) in the image, and edit the cropped portion separately from the background area based on the specified input.

[0381] According to one embodiment, the electronic device (101) may deselect a drawn area based on a designated input related to deselection (e.g., a tap input on a background area) after selection of an object (3410). For example, if a designated input related to deselection is detected while the object (3410) is selected by drawing or tapping, the electronic device (101) may deselect the selected object (3410).

[0382] FIG. 35 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0383] According to one embodiment, FIG. 35 may illustrate an example of an operation that supports selection of multiple objects (e.g., a first object (3510) and a second object (3520)) on an image (3500) based on user input.

[0384] According to one embodiment, FIG. 35 is an example <3501> , example <3503> , example <3505> , and examples <3507> As illustrated in , an example of an operation of selecting a plurality of objects (e.g., a first object (3510) and a second object (3520)) in an image (3500) as a single bundle object (or group object) may be presented. According to one embodiment, a user may select each object (e.g., a first object (3510) and a second object (3520)) in the image (3500) and designate it as a bundle object. For example, the electronic device (101) may support multi-selection of a plurality of objects in an image.

[0385] According to one embodiment, the electronic device (101) may set the selected plurality of objects (3510, 3520) as a group object (or group object) based on a designated input (e.g., touch & hold or long press) based on the last selected object (e.g., the second object (3520)) after the plurality of objects (3510, 3520) are selected. According to one embodiment, the electronic device (101) may provide a handler (3530) by expanding it to include the plurality of objects (3510, 3520). According to one embodiment, the electronic device (101) may adjust (e.g., expand) an area of ​​a handler box of the handler (3530) to include the plurality of objects (3510, 3520).

[0386] For example, the electronic device (101) may group a plurality of objects (3510, 3520) based on a specified input after multi-selection of objects and provide the group as a single selection object (e.g., a bundled object). For example, after the plurality of objects (3510, 3520) are selected, the electronic device (101) may group the selected plurality of objects (3510, 3520) in response to a specified input based on one object (e.g., the last selected object (3520)), and support an operation of image editing based on the bundled object (e.g., moving, deleting, copying, pasting, rotating, horizontally adjusting, vertically adjusting, and / or resizing based on the bundled object).

[0387] FIG. 36 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0388] According to one embodiment, FIG. 36 may illustrate an example of an operation that supports restoration of an object separated from an image.

[0389] example <3601> , example <3603> and examples <3605> As illustrated, the electronic device (101) may provide a functional object (e.g., an undo object) that supports restoration of the object (3610) based on a first handler (3620) provided through the object (3610). According to one embodiment, when the object (3610) is removed, the electronic device (101) may provide a functional object that supports restoration of the object (3610) based on a second handler (3640) provided through a masking area (3630) corresponding to the object (3610).

[0390] According to one embodiment, while the user is manipulating an object (3610) separated from an image, the user can restore the separated object (e.g., the object being edited) to its original state by using a designated function object (e.g., an undo object) of the first handler (3620) of the object (3610). For example, while editing the object (3610), the electronic device (101) can restore and provide the object (3610) to its original state in response to a user input based on the designated function object of the first handler (3620). According to one embodiment, based on the restoration of the object (3610), the electronic device (101) can set (e.g., a link) for a masking area (3630) of the object (3610) and remove the masking area (3630) from the image.

[0391] According to one embodiment, a user may remove a detached object (3610) from an image using a functional object (e.g., an eraser object or a trash can object) of a first handler (3620) of the detached object (3610). According to one embodiment, in response to a request to remove the object (3610), the electronic device (101) may remove the object (3610) from the image and maintain a masking area (3630) corresponding to (e.g., linked to) the removed object (3610). According to one embodiment, the electronic device (101) may remove the object (3610) from the image based on a user input and provide a second handler (3640) based on the masking area (3630) corresponding to the removed object (3610).

[0392] According to one embodiment, the user can restore the deleted object (3610) to its original state by using a designated function object (e.g., an undo object) of the second handler (3640) of the masking area (3630) corresponding to the object (3610) removed from the image. For example, after the object (3610) is removed, the electronic device (101) can provide the second handler (3640) to the selected masking area (3630) based on the selection of the masking area (3630) corresponding to the removed object (3610). According to one embodiment, the electronic device (101) can restore and provide the removed object (3610) to its original state in response to a user input based on the designated function object of the second handler (3640) of the masking area (3630). In one embodiment, the electronic device (101) may, based on the restoration of the object (3610), set (e.g., link) a masking area (3630) of the object (3610) and remove the masking area (3630) from the image.

[0393] FIG. 37 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0394] According to one embodiment, FIG. 37 may illustrate an example of an operation for expanding or reducing a selection area of ​​a cropped object when the separated object is a cropped object attached to an edge or an object that is cropped because it is obscured by another object.

[0395] example <3701> , example <3703> , example <3705> , and examples <3707> As illustrated, in the case of an object (3710) separated from an edge of an image (3700), a portion of the object (3710) may be cut off. In one embodiment, an object that can be selected and separated from an edge may be an object that has one side (or face) of the object attached to the edge. For example, an object that has two sides attached to the edge may not support selection and separation. For example, an object that has two sides attached to an edge of an image (3700) may be blocked.

[0396] According to one embodiment, the electronic device (101) may provide a handler (3720) (e.g., a handler box and a function object) based on the surroundings of the object (3710) when the cropped object (3710) is selected and moved. According to one embodiment, the electronic device (101) may display a mask (e.g., a grid pattern) in the area (3730) (e.g., a masking area) where the object (3710) was originally located when the object (3710) is selected and moved.

[0397] According to one embodiment, when an object (3710) attached to an edge of an image (3700) is selected and moved, the electronic device (101) may provide an expansion object (3740) that supports expansion of a handler box on a boundary surface (e.g., a right side of the handler box) of a region (e.g., a right side of the object (3710)) where the object (3710) was attached to the edge. According to one embodiment, a user may expand a handler box of a handler (3720) by using the expansion object (3740) of the handler box (e.g., expand the handler box based on a rightward drag of the expansion object (3740).

[0398] According to one embodiment, the electronic device (101) can support expanding or reducing the selection area based on the handler box even if the object is not attached to an edge and is cut off by being covered by another object.

[0399] FIG. 38 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0400] According to one embodiment, FIG. 38 may illustrate an example of an operation that supports expansion of a selection area of ​​a cropped object.

[0401] As illustrated in Figure 38, examples of various inputs that set the area to be expanded (or the area for outpainting) in the object (e.g., the cropped area) can be presented.

[0402] example <3801> As illustrated, a user may expand an area based on an input (e.g., an input drawing the cropped area in a relatively circular shape) that draws a cropped area (e.g., an area for outpainting). In one embodiment, the electronic device (101) may provide a handler box expanded based on the cropped area based on the user input.

[0403] example <3803> and examples <3811> As illustrated, a user may expand an area based on an input of moving an expanded object provided based on a cropped area (e.g., an area for outpainting) in a handler box. In one embodiment, the electronic device (101) may provide a handler box expanded based on a cropped area based on a user input.

[0404] According to one embodiment, the electronic device (101) may provide visual information about an object to be created based on a cropped area in various ways (e.g., an area where a cropped object is to be filled with outpainting, etc.) when the area is expanded based on a user input. According to one embodiment, the electronic device (101) may, for example, <3811> As illustrated, based on the area being expanded, the expanded area (e.g., the truncated area) can be provided as is, in an empty state.

[0405] According to one embodiment, the electronic device (101) is, for example <3813> As illustrated, the extended area (e.g., the cropped area) can be provided with masking (e.g., a semi-transparent grid pattern). According to one embodiment, the electronic device (101) may include, for example <3815> As exemplified, a virtual object can be drawn and provided in real time in an extended area (e.g., a cropped area). In one embodiment, the act of actually filling in the cropped area can be performed by a generative artificial intelligence on the device or server.

[0406] FIGS. 39A and 39B are diagrams illustrating examples of operations supporting image editing in an electronic device according to one embodiment of the present disclosure.

[0407] According to one embodiment, FIGS. 39A and 39B may illustrate examples of operations that support expanding a selection area when an image includes a person, for example, when the object is a person.

[0408] In one embodiment, FIG. 39A illustrates an example of supporting expansion of a selection area when an object (3910) selected in an image (3900) is a person (e.g., a person) and there is no cropped portion in the face area of ​​the object (3910). In one embodiment, FIG. 39B illustrates an example of operation when an image (3950) includes a person object and there is a cropped portion in the face area of ​​the object.

[0409] As illustrated in FIG. 39A, if the separated, cropped object (3910) is a person, the electronic device (101) may recognize a face in the object (3910) and support expansion in a specified direction (e.g., downward) relative to the face. In one embodiment, the expansion area may expand the selection area in a manner corresponding to that described in the description with reference to FIGS. 37 and 38 (e.g., handler (3920)-based adjustment).

[0410] According to one embodiment, when the electronic device (101) moves the cropped object (3910) and expands the region, if the cropped object (3910) is a person, the electronic device (101) may recognize a face and operate to enable the region to expand downward. For example, the region where the cropped object (3910) is filled with outpainting may be the region (3930) located below the object (3910). For example, if the cropped object (3910) is a person, the electronic device (101) may block the region expansion to the left or right.

[0411] As illustrated in FIG. 39B, if the image (3950) is a cropped image of a face, an unintended facial result may be generated in the generative AI. According to one embodiment, the electronic device (101) may analyze the image (3950) during the image generation process, and based on the analysis result, if a cropped portion of the face is identified in the image (3950), the electronic device (101) may block the image (3950) without transmitting it to the generative AI. For example, the electronic device (101) may not perform the image generation process and may provide an error message (3960) to the user.

[0412] For example, the user, for example <3911> and examples <3913> As illustrated, the electronic device (101) may be requested to edit (e.g., rotate) the image (3950) and execute an image generation process. In one embodiment, the electronic device (101) may, in response to the image generation process request, analyze the image (3950), and, if a portion of the face in the image (3950) is identified, perform the image generation process without executing the image generation process. <3915> As exemplified, the error message (3960) can be displayed via a pop-up.

[0413] FIG. 40 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0414] According to one embodiment, FIG. 40 may illustrate an example of an operation of expanding a selection area of ​​an object in an image (4000) and selecting multiple objects as a bundled object (or group object).

[0415] As illustrated in FIG. 40, when an object (e.g., a first object (4010)) selected in an image (4000) is separated and moved, the electronic device (101) may provide a handler (4020) (e.g., a handler box and a function object) based on the surroundings of the first object (4010). According to one embodiment, the handler box of the handler (4020) may provide an expansion object (e.g., an up, down, left, and right expansion direction indicator (e.g., an arrow button)) that may expand a selection area of ​​the object. According to one embodiment, a user may use the expansion object of the handler box to move the handler box in at least one of the up, down, left, and right directions to expand the object selection area (e.g., expand the handler box based on dragging after selecting an expansion object).

[0416] In one embodiment, the example <4001> As illustrated, a user can use an extension object of the handler box to expand the handler box to include a second object (4040) that the user wishes to include in a selection area including a first object (4010). In one embodiment, the first object (4010) may represent an object that has been separated and moved from the image (4000). In one embodiment, a handler (4020) may be provided around the first object (4010). In one embodiment, the image (4000) may include a first masking area (4030) corresponding to the first object (4010), and a link may be established between the first object (4010) and the first masking area (4030).

[0417] According to one embodiment, the electronic device (101) can recognize the expansion of the handler box based on the expansion object of the handler box. According to one embodiment, the electronic device (101) can recognize the second object (4040) in the expansion direction (e.g., rightward direction) of the handler box with reference to the first masking area (4030) of the first object (4010) in the image (4000). For example, the electronic device (101) can recognize the second object (4040) adjacent to the first object (4010) (e.g., substantially, the first masking area (4030) in the drawing) in the expansion direction of the handler box with reference to the area (e.g., the first masking area (4030)) where the first object (4010) was originally located in the image (4000).

[0418] In one embodiment, the example <4003> As illustrated, when the second object (4040) is recognized, the electronic device (101) may separate the second object (4040) from the image (4000) (e.g., crop a portion corresponding to the second object (4040)) and move the second object (4040) to a selection area to include it in the selection area. For example, the electronic device (101) may set the first object (4010) and the second object (4040) as a single bundle object (or group object). According to one embodiment, the electronic device (101) may set and display a second masking area (4050) corresponding to the second object (4040) at the original location of the second object (4040) in the image (4000).

[0419] In one embodiment, when the first object (4010) and the second object (4040) operate as a bundle object, the first masking area (4030) and the second masking area (4050) may also be expanded and displayed as masking areas (e.g., bundle masking areas) corresponding to the bundle object, and a link may be established between the bundle object and the bundle masking area.

[0420] According to one embodiment, the electronic device (101) may generate and provide an expansion object capable of expanding a handler box when another object is recognized around a separated first object (4010). According to one embodiment, when a selection area is expanded based on the expansion object, the electronic device (101) may recognize a second object (4040) based on the range to which the selection area is expanded.

[0421] According to one embodiment, when a second object (4040) is recognized within an extended range, the electronic device (101) may include the second object (4040) in the selection area of ​​the first object (4010) and provide a handler box of the handler (4020) expanded by an area including the second object (4040). According to one embodiment, the electronic device (101) may not support expansion of the handler box when the second object (4020) is not recognized within the extended range, or when another masking area (e.g., a masking area of ​​another object that has already been separated) is recognized around the first object (4010) (e.g., around the first masking area (4030)).

[0422] According to one embodiment, the electronic device (101) may not provide an extended object if no other object is recognized around the separated first object (4010), or if another masking area (e.g., a masking area of ​​another object that has already been separated) is recognized around the first object (4010) (e.g., around the first masking area (4030)).

[0423] FIG. 41 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0424] According to one embodiment, FIG. 41 may illustrate an example of an operation of expanding a selection area of ​​an object in an image (4100) and selecting multiple objects as a bundled object (or group object).

[0425] As illustrated in FIG. 41, the electronic device (101) may provide a handler (4120) (e.g., a handler box and a function object) based on the surroundings of the first object (4110) when the object selected in the image (4100) (e.g., the first object (4110)) is separated and moved. According to one embodiment, the function object may include a drawing object. According to one embodiment, the user may switch to a drawing mode using the drawing object and expand the object selection area by drawing in the drawing mode. According to one embodiment, the drawing object may also operate to be provided when another object is recognized adjacent to the object in the selection area (e.g., the first object (4110)).

[0426] In one embodiment, the example <4101> As illustrated, a user may select a drawing object based on the handler (4120). In one embodiment, the electronic device (101) may switch to a drawing mode based on the selection of a drawing object.

[0427] In one embodiment, the example <4103> As illustrated, the electronic device (101) may hide and not display the handler (4120) based on the drawing mode transition. According to one embodiment, the user may perform drawing (4140) including an area corresponding to a second object (4150) to be additionally included in a selection area including a first object (4110) in the drawing mode. In one embodiment, the first object (4110) may represent an object that has been separated and moved from the image (4100). In one embodiment, the handler (4120) may be provided around the first object (4110). In one embodiment, the image (4100) may include a first masking area (4130) corresponding to the first object (4110), and a link may be established between the first object (4110) and the first masking area (4130).

[0428] According to one embodiment, the electronic device (101) may recognize the second object (4150) based on a location corresponding to a drawn area (e.g., to the right of the first object (4110) based on the user's line of sight) with reference to the first masking area (4130) of the first object (4110) in the image (4100) according to the drawing (4140). For example, the electronic device (101) may recognize the second object (4150) adjacent to the first object (4110) (e.g., substantially, the first masking area (4130) in the drawing) at a location corresponding to the drawing area based on the area where the first object (4110) was originally located in the image (4100) (e.g., the first masking area (4130)).

[0429] In one embodiment, the example <4105> As illustrated, when the second object (4150) is recognized, the electronic device (101) may separate the second object (4150) from the image (4100) (e.g., crop a portion corresponding to the second object (4150)) and move the second object (4150) to a selection area to include it in the selection area. For example, the electronic device (101) may set the first object (4110) and the second object (4150) as a single bundle object (or group object). According to one embodiment, the electronic device (101) may set and display a second masking area (4160) corresponding to the second object (4150) at the original location of the second object (4150).

[0430] In one embodiment, when the first object (4110) and the second object (4150) operate as a bundle object, the first masking area (4130) and the second masking area (4160) may also be expanded and displayed as masking areas (e.g., bundle masking areas) corresponding to the bundle object, and a link may be established between the bundle object and the bundle masking area.

[0431] According to one embodiment, the electronic device (101) may not expand the handler box if the second object (4150) is not recognized at a location corresponding to the area drawn according to the drawing (4140) (e.g., a right area of ​​the first masking area (4130) in the drawing), or if another masking area (e.g., a masking area of ​​another object that has already been separated) is recognized around the first object (4110) (e.g., around the first masking area (4130)).

[0432] FIG. 42 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0433] According to one embodiment, FIG. 42 may illustrate an example of an operation of expanding a selection area of ​​an object in an image (4200) and selecting multiple objects as a bundled object (or group object).

[0434] As illustrated in FIG. 42, the electronic device (101) may provide a handler (4220) (e.g., a handler box and a function object) based on the periphery of the first object (4210) when the first object (4210) is separated and moved in the image (4200). In one embodiment, the image (4200) includes a first masking area (4230) corresponding to the first object (4210), and a link may be established between the first object (4210) and the first masking area (4230).

[0435] In one embodiment, the example <4201> As illustrated, a user may individually separate a first object (4210) from an image (4200) through a designated input (e.g., a first input such as a long press) based on a first object (4210). According to one embodiment, the electronic device (101) may receive the first input and separate the first object (4210) from the image (4200) (e.g., crop a portion corresponding to the first object (4210)) based on the first input. According to one embodiment, the electronic device (101) may provide a handler (4220) based on the first object (4210) (e.g., a cropped portion) and mask an area in the image (4200) where the first object (4210) was originally located to provide it as a masking area (e.g., a first masking area (4230)).

[0436] In one embodiment, a user may select a second object (4240) through a designated input (e.g., a second input such as a tap) based on a second object (4240) that the user wishes to associate with a first object (4210) in an image (4200). In one embodiment, the electronic device (101) may receive the second input and, based on the second input, separate the second object (4240) from the image (4200) (e.g., crop a portion corresponding to the second object (4240)).

[0437] In one embodiment, the example <4203> As illustrated, the electronic device (101) may move the second object (4240) (e.g., a cropped portion) to the selection area and include it in the selection area. According to one embodiment, the electronic device (101) may set the first object (4210) and the second object (4240) included in the selection area as a single bundle object (or group object). According to one embodiment, the electronic device (101) may set and display a second masking area (4250) corresponding to the second object (4240) at the original location of the second object (4240) in the image (4200).

[0438] In one embodiment, when the first object (4210) and the second object (4240) operate as a bundle object, the first masking area (4230) and the second masking area (4250) may also be expanded and displayed as masking areas (e.g., bundle masking areas) corresponding to the bundle objects, and a link may be established between the bundle objects and the bundle masking areas.

[0439] FIG. 43 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0440] According to one embodiment, FIG. 43 may represent an example of a result image (4300) generated according to an image generation process (e.g., inpainting and / or outpainting) performed by generative artificial intelligence after image editing based on a selection area expansion operation as illustrated in FIGS. 40 to 42.

[0441] According to one embodiment, a user may perform image editing based on selection area expansion based on the performance of the operations exemplified in FIGS. 40 to 42, and request image generation based on the edited image. In one embodiment, the image generation request may be performed by a designated command input. For example, the user may request image generation (or completion of image editing) based at least on a designated object (e.g., a creation object) for image generation (e.g., regeneration or reconstruction), a designated voice command, and / or a designated motion gesture (e.g., shaking the electronic device (101). According to one embodiment, the electronic device (101) may determine an operation entry for generating a new image based on the edited image based on detecting a designated command input while performing image editing based on user interaction.

[0442] According to one embodiment, the electronic device (101) may perform a process (e.g., an image generation process) related to generating a new image based on an edited image based on receiving a specified command input. According to one embodiment, the electronic device (101) may interact with a user to perform the image generation process with operations corresponding to those described in the description with reference to FIGS. 17 and 18. According to one embodiment, the image generation process may include an operation of generating (e.g., regenerating or reprocessing) an image based on the edited image based on artificial intelligence (AI) (e.g., generative AI). In one embodiment, the image generation process may be provided on-device and / or server-based.

[0443] According to one embodiment, the electronic device (101) may obtain a result image (4300) based on an image generation process and display the result image (4300) on a display. According to one embodiment, the electronic device (101) may obtain a new image (e.g., a result image (4300)) in relation to an instruction (or prompt) related to image generation. For example, the electronic device (101) may obtain (e.g., generate) a new image (e.g., a result image (4300)) according to an image generation process (e.g., inpainting and / or outpainting based on an edited image) executed in relation to an instruction (or prompt) in on-device artificial intelligence. For example, the electronic device (101) may obtain (or receive) from the server an image (e.g., a result image (4300)) generated from the server according to an image generation process (e.g., inpainting and / or outpainting based on an edited image) executed in connection with instructions (or prompts) from the server artificial intelligence.

[0444] According to one embodiment, the electronic device (101) may display a result image (4300) on a display. According to one embodiment, the result image (4300) may be, for example, an image in which a blank area (e.g., a masking area) generated by object editing and an object is reconstructed. For example, a masking area may be created (e.g., inpainted), and separate bundled objects (e.g., a first object and a second object) may be included in a background image (e.g., combined) and provided.

[0445] According to one embodiment, the result image (4300) may be provided with N (e.g., at least 4) or more result images (e.g., multiple result images). For example, the electronic device (101) may obtain multiple result images (4350) (e.g., a first result image, a second result image, a third result image, and a fourth result image) based on generative artificial intelligence, and may provide a preview (e.g., a thumbnail) corresponding to each of the multiple result images (4350) as a bottom menu of the result image (4300). According to one embodiment, the user may select a desired image from among the multiple result images (4350).

[0446] According to one embodiment, the electronic device (101) may receive an input for selecting one image from a plurality of result images (4350). According to one embodiment, the electronic device (101) may display the image selected based on the input as a result image (4300) on the display. According to one embodiment, the electronic device (101) may display the result image (4300) and complete image generation based on a designated input related to completion of image generation (e.g., input received via a “Done” button).

[0447] FIG. 44 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0448] According to one embodiment, FIG. 44 may illustrate an example of an operation of separating a bundled object into independent objects in an image and separating a selection area of ​​the bundled object based on the object separation.

[0449] In one embodiment, the example <4401> As illustrated in , the electronic device (101) and the user may perform an operation of setting the first object (4410) and the second object (4420) as a single bundle object by corresponding operations described in the description portions referring to FIGS. 40 to 42. According to one embodiment, the electronic device (101) may set and display a first masking area (4415) corresponding to the first object (4410) at the original position of the first object (4410) in the image, and may set and display a second masking area (4425) corresponding to the second object (4420) at the original position of the second object (4420). According to one embodiment, the electronic device (101) may provide the first masking area (4415) and the second masking area (4425) as bundle masking areas for the bundle object of the first object (4410) and the second object (4420).

[0450] According to one embodiment, the electronic device (101) may provide a handler (4440) (e.g., a handler box and a function object) based on the periphery of a bundled object. According to one embodiment, the function object may include a cropping object (or a cut object). According to one embodiment, the user may use the cropping object to switch to an object separation mode (or a cut mode) and separate an object from an object selection area in the object separation mode. According to one embodiment, the cut object may also be provided when a bundled object of multiple objects is recognized.

[0451] In one embodiment, the example <4403> As illustrated in FIG. 40 to FIG. 42, a user may include a third object (4430) in a selection area of ​​a bundle object including a first object (4410) and a second object (4420) by performing any one of the actions based on FIGS. 40 to 42. According to one embodiment, the electronic device (101) may set the first object (4410), the second object (4420), and the third object (4430) as one bundle object. According to one embodiment, the electronic device (101) may set and display a third masking area (4435) corresponding to the third object (4430) at the original location of the third object (4430).

[0452] In one embodiment, a bundle object of a first object (4410), a second object (4420), and a third object (4430) may represent an object that has been separated and moved from an image. In one embodiment, a handler (4440) may be provided around the bundle object. In one embodiment, the function object of the handler (4440) may include a cropping object.

[0453] In one embodiment, the image includes bundle masking regions (e.g., a first masking region (4415) corresponding to the first object (4410), a second masking region (4425) corresponding to the second object (4420), and a third masking region (4435) corresponding to the third object (4430)) corresponding to bundle objects (e.g., a first object (4410), a second object (4420), and a third object (4430)), and a link can be established between the bundle objects and the bundle masking regions. For example, when a first object (4410), a second object (4420), and a third object (4430) operate as a bundle object, the first masking area (4415), the second masking area (4425), and the third masking area (4435) may also be expanded and displayed as bundle masking areas corresponding to the bundle objects, and a link may be established between the bundle objects and the bundle masking areas.

[0454] In one embodiment, the example <4405> As illustrated, the user can separate the third object (4430) from the bundled objects included in the selection area, thereby separating the third object (4430) from the bundled objects of the first object (4410) and the second object (4420). According to one embodiment, when the user wants to modify the selection area in which two or more objects are combined, the user can separate the object at a desired location using the crop object of the handler (4440).

[0455] According to one embodiment, a user may select a first crop object based on a handler (4440) of a bundle object. According to one embodiment, the electronic device (101) may switch to an object separation mode (or cropping mode) based on the selection of the first crop object. According to one embodiment, the electronic device (101) may provide a divider (4445) based on the handler (4440) (e.g., a handler box) based on the switching to the object separation mode.

[0456] According to one embodiment, a user may select at least one object (or area) (e.g., a third object (4430)) to be separated from a bundled object using a divider (4445) in object separation mode. For example, the user may manipulate the divider (4445) (e.g., moving the divider (4445) left and right) to include the third object (4430) in a selection area to be separated from the bundled object. According to one embodiment, the user may select the third object (4430) to be separated from the bundled object using the divider (4445) (e.g., including it in the selection area) and select a second cropping object. In one embodiment, the second cropping object and the divider (4445) may operate in conjunction with each other. For example, in object separation mode, the second cropping object may move in response to the left and right movement of the divider (4445).

[0457] In one embodiment, the example <4407> As illustrated, when a second cropping object is selected, the electronic device (101) may separate a third object (4430) of a set selection area from a bundle object. According to one embodiment, the electronic device (101) may separate a third masking area (4435) corresponding to the third object (4430) from the bundle masking area based on the separation of the third object (4430) from the bundle object. According to one embodiment, the electronic device (101) may display a handler (4450) around the third object (4430) separated from the bundle object, and may set and display (e.g., dotted line or highlighting) a third masking area (4435) corresponding to the third object (4430).

[0458] In one embodiment, a handler (4450) generated based on a third object (4430) may not include a crop object because the third object (4430) is a single object. For example, if the selected object is a single object, the handler (4450) for the single object may not include a crop object. For example, a crop object may be provided if the object is a bundle of multiple bundled objects.

[0459] FIG. 45 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0460] According to one embodiment, FIG. 45 may illustrate an example of an operation that supports modification of a result image (4550) through an artificial intelligence eraser (AI eraser) function based on the result image (4550) according to an image generation process (e.g., inpainting and / or outpainting).

[0461] In one embodiment, the example <4501> As illustrated, a user may select (or designate) at least one target object (e.g., object (4510)) for image editing in an image (4500). According to one embodiment, the electronic device (101) may select at least one object (4510) corresponding to a user input in the image (4500) as a target object for image editing.

[0462] In one embodiment, the example <4503> As illustrated, the electronic device (101) may crop (e.g., separate from the image (4500)) and move an object (4510) selected from the image (4500) based on a specified input (e.g., a tap-and-hold gesture). According to one embodiment, the electronic device (101) may select the object (4510) in response to a first input (e.g., an input for selecting a region portion of the object (4510) in the image (4500), crop a corresponding portion corresponding to the object (4510) in response to a second input (e.g., a tap-and-move gesture for selecting and moving the object (4510), and move the cropped portion (e.g., the object (4510)).

[0463] In one embodiment, the example <4503> As illustrated, the electronic device (101) may provide a handler (4520) (e.g., a handler box and a function object) based on the surroundings of the object (4510) when the object (4510) is selected and moved. According to one embodiment, the electronic device (101) may display a mask (e.g., a grid pattern) in the area where the object (4510) was originally located (e.g., a masking area (4515)) when the object (4510) is selected and moved.

[0464] In one embodiment, the example <4505> As exemplified, the electronic device (101) can interact with a user to perform image editing and perform an image generation process based on the edited image.

[0465] In one embodiment, a user may edit an object (4510) based on a handler (4520) (e.g., tilt, straighten, vertically, and / or horizontally) and request image generation based on a designated object (4540) for image generation (e.g., a generation object).

[0466] According to one embodiment, the electronic device (101) may perform a process (e.g., an image generation process) related to generating a new image based on the edited image (4500) based on receiving an input through a designated object (4540). According to one embodiment, the electronic device (101) may provide a related interface to the user while the image generation process is in progress. For example, the related interface may include a guide object (4530) that notifies that the image generation process is in progress. For example, the related interface may be in the form of dimming a background including an image and providing a guide object (4530) on the dimmed background.

[0467] In one embodiment, the image generation process may include generating (e.g., regenerating or reprocessing) an image based on an image edited using artificial intelligence (AI) (e.g., generative AI). In one embodiment, the image generation process may be provided on-device and / or server-based.

[0468] In one embodiment, the example <4507> As illustrated, the electronic device (101) may obtain a result image (4550) based on an image generation process and display the result image (4550) on a display. According to one embodiment, the electronic device (101) may obtain a new image (e.g., a result image (4300)) in relation to an instruction (or prompt) related to image generation. For example, the electronic device (101) may obtain (e.g., generate) a new image (e.g., a result image (4550)) according to an image generation process (e.g., inpainting and / or outpainting based on an edited image) executed in relation to an instruction (or prompt) in on-device artificial intelligence. For example, the electronic device (101) may obtain (or receive) from the server an image (e.g., a result image (4550)) generated from the server according to an image generation process (e.g., inpainting and / or outpainting based on an edited image) executed in connection with instructions (or prompts) from the server artificial intelligence.

[0469] According to one embodiment, the electronic device (101) may display a result image (4550) on the display. According to one embodiment, the result image (4550) may be an image in which, for example, a blank area (e.g., a masking area (4515)) generated by object editing is filled in, and an object and / or a background image is reconstructed. For example, a masking area (4515) may be created (e.g., inpainted) and the object (4510) may be provided combined with a background image. According to one embodiment, an object (or area) that is not desired (or unintended) by the user may be added to the result image (4550).

[0470] According to one embodiment, the electronic device (101) may analyze the resulting image (4550) generated according to the image generation process and determine whether correction is necessary. For example, the electronic device (101) may determine that correction is necessary if a new object (or region) (e.g., an object (or region) not desired (or unintended) by the user) is added (or detected) to the reconstructed object separated from the background image.

[0471] According to one embodiment, the electronic device (101) may provide an AI eraser function based on determining the correction of the result image (4550). For example, the electronic device (101) may provide an AI eraser object (4560) (or indicator or button) capable of activating the AI ​​eraser function as a bottom menu of the result image (4550). According to one embodiment, the user may execute the AI ​​eraser function based on selection of the AI ​​eraser object (4560).

[0472] In one embodiment, the AI ​​eraser function may represent a function that selects and erases unnecessary objects from an image, such as object erasers, shadow erasers, and / or light reflection erasers. For example, a user may execute the AI ​​eraser function to select an object to be corrected in an image. According to one embodiment, the electronic device (101) may analyze the selected object using AI based on the user's object selection and erase it from the image.

[0473] According to one embodiment, the electronic device (101) can complete image generation based on receiving input through a designated object (4570) related to completion of image generation after the result image (4550) is displayed or after the result image (4550) is corrected.

[0474] FIG. 46 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0475] According to one embodiment, FIG. 46 may illustrate an example of an operation of separating an object from an image and restoring the object based on a masking area corresponding to the object. According to one embodiment, FIG. 46 may illustrate an example of a case where there is one object (4610) separated from an image and one masking area (4615) corresponding to the one object (4610) is set.

[0476] In one embodiment, the example <4601> and examples <4603> As illustrated, a user may select a first object (4610) from an image, separate it from the image, and move the first object (4610) to another area on the image. According to one embodiment, the electronic device (101) may separate the first object (4610) from the image (e.g., crop a portion corresponding to the first object (4610)) based on a user input, and move and display the first object (4610) (e.g., a cropped portion) to a position corresponding to the user input.

[0477] According to one embodiment, the electronic device (101) may provide a first handler (4620) around the first object (4610). According to one embodiment, the electronic device (101) may set and display (e.g., display a grid pattern, dotted line, or highlighting) a first masking area (4615) corresponding to the first object (4610) based on the original location of the first object (4610) in the image (e.g., the area where the first object (4610) was originally located on the image). According to one embodiment, the electronic device (101) may set a link between the first object (4610) and the first masking area (4615). In one embodiment, the first masking area (4615) may support a function of controlling the first object (4610) and may visually display information about the area where the first object (4610) was originally located.

[0478] In one embodiment, the example <4605> As illustrated, a user may select a first masking area (4615) corresponding to a first object (4610). According to one embodiment, the electronic device (101) may provide a second handler (4630) around the first masking area (4615) based on an input of selecting the first masking area (4615). In one embodiment, the electronic device (101) may not display the first handler (4620) provided around the first object (4610) by making it hidden.

[0479] In one embodiment, the second handler (4630) of the first masking area (4615) may include an undo (or revert) object. In one embodiment, the undo object may include an object that supports a function to restore (e.g., revert) to a previous state. For example, the undo object may include an object that supports a function (e.g., a cancel function) to undo a previously executed editing command and return to the state prior to issuing the editing command (e.g., the state prior to execution).

[0480] In one embodiment, the example <4605> Wow, example <4607> As illustrated, after editing (e.g., moving) a first object (4610), a user may want to restore (e.g., restore to its original position) the edited (e.g., moved) first object (4610) in the image. For example, the user may use the second handler (4630) (e.g., undo object) of the first masking area (4615) to restore the moved first object (4610) to its initial state (e.g., the state before editing (e.g., moving) the image). For example, the user may select the undo object of the second handler (4630). According to one embodiment, in response to an input of selecting the undo object of the second handler (4630), the electronic device (101) may restore the first object (4610) to the first masking area (4615) corresponding to the area where the first object (4610) was originally located in the image, thereby restoring the image to its pre-editing state.

[0481] FIG. 47 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0482] According to one embodiment, FIG. 47 may illustrate an example of an operation of removing an object separated from an image and restoring the object based on a masking area corresponding to the removed object. According to one embodiment, FIG. 47 may illustrate an example of a case where there is one object (4710) separated from an image and one masking area (4715) corresponding to one object (4710) is set.

[0483] In one embodiment, the example <4701> and examples <4703> As illustrated, a user may select a first object (4710) in an image to separate it from the image and remove the first object (4710) from the image. According to one embodiment, the user may remove the first object (4710) from the image by using an eraser object (or a trash can object) of a first handler (4720) generated around the first object (4710). According to one embodiment, the electronic device (101) may separate the first object (4710) from the image (e.g., crop a portion corresponding to the first object (4710)) based on an input to remove the first object (4710), and may remove (or hide) the first object (4710) (e.g., cropped portion) based on the user input so that it is not displayed.

[0484] According to one embodiment, when the first object (4710) is separated, the electronic device (101) may set and display (e.g., display a grid pattern, dotted line, or highlighting) a first masking area (4715) corresponding to the first object (4710) based on the original location of the first object (4710) in the image (e.g., the area where the first object (4710) was originally located on the image). According to one embodiment, the electronic device (101) may set a link between the first object (4710) and the first masking area (4715). In one embodiment, the first masking area (4715) may support a function of controlling the first object (4710) and may visually display information about the area where the first object (4710) was originally located.

[0485] In one embodiment, the example <4703> As illustrated, a user may select a first masking area (4715) corresponding to a first object (4710). According to one embodiment, the electronic device (101) may provide a second handler (4730) around the first masking area (4715) based on an input of selecting the first masking area (4715).

[0486] In one embodiment, the second handler (4730) of the first masking area (4715) may include an undo (or revert) object. In one embodiment, the undo object may include an object that supports a function to restore (e.g., revert) to a previous state. For example, the undo object may include an object that supports a function (e.g., a cancel function) to undo a previously executed editing command and return to the state prior to issuing the editing command (e.g., the state prior to execution).

[0487] In one embodiment, the example <4703> and examples <4705> As illustrated, after editing (e.g., removing) a first object (4710), a user may want to restore (e.g., restore to its original state) the edited (e.g., removed) first object (4710) from the image. For example, the user may use the second handler (4730) (e.g., undo object) of the first masking area (4715) to restore the removed second object (4710) to its initial state (e.g., before image editing (e.g., moving and removing)). For example, the user may select the undo object of the second handler (4730). According to one embodiment, in response to an input of selecting the undo object of the second handler (4730), the electronic device (101) may restore the first object (4710) to the first masking area (4715) corresponding to the area where the first object (4710) was originally located in the image, thereby restoring the image to its pre-editing state.

[0488] FIGS. 48A and 48B are diagrams illustrating examples of operations supporting image editing in an electronic device according to one embodiment of the present disclosure.

[0489] According to one embodiment, FIGS. 48A and 48B may illustrate an example of an operation of separating a plurality of objects (e.g., a first object (4810), a second object (4820), and a third object (4830)) from an image, and restoring a target object (e.g., at least one object) based on masking regions (e.g., a first masking region (4815), a second masking region (4825), and a third masking region (4835)) corresponding to each of the plurality of objects (4810, 4820, 4830). In one embodiment, FIGS. 48A and 48B may illustrate an example in which three objects (4810, 4820, 4830) are separated from an image, and three masking areas (4815, 4825, 4835) are set to correspond to the three objects (4810, 4820, 4830), respectively. In one embodiment, FIGS. 48A and 48B may illustrate an example in which the first object (4810), the second object (4820), and the third object (4830) are single objects, not bundled objects.

[0490] In one embodiment, the example <4801> and examples <4803> As illustrated, the user may select a first object (4810), a second object (4820), and a third object (4830) from the image, respectively, to separate them from the image, and move the first object (4810), the second object (4820), and the third object (4830) to different areas on the image, respectively. According to one embodiment, the electronic device (101) may separate the first object (4810), the second object (4820), and the third object (4830) from the image (e.g., crop a portion corresponding to each of the first object (4810), the second object (4820), and the third object (4830)) based on a user input, and move and display the first object (4810), the second object (4820), and the third object (4830) to their respective positions corresponding to the user input.

[0491] According to one embodiment, the number of movable objects in an image may be one or more. According to one embodiment, the electronic device (101) may limit the number of movable objects when supporting movement of objects. For example, the electronic device (101) may limit the number of movable objects according to the settings of the electronic device (101). For example, the number of movable objects may be limited to 3, 5, or 10. According to one embodiment, the electronic device (101) may not include removed objects in the count of movable objects, and may limit the number by only counting the number of objects whose inclination, position, and / or size have changed. According to one embodiment, the electronic device (101) may provide feedback (e.g., a toast message) for a user input for selecting an additional object when the number is limited.

[0492] According to one embodiment, the electronic device (101) may provide a first handler (4840) around the last selected object (e.g., the first object (4810)). According to one embodiment, the electronic device (101) may set and display a first masking area (4815) corresponding to the first object (4810) based on the original location of the first object (4810) in the image (e.g., the area where the first object (4810) was originally located on the image). According to one embodiment, the electronic device (101) may set and display a second masking area (4825) corresponding to the second object (4820) based on the original location of the second object (4820) in the image (e.g., the area where the second object (4820) was originally located on the image). According to one embodiment, the electronic device (101) may set and display a third masking area (4835) corresponding to the third object (4830) based on the original location of the third object (4830) in the image (e.g., the area where the third object (4830) was originally located on the image).

[0493] According to one embodiment, the electronic device (101) can establish a link between a first object (481) and a first masking area (4815), a link between a second object (4820) and a second masking area (4825), and a link between a third object (4830) and a third masking area (4835).

[0494] In one embodiment, each of the masking areas (4815, 4825, 4835) supports a function of controlling an object (4810, 4820, 4830) corresponding to the masking area (4815, 4825, 4835), respectively, and can visually display information about an area where the object was originally located. According to one embodiment, a masking area (e.g., the first masking area (4815)) corresponding to a currently selected object (e.g., the first object (4810)) among the first object (4810), the second object (4820), or the third object (4830) can be highlighted (e.g., dotted line or highlighted) to provide a selection state. For example, the electronic device (101) can support a user to easily identify the original location of the object through highlighting (or selection indication) (e.g., dotted line or highlighting) of the masking area of ​​the selected object.

[0495] In one embodiment, the example <4805> As illustrated, a user may select a masking area of ​​an object among edited (e.g., moved) objects to be restored (e.g., moved) to its original position. According to one embodiment, the user may select a third masking area (4835) corresponding to a third object (4830). According to one embodiment, the electronic device (101) may provide a second handler (4850) around the third masking area (4835) based on an input for selecting the third masking area (4835). In one embodiment, the electronic device (101) may hide the first handler (4840) provided around the first object (4810) and not display it.

[0496] In one embodiment, the second handler (4850) of the third masking area (4835) may include an undo (or revert) object. In one embodiment, the undo object may include an object that supports a function to restore (e.g., revert) to a previous state. For example, the undo object may include an object that supports a function (e.g., a cancel function) to undo a previously executed editing command and return to the state prior to issuing the editing command (e.g., the state prior to execution).

[0497] In one embodiment, the example <4807> and examples <4809> As illustrated, after editing (e.g., moving) a third object (4830), a user may want to restore (e.g., restore to its original position) the edited (e.g., moved) third object (4830) in the image. For example, the user may use the second handler (4850) (e.g., undo object) of the third masking area (4835) to restore the moved third object (4830) to its initial state (e.g., the state before editing (e.g., moving) the image). For example, the user may select the undo object of the second handler (4850). According to one embodiment, in response to an input of selecting the undo object of the second handler (4850), the electronic device (101) may restore the third object (4830) to the third masking area (4835) corresponding to the area where the third object (4830) was originally located in the image, thereby restoring the image to its pre-editing state. According to one embodiment, a first object (4810), a first masking area (4815) corresponding to the first object (4810), a second object (4820), and a second masking area (4825) corresponding to the second object (4820) may remain separated, and a third object (4830) may be restored by moving it to its original position.

[0498] According to one embodiment, the electronic device (101) may include an operation to interact with a user and, based on the restoration operation described above, restore and provide to the user a moved first object (4810) and / or a moved second object (4820).

[0499] FIGS. 49A and 49B are diagrams illustrating examples of operations supporting image editing in an electronic device according to one embodiment of the present disclosure.

[0500] According to one embodiment, FIGS. 49A and 49B may illustrate an example of an operation of separating a plurality of objects (e.g., a first object (4910), a second object (4920), and a third object (4930)) from an image, and restoring a target object (e.g., at least one object) based on masking regions (e.g., a first masking region (4915), a second masking region (4925), and a third masking region (4935)) corresponding to each of the plurality of objects (4910, 4920, 4930). In one embodiment, FIGS. 49A and 49B may illustrate an example in which three objects (4910, 4920, 4930) are separated from an image, and three masking areas (4915, 4925, 4935) corresponding to the three objects (4910, 4920, 4930) are set, respectively. In one embodiment, FIGS. 49A and 49B may illustrate an example in which the first object (4910), the second object (4920), and the third object (4930) are single objects, not bundled objects.

[0501] In one embodiment, the example <4901> and examples <4903> As illustrated, the user can select a first object (4910), a second object (4920), and a third object (4930) from the image, respectively, to separate them from the image, and remove the first object (4910), the second object (4920), and the third object (4930) from the image, respectively.

[0502] In one embodiment, the example <4903> , example <4905> and examples <4907> As illustrated, a user can remove a first object (4910) from an image by using an eraser object (or trash can object) of a first handler (4940) generated around the first object (4910). In one embodiment, a user can remove a second object (4920) from an image by using an eraser object (or trash can object) of a second handler (4950) generated around the second object (4920). In one embodiment, a user can remove a third object (4930) from an image by using an eraser object (or trash can object) of a third handler (4960) generated around the third object (4930).

[0503] According to one embodiment, the electronic device (101) may separate a first object (4910), a second object (4920), and a third object (4930) from an image based on a user input, and may remove (or hide) the separated first object (4910), second object (4920), and third object (4930) so as not to display them.

[0504] According to one embodiment, when separating a first object (4910), the electronic device (101) may set and display a first masking area (4915) corresponding to the first object (4910) based on the original location of the first object (4910) in the image (e.g., the area where the first object (4910) was originally located on the image). According to one embodiment, when separating a second object (4920), the electronic device (101) may set and display a second masking area (4925) corresponding to the second object (4920) based on the original location of the second object (4920) in the image (e.g., the area where the second object (4920) was originally located on the image). According to one embodiment, when the third object (4930) is separated, the electronic device (101) may set and display a third masking area (4935) corresponding to the third object (4930) based on the original location of the third object (4930) in the image (e.g., the area where the third object (4030) was originally located on the image).

[0505] According to one embodiment, the electronic device (101) can establish a link between a first object (4910) and a first masking area (4915), a link between a second object (4920) and a second masking area (4925), and a link between a third object (4930) and a third masking area (4935).

[0506] In one embodiment, each of the masking areas (4915, 4925, 4935) supports a function of controlling an object (4910, 4920, 4930) corresponding to the masking area (4915, 4925, 4935), and can visually display information about an area where the object was originally located. According to one embodiment, the electronic device (101) can support a user to easily identify the original location of the object by highlighting (or selecting) (e.g., dotted line or highlighting) the masking area of ​​the selected object.

[0507] In one embodiment, the example <4909> Wow, example <4911> As illustrated, a user may select a masking area of ​​an object among edited (e.g., removed) objects to be restored to its original position. In one embodiment, the user may select a first masking area (4915) corresponding to a first object (4910). In one embodiment, the electronic device (101) may provide a fourth handler (4970) around the first masking area (4915) based on an input for selecting the first masking area (4915).

[0508] In one embodiment, the fourth handler (4970) of the first masking area (4915) may include an undo (or revert) object. In one embodiment, the undo object may include an object that supports a function to restore (e.g., revert) to a previous state. For example, the undo object may include an object that supports a function (e.g., a cancel function) to undo a previously executed editing command and return to the state prior to issuing the editing command (e.g., the state prior to execution).

[0509] In one embodiment, the example <4911> and examples <4913> As illustrated, after editing (e.g., removing) a first object, a user may want to restore (e.g., restore to its original position) the edited (e.g., removed) first object (4910) from the image. For example, the user may use the seventh handler (4970) (e.g., undo object) of the first masking area (4915) to restore the removed first object (4910) to its initial state (e.g., the state before editing (e.g., removing) the image). For example, the user may select the undo object of the seventh handler (4970). According to one embodiment, in response to an input of selecting the undo object of the seventh handler (4970), the electronic device (101) may restore the first object (4910) to the first masking area (4915) corresponding to the area where the first object (4910) was originally located in the image, thereby restoring the image to its pre-editing state. In one embodiment, the second object (4920) and the third object (4930) may remain in a removed state, and the first object (4910) may be restored by moving it to its original position.

[0510] According to one embodiment, the electronic device (101) may interact with the user to provide the user with an action of restoring the removed second object (4920) and / or the removed third object (4930) based on the restoration action described above.

[0511] FIG. 50 is a diagram illustrating an example of an operation supporting image editing in an electronic device according to one embodiment of the present disclosure.

[0512] According to one embodiment, as illustrated in FIG. 50, the electronic device (101) may display an image selected by a user on a display. According to one embodiment, the image may be provided through a designated user interface related to image editing. According to one embodiment, based on the execution of image editing on the selected image, the electronic device (101) may enter an editing mode (e.g., an artificial intelligence mode or an artificial intelligence editing mode) and provide a user interface of the editing mode including the selected image. According to one embodiment, based on the entry into the editing mode, the electronic device (101) may change to a state in which the tilt of the image can be adjusted.

[0513] According to one embodiment, the electronic device (101) may provide a cropper (5000) (or image cropper) based on an image to indicate that the edit mode has been entered based on the entry into the edit mode. According to one embodiment, the cropper (5000) may or may not be displayed depending on the settings of the electronic device (101). According to one embodiment, FIG. 50 may illustrate an example of a screen and its operation when the cropper (5000) is provided.

[0514] According to one embodiment, when generating a background (e.g., outpainting) around an object attached to the edge of an image, the electronic device (101) may provide an extension area for background generation based on the outside of the image (or cropper (5000)), rather than the inside of the image. According to one embodiment, the electronic device (101) may display an area (e.g., a background generation area or a background extension area) where the background area of ​​the image is expected to be expanded to the outside of the image, rather than the inside of the cropper (5000).

[0515] In one embodiment, a user may perform image editing (e.g., background expansion or outpainting). In one embodiment, the electronic device (101) may display an image and enter an editing mode (e.g., an AI mode or an AI editing mode) to support image editing based on user input. In one embodiment, the electronic device (101) may support editing to recreate an image by background expansion of the image in the editing mode, and may support background expansion based on various designated inputs related to background expansion (e.g., inputs related to changing the tilt of the image (2201) and / or manipulation of the cropper (5000).

[0516] In one embodiment, a user may set a background generation area based on input specified in the edit mode. For example, the user may set an area (e.g., a background generation area or a background expansion area) into which the background area is expected to expand based on the specified input.

[0517] In one embodiment, the specified input may include an input based on a cropper (5000) that is fit to the image, an input of a specified touch gesture on the image (e.g., a touch gesture for rotating the image left or right based on a two-finger touch), and / or an input of manipulating an editing tool in a bottom menu (e.g., an image rotation tool or a dialer). For example, the specified input may include an interaction of selecting a background expansion button, an interaction of tapping and moving the image to either side while it is fit to the cropper (5000), an interaction of reducing the size of the image after pinch zooming out, and / or an interaction of adjusting the tilt of the image.

[0518] According to one embodiment, the electronic device (101) may mask and provide an expandable area based on a specified input. For example, the electronic device (101) may mask an area where a background area is expected to expand based on a specified input (e.g., a background generation area or a background expansion area) and visually provide the masked area to the user.

[0519] According to one embodiment, the electronic device (101) may generate and display a masking area representing an area where a background may be generated around the periphery of an image based on a specified input. In one embodiment, the masking area may represent an expandable area around the image (e.g., at least one of the top / bottom / left / right areas of the image). According to one embodiment, the masking area may be defined as about n% (e.g., n is a natural number) of the image, such as about 120% of the original, about 140% of the original, and / or about 160% of the original, based on the specified input. According to one embodiment, the masking area may be larger than an area displayed on the display of the electronic device (101).

[0520] According to one embodiment, when the electronic device (101) executes the background expansion function (or background expansion mode), it may provide a masking area (or an expandable area) (e.g., at least one area among the upper / lower / left / right areas of the image) around an image, and a real-time change status of the masking area (e.g., the degree to which the masking area is displayed) in response to a specified input.

[0521] According to one embodiment, the electronic device (101) can show the user visual information about the area where the image is to be generated (e.g., a masking area), and the user can check and continuously adjust the composition related to the resulting image to be generated in real time.

[0522] FIG. 51 is a diagram illustrating an example of an operation that supports image editing in an electronic device according to one embodiment of the present disclosure.

[0523] Referring to Figure 51, an example <5101> As illustrated, the electronic device (101) can display an image (5100) (e.g., a landscape photo, a portrait photo, a background photo, an object photo, or an animal photo) on the display.

[0524] In one embodiment, an example <5103> As illustrated, the electronic device (101) may display an image (5100) and enter an editing mode (e.g., an artificial intelligence mode or an artificial intelligence editing mode) to support image editing. According to one embodiment, the electronic device (101) may display a cropper (5110) fitted to the image (5100) when entering the editing mode. According to one embodiment, the electronic device (101) may provide an editing tool (5120) (e.g., an image rotation tool or a dialer) to support image editing when entering the editing mode. In one embodiment, the editing tool (5120) may provide a user input and provide a real-time change state of the image (5100) (e.g., information on a rotation state (e.g., tilt or angle information)) in response to image editing based on the user input.

[0525] In one embodiment, an example <5105> As illustrated in , the user may perform a touch gesture input (e.g., a touch gesture for rotating the image left or right based on a two-finger touch) on the image (5100) or a touch gesture input (e.g., a touch gesture for manipulating the editing tool (5120) (e.g., an image rotation tool or a dialer)) to manipulate the editing tool (5120). According to one embodiment, the electronic device (101) may change the inclination of the image (5100) based on the user input (e.g., an input on the image or an input based on the editing tool (5120). According to one embodiment, FIG. 51 may show an example of a state in which the image (5100) is rotated left by the user input. According to one embodiment, in the editing mode, when the electronic device (101) changes the inclination of the image (5100) based on the user input, the electronic device (101) may hide the cropper (5120) fitted to the image (5100) and not display it.

[0526] According to one embodiment, the electronic device (101) may display an area where a background area is expected to be expanded (e.g., a background generation area or a background expansion area) based on a change in the tilt of the image (5100). For example, the electronic device (101) may generate and display a masking area indicating an area where a background may be generated around the outer periphery of the image (5100) in response to a rotation (e.g., a change in the tilt) of the image (5100) based on a user input. In one embodiment, the masking area may indicate an expandable area around the image (5100) (e.g., at least one of the upper / lower / left / right areas of the image). According to one embodiment, the electronic device (101) may set an outer area of ​​the image (5100) as a background expansion area.

[0527] In one embodiment, the electronic device (101) may provide real-time change status of the image (5100) (e.g., change status according to left rotation progress) based on the editing tool (5120) in response to rotation of the image based on user input. For example, the electronic device (101) may provide information on the rotation status of the image (5100) (e.g., tilt or angle information) (e.g., current position value of the dialer (e.g., tilt angle between about -15 degrees and about 15 degrees).

[0528] In one embodiment, a user may request image generation while adjusting the tilt of the image. In one embodiment, the image generation request may be performed by a designated command input. For example, the user may request image generation based at least on a designated object (5130) for image generation (or regeneration) (e.g., a generation object), a designated voice command, and / or a designated motion gesture (e.g., shaking the electronic device (101). In one embodiment, the electronic device (101) may determine to enter an operation to generate a new image based on the edited image based on receiving a designated command input while performing image editing based on interaction with the user.

[0529] In one embodiment, the electronic device (101) may perform a process (e.g., an image generation process) related to generating a new image based on an edited image (e.g., a tilt-adjusted image) based on receiving a specified command input. In one embodiment, the image generation process may include an operation of generating (e.g., regenerating or reprocessing) an image based on the edited image based on artificial intelligence (AI) (e.g., generative AI). In one embodiment, the image generation process may be provided on-device and / or server-based.

[0530] In one embodiment, the example <5107> As illustrated, the electronic device (101) may display a result image (5150) on the display based on the image generation process. According to one embodiment, the electronic device (101) may obtain a new image (e.g., result image (5150)) in relation to an instruction (or prompt) related to image generation. For example, the electronic device (101) may obtain (e.g., generate) a new image (e.g., result image (5150)) in accordance with an image generation process (e.g., inpainting and / or outpainting based on an edited image) executed in relation to an instruction (or prompt) in on-device artificial intelligence. For example, the electronic device (101) may obtain (or receive) an image (e.g., result image (5150)) generated from a server in accordance with an image generation process (e.g., inpainting and / or outpainting based on an edited image) executed in relation to an instruction (or prompt) in server artificial intelligence.

[0531] In one embodiment, the electronic device (101) may display a result image (5150) on a display. In one embodiment, the result image (5150) may be, for example, an image in which an expanded blank area (e.g., a masking area) is filled in according to the inclination of the image (5100), and an object and / or background image is reconstructed. For example, the masking area may be provided by generating (e.g., outpainting).

[0532] FIGS. 52 and 53 are drawings illustrating examples of operations supporting image editing in an electronic device according to one embodiment of the present disclosure.

[0533] According to one embodiment, FIGS. 52 and 53 may illustrate an example operation that supports adjustment of an extended area (5240) (e.g., a masking area) using a cropper (5210) fitted to an image (5200).

[0534] Example of Fig. 52 <5201> , example <5203> and examples <5205> As illustrated, a user can use the cropper (5210) to set the size to which the image (5200) is expanded and / or the direction in which the image (5200) is expanded. According to one embodiment, FIG. 52 illustrates an example of a state in which an image (5200) is expanded to the right by user input using the cropper (5210).

[0535] In one embodiment, the example <5205> As illustrated, the user can select the right boundary (e.g., the right boundary of the cropper (5210) based on the user's line of sight) of the cropper (5210) fitted to the image (5200) and move it in the right direction. For example, the user can perform an input of tapping and moving the right boundary of the cropper (5210).

[0536] In one embodiment, the example <5205> , example <5207> and examples <5209> As illustrated, the electronic device (101) may expand the right area of ​​the cropper (5210) based on a user input to display the masking area (5220, 5230). According to one embodiment, when the user input reaches the boundary of the display (e.g., the boundary of the display area), the electronic device (101) may reduce the display ratio of the displayed image (5200) and display an expansion of the masking area (5220, 5230) proportionally thereto.

[0537] For example, as illustrated in FIG. 53, the electronic device (101) adjusts the display ratio of the image to expand the basic background area (e.g., the example <5301> Background extension area (5300) larger than the size of the reference) (e.g. Example <5303> According to one embodiment, the electronic device (101) can support background expansion up to M% of the size of the basic background expansion area through the cropper (5310). For example, the electronic device (101) can support background expansion up to about 15% of the size of the basic background expansion area when M% is set to about 15%.

[0538] FIGS. 54A and 54B are diagrams illustrating examples of operations supporting image editing in an electronic device according to one embodiment of the present disclosure.

[0539] According to one embodiment, FIGS. 54A and 54B may illustrate examples of operations that support various adjustments of an expansion area (or background expansion area) (e.g., a masking area) in an electronic device (101). According to one embodiment, a user may determine the size or direction of an image to be expanded through user input, and may adjust the background expansion area using an image or a cropper.

[0540] According to one embodiment, examples <5401A>, <5403A>, <5405A>, <5407A>, <5409A>, and <5411A> of FIG. 54A may represent examples of expansion areas according to user input. According to one embodiment, examples <5401B>, <5403B>, <5405B>, <5407B>, <5409B>, and <5411B> of FIG. 54B may represent various examples of result images (e....

Claims

1. In electronic devices, display; communication circuit; At least one processor comprising processing circuitry; and Contains memory that stores instructions, The above instructions, when individually and / or collectively executed by the at least one processor, cause the electronic device to: Display the image on the above display, Receive a first input for selecting a portion corresponding to an object in the above image, Based on the first input, crop a portion corresponding to the object in the image, Detecting a second input that moves the cropped portion from its original location within the image, Based on the movement of the cropped portion, an area corresponding to the original position of the cropped portion in the image is set as a masking area and displayed, Receiving a third input selecting one of the cropped portion or the masked area, If the third input corresponds to selecting the cropped portion, a first handler related to editing the cropped portion is provided through the cropped portion, If the third input corresponds to selecting the masking area, a second handler related to editing the cropped portion is provided through the masking area, Editing the image in response to a fourth input received based on the first handler or the second handler, Based on the fifth input, generate instructions to perform inpainting and / or outpainting on the edited image, Obtaining a result image in relation to the above instructions, and An electronic device that displays the above result image through the above display.

2. In the first paragraph, when the instructions are executed individually and / or collectively by the at least one processor, the electronic device, Establish a link between the cropped portion and the masking area, Based on the input for selecting the above masking area, the link established between the cropped portion and the masking area is recognized, An electronic device for determining a cropped portion corresponding to the masking area based on the above link.

3. In paragraph 1, An electronic device wherein the result image includes an image reconstructed by the inpainting and / or the outpainting performed based on the masked area and / or the cropped portion of the edited image.

4. In the second paragraph, when the instructions are individually and / or collectively executed by the at least one processor, the electronic device, An electronic device configured to highlight the masking area corresponding to the cropped portion when the third input is detected through the cropped portion.

5. In the second paragraph, when the instructions are individually and / or collectively executed by the at least one processor, the electronic device, An electronic device that selects and displays a cropped portion corresponding to the masked area when a cropped portion corresponding to the masked area exists on the image.

6. In the second paragraph, the instructions, when individually and / or collectively executed by the at least one processor, cause the electronic device to: If a cropped portion corresponding to the above masking area exists on the image, a first handler is provided based on the cropped portion, An electronic device that provides a second handler based on the masking area when a cropped portion corresponding to the masking area is removed from the image.

7. In the first paragraph, the instructions, when individually and / or collectively executed by the at least one processor, cause the electronic device to: An electronic device that, when an input specified through the second handler is detected, causes a cropped portion corresponding to the masking area to be restored and displayed in the masking area.

8. In the first paragraph, when the instructions are individually and / or collectively executed by the at least one processor, the electronic device, When cropping a portion corresponding to the above object, the area corresponding to the cropped portion in the image is recognized, Set the masking area based on the above recognized area, An electronic device for masking a grid pattern in the above masking area.

9. In the second paragraph, the instructions, when individually and / or collectively executed by the at least one processor, cause the electronic device to: An electronic device that allows the cropped portion and the masked area to inter-operate based on a link between the cropped portion and the masked area.

10. In the first paragraph, when the instructions are executed individually and / or collectively by the at least one processor, the electronic device, Based on the given input, determine the objects adjacent to the first cropped portion, Crop the portion corresponding to the adjacent object in the above image, An electronic device that sets cropped portions into a single group object.

11. In the 10th paragraph, the instructions, when individually and / or collectively executed by the at least one processor, cause the electronic device to: An electronic device for moving a cropped portion corresponding to an adjacent object into the handler based on an extension of the handler provided through the first cropped portion.

12. In the first paragraph, when the instructions are individually and / or collectively executed by the at least one processor, the electronic device, Based on the given input, determine the object adjacent to the masking area corresponding to the first cropped portion, Crop the portion corresponding to the adjacent object in the above image, An electronic device that sets cropped portions into a single group object.

13. In the first paragraph, the instructions, when individually and / or collectively executed by the at least one processor, cause the electronic device to: An electronic device that provides a function for comparing the image and the result image when providing the result image.

14. In the method of operating an electronic device, The act of displaying an image on a display; An action of receiving a first input for selecting a portion corresponding to an object in the above image; An operation of cropping a portion corresponding to the object in the image based on the first input; An action of detecting a second input that moves the cropped portion from its original location within the image; An action of setting and displaying an area corresponding to the original location of the cropped portion in the image as a masking area based on the movement of the cropped portion; An action of receiving a third input selecting one of the cropped portion or the masked area; An action of providing a first handler related to editing of the cropped portion through the cropped portion, if the third input corresponds to selecting the cropped portion; An action of providing a second handler related to editing of the cropped portion through the masking area, if the third input corresponds to selecting the masking area; An action of editing the image in response to a fourth input received based on the first handler or the second handler; An action of generating instructions to perform inpainting and / or outpainting on the edited image based on the fifth input; In relation to the above instruction, an operation of obtaining a result image; and A method comprising an action of displaying the result image through the display.

15. A non-transitory computer-readable recording medium storing instructions that, when executed by a processor of an electronic device, cause the electronic device to perform operations, said operations comprising: The action of displaying an image on a display; An action of receiving a first input for selecting a portion corresponding to an object in the above image; An operation of cropping a portion corresponding to the object in the image based on the first input; An action of detecting a second input that moves the cropped portion from its original location within the image; An action of setting and displaying an area corresponding to the original location of the cropped portion in the image as a masking area based on the movement of the cropped portion; An action of receiving a third input selecting one of the cropped portion or the masked area; If the third input corresponds to selecting the cropped portion, an action of providing a first handler related to editing of the cropped portion through the cropped portion; If the third input corresponds to selecting the masking area, an action of providing a second handler related to editing of the cropped portion through the masking area; An action of editing the image in response to a fourth input received based on the first handler or the second handler; An action for generating instructions to perform inpainting and / or outpainting based on the edited image, based on the fifth input; In relation to the above instructions, the operation of obtaining a result image, and A recording medium including an action of displaying the above result image through the display.

Images