Method for generating composite images and electronic device performing the same

Abstract

A method for generating an image, according to an embodiment, may include the operations of: determining an area of a foreground object from an original image; generating a foreground object image by separating the foreground object from the original image; determining whether the type of the foreground object image corresponds to a first type; and generating at least one composite image including at least a portion of the foreground object image, when the type of the foreground object image corresponds to the first type.

Description

Method for generating a synthetic image and an electronic device for performing the same

[0001] One embodiment relates to a technique for generating a synthetic image, and more particularly to a technique for generating a synthetic image based on a foreground object image.

[0002] Generative models can generate a variety of images. If the generated images appear low-quality to humans, they will not be selected by users. If low-quality images are provided to users through a generative model, their trust in the model may be lowered. Technologies may be needed to determine whether the provided images are low-quality and prevent them from being provided to users.

[0003] In one embodiment, an electronic device includes at least one processor including a processing circuit, and a memory including one or more storage media storing instructions, wherein the instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: determine an area of ​​a foreground object from an original image; generate a foreground object image by separating the foreground object from the original image; determine whether a type of the foreground object image corresponds to a first type; and, if the type of the foreground object image corresponds to the first type, generate one or more composite images including at least a portion of the foreground object image.

[0004] In one embodiment, a method of generating an image, performed by an electronic device, may include determining an area of ​​a foreground object from an original image, generating a foreground object image by separating the foreground object from the original image, determining whether a type of the foreground object image corresponds to a first type, and generating one or more composite images including at least a portion of the foreground object image if the type of the foreground object image corresponds to the first type.

[0005] FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment.

[0006] FIG. 2 is a flowchart of a method for generating a foreground object image and a background image based on an original image, according to one embodiment.

[0007] FIG. 3 is a flowchart of a method for generating one or more composite images, according to one embodiment.

[0008] FIG. 4 illustrates an output area of ​​a display and a foreground object image according to one embodiment.

[0009] FIG. 5 illustrates the distance between the bottom of the foreground object image and the bottom of the original image, according to one embodiment.

[0010] FIG. 6 is a flowchart of a method for determining the type of a foreground object based on whether a foreground object image has a defect, according to one embodiment.

[0011] FIG. 7 is a flowchart of a method for generating one or more images including at least a portion of a foreground object image based on a plurality of background types excluding a target background type, according to one embodiment.

[0012] FIG. 8 is a flowchart of a method for generating one or more composite images including at least a portion of a foreground object image whose position has been shifted, according to one embodiment.

[0013] FIG. 9 illustrates a foreground object image whose position has been shifted so that the bottom of the original image corresponds to the bottom of the foreground object image, according to one embodiment.

[0014] FIG. 10 illustrates a foreground object image whose position has been shifted so that a preset height of the foreground object image corresponds to the bottom of the original image, according to one embodiment.

[0015] FIG. 11 is a flowchart of a method for generating one or more composite images based on a second foreground object image generated by inpainting a defect in a foreground object image, according to one embodiment.

[0016] FIG. 12 illustrates a method for generating a second foreground object image by inpainting a defect in a foreground object image, according to one embodiment.

[0017] FIG. 13 is a flowchart of a method for generating a second foreground object image by inpainting a defect in a foreground object image based on one or more prompt elements, according to one embodiment.

[0018] FIG. 14 is a flowchart of a method for generating one or more composite images including at least a portion of a foreground object image and a subregion of a background image, according to one embodiment.

[0019] FIG. 15 illustrates a method for generating one or more composite images including at least a portion of a foreground object image and a subregion of a background image, according to one embodiment.

[0020] FIG. 16 illustrates one or more composite images according to one embodiment.

[0021] FIG. 17 illustrates a generative AI system according to one embodiment.

[0022] Hereinafter, various embodiments of the present disclosure will be described with reference to the attached drawings. However, this is not intended to limit the present disclosure to specific embodiments, and it should be understood that the present disclosure encompasses various modifications, equivalents, and / or alternatives of the embodiments.

[0023] FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment.

[0024] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to an embodiment. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with the electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (104) or the server (108) via a second network (199) (e.g., a long-range wireless communication network). According to an embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0025] The processor (120) may control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing, for example, software (e.g., a program (140)), and may perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculation, the processor (120) may store a command or data received from another component (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the command or data stored in the volatile memory (132), and store the resulting data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or a secondary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together therewith. For example, if the electronic device (101) includes a main processor (121) and a secondary processor (123), the secondary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a specified function. The secondary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0026] The auxiliary processor (123) may control at least a part of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0027] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0028] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0029] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0030] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0031] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0032] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0033] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0034] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0035] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0036] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0037] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0038] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0039] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0040] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0041] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) may support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0042] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. According to some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0043] In one embodiment, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0044] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0045] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service by itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0046] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments disclosed in this document are not limited to the aforementioned devices.

[0047] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0048] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0049] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more commands stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one command among the one or more commands stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one command called. The one or more commands may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0050] According to one embodiment, the method according to the various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0051] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0052] FIG. 2 is a flowchart of a method for generating a foreground object image and a background image based on an original image, according to one embodiment.

[0053] According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1) can output an original image (210) through a display (e.g., the display module (160) of FIG. 1). For example, the original image (210) may be an image stored in a gallery of the electronic device. For example, the original image (210) may be an image generated by a camera of the electronic device (e.g., the camera module (180) of FIG. 1).

[0054] According to one embodiment, the electronic device can determine an area of ​​a foreground object within an original image (210) based on a received user object selection input. For example, if the object selection input is a touch input to the display module (160), the area (212) of the foreground object can be determined based on coordinates of the touch input. For example, if the coordinates of the touch input are within an area (212) of a foreground object representing a deer, the area (212) of the foreground object can be determined.

[0055] According to one embodiment, the electronic device can generate a foreground object image (220) based on an area (212) of the foreground object.

[0056] According to one embodiment, the electronic device may generate a background image (230) by separating an area (212) of a foreground object from an original image (210). For example, the background image (230) may be an area from which an area (212) of a foreground object is deleted from the original image (210). The color value of a pixel in the area from which the area (212) of the foreground object is deleted may be 0. For example, the electronic device may generate the background image (230) by deleting an area (212) of a foreground object from the original image (210) and then inpainting the deleted area.

[0057] According to one embodiment, the electronic device may generate one or more composite images based on a foreground object image (220). For example, the composite image may be an image that includes at least a portion of the foreground object image (220) and has a background that is different from the background image (230). If the foreground object image (220) is of low quality, the foreground object image (220) may not blend well with some of the plurality of backgrounds. A composite image generated in which the foreground object image (220) does not blend well with the background may be referred to as a low-quality composite image. Below, a method for generating a composite image so as not to generate a low-quality composite image is described in detail with reference to FIGS. 3 to 16.

[0058] FIG. 3 is a flowchart of a method for generating one or more composite images, according to one embodiment.

[0059] The following operations 310 to 350 may be performed by an electronic device (e.g., the electronic device (101) of FIG. 1). For example, the electronic device may include a processor (e.g., the processor (120) of FIG. 1), a memory (e.g., the memory (130) of FIG. 1), and a display (e.g., the display module (160) of FIG. 1).

[0060] According to one embodiment, the electronic device can output an original image through a display. The original image may be an image selected by a user. The electronic device can receive an object selection input from the user. For example, the electronic device can receive the object selection input as a touch input to the display module (160).

[0061] In operation 310, the electronic device may determine an area of ​​a foreground object from an original image. For example, the electronic device may determine an area of ​​a foreground object corresponding to an object selection input.

[0062] In operation 320, the electronic device may generate a foreground object image by separating the foreground object from the original image. For example, the foreground object image may be generated as a layer within a base image having the size of the original image or the size of the display output area. The base image may be an image having a color value of 0. Pixels within the foreground object image may have coordinate values ​​on the base image.

[0063] In operation 330, the electronic device may determine whether the type of the foreground object image corresponds to a first type. For example, the first type may be a low-quality type.

[0064] According to one embodiment, if a ratio of a size of a foreground object image to a size of an output area of ​​a display of an electronic device is less than or equal to a threshold value, the type of the foreground object image may be determined to be a low-quality type.

[0065] In one embodiment, if the foreground object image is not connected to the bottom of the original image (or base image), the type of the foreground object image may be determined to be a low-quality type.

[0066] According to one embodiment, if the distance between the bottom of the foreground object image and the bottom of the original image (or, base image) is greater than a preset distance, the type of the foreground object image may be determined to be a low-quality type.

[0067] According to one embodiment, if the foreground object image is a defective image, the type of the foreground object image may be determined to be a low-quality type.

[0068] In addition to the above embodiments, the electronic device may determine the type of foreground object image in which the user is unlikely to clearly recognize the foreground object as a low-quality type. One or more conditions for determining whether the type of foreground object image is a low-quality type may be preset in the electronic device.

[0069] In operation 340, the electronic device may generate one or more composite images including at least a portion of the foreground object, if the type of the foreground object image corresponds to the first type. For example, the electronic device may generate a composite image including at least a portion of the foreground object based on the type of the foreground object image. For example, the method for generating the composite image may vary depending on the type of the foreground object image.

[0070] According to one embodiment, the electronic device can generate a composite image by synthesizing a foreground object image with each of a plurality of target background images corresponding to a plurality of background types. For example, the plurality of background types can include a blur background type, a grayscale background type, a color tint background type, a monochromatic background type, a gradient color background type, a marker background type, a pencil background type, and / or a line art background type. For example, the electronic device can generate a first target background image by applying a first background type to a background image of an original image. For example, the electronic device can generate a second target background image for a second background type.

[0071] According to one embodiment, an electronic device can generate a composite image by synthesizing a plurality of target background images corresponding to a plurality of background types with a foreground object image using a generative model. The generative model or generative AI system is described in detail below with reference to FIG. 17.

[0072] Below, a method for generating a synthetic image based on the type of foreground object image is described in detail with reference to FIGS. 7 to 16.

[0073] In operation 350, the electronic device may display one or more composite images on the display of the electronic device. The user may view the one or more composite images displayed on the display and select one of the one or more composite images. The electronic device may set the selected composite image as the background or wallpaper of the electronic device.

[0074] FIG. 4 illustrates an output area of ​​a display and a foreground object image according to one embodiment.

[0075] According to one embodiment, an electronic device (e.g., electronic device (101) of FIG. 1) can determine whether a ratio of a size of a foreground object image (420) to a size of an output area (410) (or, a base image) of a display (e.g., display module (160) of FIG. 1) is less than or equal to a threshold value.

[0076] According to one embodiment, the electronic device may determine the boundary (430) of the foreground object image (420) based on the coordinate values ​​of pixels within the object image (420). For example, the electronic device may determine the boundary (430) of the foreground object image (420) as four vertices determined based on the leftmost coordinate on the horizontal axis, the rightmost coordinate on the horizontal axis, the topmost coordinate on the vertical axis, and the bottommost coordinate on the vertical axis among the coordinates of the pixels within the object image (420). The electronic device may determine the area within the boundary (430) of the foreground object image (420) as the size of the foreground object image (420).

[0077] The electronic device may determine whether a ratio of the size (e.g., area) of a boundary (430) of a foreground object image (420) to the size (e.g., area) of an output area (410) of a display (or, base image) is less than or equal to a preset threshold value (e.g., 30%). If the ratio is less than or equal to the preset threshold value, the foreground object image (420) may be determined to be of low quality.

[0078] FIG. 5 illustrates the distance between the bottom of the foreground object image and the bottom of the original image, according to one embodiment.

[0079] According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1) may calculate a distance (530) between a lower boundary (522) of a foreground object image (520) and a lower edge (512) of an original image (510) (or, a base image, an output area of ​​a display). For example, if the distance (530) is not 0 (i.e., if the foreground object image (520) is not connected to the lower edge (512) of the original image (510), the foreground object image (520) may be determined to be of low quality. For example, if the distance (530) is greater than or equal to a preset distance, the foreground object image (520) may be determined to be of a low quality type.

[0080] FIG. 6 is a flowchart of a method for determining the type of a foreground object based on whether a foreground object image has a defect, according to one embodiment.

[0081] According to one embodiment, operations 610 and 620 below may be related to operation 330 described above with reference to FIG. 3. For example, operation 330 may include operations 610 and 620. Operations 610 and 620 may be performed by an electronic device (e.g., electronic device (101) of FIG. 1). For example, the electronic device may include a processor (e.g., processor (120) of FIG. 1), a memory (e.g., memory (130) of FIG. 1), and a display (e.g., display module (160) of FIG. 1).

[0082] In operation 610, the electronic device may determine whether the foreground object image is a defective image. For example, the electronic device may determine whether the shape of the foreground object is complete using an artificial intelligence model (e.g., a generative model). For example, referring to FIG. 4 , it may be determined whether the shape of the deer, which is the foreground object in the foreground object image (420) described above, is complete. The shape of the deer appearing in the foreground object image (420) may be determined to be incomplete.

[0083] According to one embodiment, an electronic device can determine the type of a foreground object. For example, the types of foreground objects may include people, animals, and objects. The electronic device can determine whether a defect in the foreground object is restorable based on the type of the foreground object. If the defect in the foreground object is restorable, the electronic device can restore the defect in the foreground object. A foreground object with a restored defect may not be determined to be a low-quality image.

[0084] In operation 620, the electronic device may determine that the type of the foreground object image corresponds to the first type when the foreground object image is determined to be a defective image.

[0085] FIG. 7 is a flowchart of a method for generating one or more images including at least a portion of a foreground object image based on a plurality of background types excluding a target background type, according to one embodiment.

[0086] According to one embodiment, operations 710 and 720 below may be related to operation 340 described above with reference to FIG. 3. For example, operation 340 may include operations 710 and 720. Operations 710 and 720 may be performed by an electronic device (e.g., electronic device (101) of FIG. 1). For example, the electronic device may include a processor (e.g., processor (120) of FIG. 1), a memory (e.g., memory (130) of FIG. 1), and a display (e.g., display module (160) of FIG. 1).

[0087] In operation 710, the electronic device may determine at least one target background type corresponding to a first type among a plurality of background types. For example, the electronic device may determine at least one target background type corresponding to a type of a foreground object image among the plurality of background types.

[0088] According to one embodiment, the plurality of background types may include a blur background type, a grayscale background type, a color tint background type, a monochromatic background type, a gradient color background type, a marker background type, a pencil background type, and / or a line art background type. For example, if the type of the foreground object image corresponds to the first type, the blur background type, the monochromatic background, and the grayscale background types may be determined as the target background types. For example, a target background image corresponding to the target background type may be generated.

[0089] According to one embodiment, the electronic device determines the degree of incongruity between the foreground object and the first background according to the first background type based on at least one color element, lighting element, or shadow element of the foreground object image, and if the determined degree of incongruity is greater than or equal to a threshold value, the first background type may be determined as the target background type. For example, the degree of incongruity may be determined based on similarity between elements of the foreground object image and elements of the first background. For example, a difference between an average of color values ​​of the foreground object image and an average of color values ​​of the first background may be determined as the degree of incongruity.

[0090] In operation 720, the electronic device may generate one or more composite images including at least a portion of a foreground object image based on a plurality of background types excluding at least one target background type. By excluding the target background type from the background types for generating the composite images, the generated composite images may not include low-quality composite images.

[0091] In one embodiment, the number of one or more composite images may vary depending on the number of excluded target background types.

[0092] According to one embodiment, when the type of the first foreground object image is determined to be the first type, the number of one or more composite images generated based on the first foreground object image may be less than the number of one or more composite images generated based on the second foreground object image when the type of the second foreground object image is determined to be the second type (e.g., a normal image type or a high-quality image type).

[0093] According to one embodiment, the electronic device can generate one or more synthetic images including at least a portion of a foreground object image based on a plurality of background types excluding at least one target background type using a generative model.

[0094] FIG. 8 is a flowchart of a method for generating one or more composite images including at least a portion of a foreground object image whose position has been shifted, according to one embodiment.

[0095] According to one embodiment, operations 810 and 820 below may be related to operation 340 described above with reference to FIG. 3. For example, operation 340 may include operations 810 and 820. Operations 810 and 820 may be performed by an electronic device (e.g., electronic device (101) of FIG. 1). For example, the electronic device may include a processor (e.g., processor (120) of FIG. 1), a memory (e.g., memory (130) of FIG. 1), and a display (e.g., display module (160) of FIG. 1).

[0096] In operation 810, the electronic device can move the foreground object image so that the position of the bottom of the foreground object image corresponds to the position of the bottom of the original image (or, the base image, the output area of ​​the display (e.g., the display module (160) of FIG. 1)). A method of moving the foreground object image is described in detail below with reference to FIGS. 9 and 10.

[0097] In operation 820, the electronic device may generate one or more composite images including at least a portion of the foreground object image. For example, the electronic device may generate the composite image by synthesizing at least a portion of the foreground object image whose position has been shifted and a background image.

[0098] According to one embodiment, the electronic device can generate a composite image by synthesizing at least a portion of a shifted foreground object image and a background image using a generative model.

[0099] FIG. 9 illustrates a foreground object image whose position has been shifted so that the bottom of the original image corresponds to the bottom of the foreground object image, according to one embodiment.

[0100] According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1) may move a foreground object image (520) such that a bottom (512) of the original image (510) (or, a base image, an output area of ​​a display (e.g., a display module (160) of FIG. 1)) described above with reference to FIG. 5 corresponds to a bottom boundary (522) of the foreground object image (520) described above with reference to FIG. 5.

[0101] According to one embodiment, the electronic device may determine which portion of the foreground object image (520) is low quality and move the foreground object image (520) based on the portion determined to be low quality. For example, if the electronic device determines that the upper portion or the lower portion of the foreground object image (520) is low quality, the electronic device may move the foreground object image (520) so that the upper portion or the lower portion of the foreground object image (520) corresponds to the upper or lower portion of the display. For example, if the electronic device determines that the left portion or the right portion of the foreground object image (520) is low quality, the electronic device may move the foreground object image (520) so that the left portion or the right portion of the foreground object image (520) corresponds to the left end or the right end of the display.

[0102] FIG. 10 illustrates a foreground object image whose position has been shifted so that a preset height of the foreground object image corresponds to the bottom of the original image, according to one embodiment.

[0103] According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1) may move a foreground object image (520) such that a preset height of the foreground object image (520) described above with reference to FIG. 5 corresponds to a bottom (512) of the original image (510) (or, a basic image, an output area of ​​a display (e.g., a display module (160) of FIG. 1)) described above with reference to FIG. 5. For example, the preset height of the foreground object image (520) may be a height of a boundary line having a height (1010) from a bottom boundary line (522) of the foreground object image (520). For example, the height (1010) may correspond to a distance (530) described above with reference to FIG. 5. As the position of the foreground object image (520) is moved, the area (1020) may be deleted.

[0104] According to one embodiment, the electronic device may generate a synthetic image using the remaining area excluding the area (1020) among the entire area of ​​the foreground object image (520). For example, the electronic device may generate a synthetic image using the remaining area excluding the area (1020) among the entire area of ​​the foreground object image (520) using a generative model.

[0105] FIG. 11 is a flowchart of a method for generating one or more composite images based on a second foreground object image generated by inpainting a defect in a foreground object image, according to one embodiment.

[0106] According to one embodiment, operations 1110 and 1120 below may be related to operation 340 described above with reference to FIG. 3. For example, operation 340 may include operations 1110 and 1120. Operations 1110 and 1120 may be performed by an electronic device (e.g., electronic device (101) of FIG. 1). For example, the electronic device may include a processor (e.g., processor (120) of FIG. 1), a memory (e.g., memory (130) of FIG. 1), and a display (e.g., display module (160) of FIG. 1).

[0107] In operation 1110, the electronic device can generate a second foreground object image by inpainting a defect in the foreground object image if the foreground object image is determined to be a defective image.

[0108] In one embodiment, the electronic device may use an artificial intelligence model (e.g., a generative model) to determine whether a foreground object image is defective.

[0109] In one embodiment, the electronic device can generate a second foreground object image by inpainting defects in the foreground object image using an artificial intelligence model.

[0110] In operation 1120, the electronic device may generate one or more composite images that include at least a portion of the second foreground object image.

[0111] According to one embodiment, the electronic device may determine the type of the second foreground object image as a low-quality type when a ratio of a size of the second foreground object image to a size of an output area of ​​the display is less than or equal to a threshold value.

[0112] In one embodiment, if the second foreground object image is not connected to the bottom of the original image (or base image), the type of the second foreground object image may be determined to be a low-quality type.

[0113] According to one embodiment, if the distance between the bottom of the second foreground object image and the bottom of the original image (or, base image) is greater than a preset distance, the type of the second foreground object image may be determined to be a low-quality type.

[0114] According to one embodiment, if the type of the second foreground object image is not a low-quality type, one or more synthetic images including the second foreground object image can be generated.

[0115] According to one embodiment, when the type of the second foreground object image is a low-quality type, operations 720 and 730 described above with reference to FIG. 7 may be performed on the second foreground object image. By performing operations 720 and 730 on the second foreground object image, one or more composite images including at least a portion of the second foreground object image may be generated.

[0116] According to one embodiment, when the type of the second foreground object image is a low-quality type, operations 810 and 820 described above with reference to FIG. 8 may be performed on the second foreground object image. By performing operations 810 and 820 on the second foreground object image, one or more composite images including at least a portion of the second foreground object image may be generated.

[0117] For example, the electronic device can use the generative model to generate one or more synthetic images that include at least a portion of a second foreground object image.

[0118] FIG. 12 illustrates a method for generating a second foreground object image by inpainting a defect in a foreground object image, according to one embodiment.

[0119] According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1) can output an original image (1210) through a display (e.g., the display module (160) of FIG. 1). For example, the original image (1210) may be an image stored in a gallery of the electronic device. For example, the original image (1210) may be an image generated by a camera of the electronic device (e.g., the camera module (180) of FIG. 1).

[0120] According to one embodiment, the electronic device can determine an area of ​​a foreground object within an original image (1210) based on a received user object selection input. For example, if the object selection input is a touch input to the display module (160), the area (1212) of the foreground object can be determined based on coordinates of the touch input. For example, if the coordinates of the touch input are within an area (1212) of a foreground object representing a person, the area (1212) of the foreground object can be determined.

[0121] According to one embodiment, the electronic device may generate a foreground object image (1220) based on a region (1212) of a foreground object. The foreground object image (1220) may not include a foreground object region (1214) representing a flower. If the foreground object region (1214) representing a flower is additionally selected by the user, the foreground object image (1220) may display an object representing a person and an object representing a flower together.

[0122] According to one embodiment, the electronic device may determine the completeness of a foreground object included in a foreground object image (1220) using an artificial intelligence model (e.g., a generative model).

[0123] In one embodiment, the electronic device may generate a second foreground object (1230) by inpainting defects of an imperfect foreground object using an artificial intelligence model. An image including the second foreground object may be referred to as a second foreground object image (1230).

[0124] FIG. 13 is a flowchart of a method for generating a second foreground object image by inpainting a defect in a foreground object image based on one or more prompt elements, according to one embodiment.

[0125] According to one embodiment, operations 1310 and 1320 below may be related to operation 1110 described above with reference to FIG. 11. For example, operation 1110 may include operations 1310 and 1320. Operations 1310 and 1320 may be performed by an electronic device (e.g., the electronic device (101) of FIG. 1). For example, the electronic device may include a processor (e.g., the processor (120) of FIG. 1), a memory (e.g., the memory (130) of FIG. 1), and a display (e.g., the display module (160) of FIG. 1).

[0126] In operation 1310, the electronic device may receive one or more prompt elements for a foreground object image. For example, the electronic device may receive one or more prompt elements from a user. The prompt elements may be keywords or words. The user may input the prompt elements into the electronic device through a user interface.

[0127] In one embodiment, an artificial intelligence model (e.g., a generative model) of an electronic device may identify the type of a foreground object image and provide the user with one or more candidate prompt elements based on the identified type. For example, if the foreground object image is identified as a person, the user may be provided with names of various objects as one or more candidate prompt elements associated with the person. The user may input the prompt element into the electronic device by selecting a desired prompt element from among the one or more provided candidate prompt elements.

[0128] In operation 1320, the electronic device may generate a second foreground object image by inpainting a defect in the foreground object image based on one or more prompt elements. For example, the electronic device may composite an object corresponding to the first prompt element into the foreground object image. For example, if the first prompt element represents a mobile phone, the electronic device may composite a mobile phone object into the foreground object image.

[0129] FIG. 14 is a flowchart of a method for generating one or more composite images including at least a portion of a foreground object image and a subregion of a background image, according to one embodiment.

[0130] According to one embodiment, operations 1410 and 1420 below may be related to operation 340 described above with reference to FIG. 3. For example, operation 340 may include operations 1410 and 1420. Operations 1410 and 1420 may be performed by an electronic device (e.g., electronic device (101) of FIG. 1). For example, the electronic device may include a processor (e.g., processor (120) of FIG. 1), a memory (e.g., memory (130) of FIG. 1), and a display (e.g., display module (160) of FIG. 1).

[0131] In operation 1410, the electronic device may determine a background image subregion to be used to generate one or more composite images among the background image regions of the original image, if the type of the foreground object image corresponds to the first type. For example, the background image subregion may be a region among the background image regions that is connected to a defective portion of the foreground object image. For example, the background image subregion may be a foreground object that is connected to a defective portion of the foreground object image. Thereafter, the foreground object image and the background image subregion may be processed as an integrated object, and the foreground object image and the background image subregion may be named a third foreground object image. The background image subregion is described in detail below with reference to FIG. 15.

[0132] In operation 1420, the electronic device may generate one or more composite images including at least a portion of a foreground object image and a subregion of a background image.

[0133] According to one embodiment, if the type of the third foreground object image is not a low-quality type, one or more synthetic images including the third foreground object image can be generated.

[0134] According to one embodiment, when the type of the third foreground object image is a low-quality type, operations 720 and 730 described above with reference to FIG. 7 may be performed on the third foreground object image. By performing operations 720 and 730 on the third foreground object image, one or more composite images including at least a portion of the third foreground object image may be generated.

[0135] According to one embodiment, when the type of the third foreground object image is a low-quality type, operations 810 and 820 described above with reference to FIG. 8 may be performed on the third foreground object image. By performing operations 810 and 820 on the third foreground object image, one or more composite images including at least a portion of the third foreground object image may be generated.

[0136] FIG. 15 illustrates a method for generating one or more composite images including at least a portion of a foreground object image and a subregion of a background image, according to one embodiment.

[0137] According to one embodiment, an electronic device (e.g., electronic device (101) of FIG. 1) may determine a background image sub-region (1530) for a foreground object image (1520) when the foreground object image (1520) is a defective image. For example, the background image sub-region (1530) may be an object among a plurality of objects in an original image that is connected to a defective portion of the foreground object image.

[0138] According to one embodiment, the electronic device can generate a composite image (1550) by synthesizing a background image (1540) for a foreground object image (1520) and a background image subregion (1530).

[0139] FIG. 16 illustrates one or more composite images according to one embodiment.

[0140] According to one embodiment, an electronic device (e.g., electronic device (101) of FIG. 1) may generate one or more composite images (1620) based on an original image (1610). The one or more composite images (1620) may be images generated by applying a plurality of different background types to the original image (1610). For example, when a type of a foreground object image corresponds to a first type, the plurality of background types applied to the composite images (1620) may be types excluding at least one target background type corresponding to the first type.

[0141] FIG. 17 illustrates a generative AI system according to one embodiment.

[0142] The User Query / Response Interface can receive user input. User input can be in the form of natural language, images, and / or videos. Context information can also be transmitted when the user input is transmitted. Context information can include various additional information at the time of user input. For example, it can include information about the application the user is currently using or the user's location. Furthermore, user input can be a mixture of natural language, images, sounds, and context information. Furthermore, user input can be in non-natural language forms, such as selecting a menu. The User Query / Response Interface can output the results of a generative artificial intelligence system to the user. The output can be in the form of natural language or specific content, and can also be provided in the form of actions requested by the user. The User Query Interface can output the results of a generative artificial intelligence system to the user. The output can be in the form of natural language or specific content, and can also be provided in the form of actions requested by the user.

[0143] The AI ​​framework can receive user input and coordinate and control each component necessary to carry out the user's intention based on the user's query.

[0144] User input received from the User Query / Response Interface can be sent to the Prompt design component. The Prompt design component can be used to generate prompts suitable for inputting user input into an LLM or LMM. The Prompt design component can be an AI component that uses a machine learning algorithm or neural network to develop better prompts over time. The Prompt design component can access a knowledge component containing user preference data, a prompt library, and prompt examples based on the user input to generate prompts and pass the generated prompts to the LLM or LMM.

[0145] The API / Plug-in management component can communicate with external information when additional information is requested when user input is passed as input to a generative model. The API / Plug-in management component establishes a channel for communication with external entities within the AI ​​Interface via APIs, enabling access to various data sources. Furthermore, if an application or service needs to perform an action that ultimately implements user input, rather than an intermediate result, the API / Plug-in management component can request such action via APIs. Information obtained from external sources can be used to generate prompts in the Prompt design component alongside user input, or can be passed as input to the generative model.

[0146] The Refiner component can fine-tune the output from generative models. For example, the Refiner component can verify that the content generated by LLMs and / or LMMs is not irrelevant, biased, or harmful. Furthermore, the Refiner component can determine the degree to which the output matches the user's desired outcome and, if necessary, initiate additional processing. Additionally, the Refiner component can configure and provide users with hints to avoid undesirable output.

[0147] Generative AI models generally refer to artificial intelligence neural networks that generate new forms of data based on user input. Generative AI models can include models that generate images and / or models that generate language. Representative image-generating models include generative adversarial networks (GANs) and variational autoencoders (VAEs), with examples including diffusion-based generative models that use VAE and Transformer architectures. Language-generating models are trained to statistically optimize output based on input values, and examples include models like CHAT-GPT 3 and CHAT-GPT 4. Additionally, there are large multimodal models (LMMs) that can recognize various types of data input, such as text, images, and voice, and generate corresponding new data.

[0148] According to one embodiment, an electronic device (101) includes at least one processor (120) including a processing circuit, and a memory (130) including one or more storage media storing instructions, wherein the instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device to: determine an area of ​​a foreground object from an original image; generate a foreground object image by separating the foreground object from the original image; determine whether a type of the foreground object image corresponds to a first type; and, if the type of the foreground object image corresponds to the first type, generate one or more composite images including at least a portion of the foreground object image.

[0149] According to one embodiment, when the instructions are individually or collectively executed by at least one processor (120), the electronic device may: display one or more composite images via a display of the electronic device.

[0150] According to one embodiment, the first type may be a type in which the ratio of the size of the foreground object image to the size of the output area of ​​the display of the electronic device is less than or equal to a threshold value.

[0151] According to one embodiment, the first type may be a type in which the foreground object image is not connected to the bottom of the original image.

[0152] According to one embodiment, the first type may be a type in which the distance between the bottom of the foreground object image and the bottom of the original image is greater than or equal to a preset distance.

[0153] According to one embodiment, when the instructions are individually or collectively executed by at least one processor (120), the electronic device may be configured to: determine whether a foreground object image is a defective image, and if the foreground object image is determined to be a defective image, determine that a type of the foreground object image corresponds to the first type.

[0154] According to one embodiment, when the instructions are individually or collectively executed by at least one processor (120), the electronic device may be configured to: determine at least one target background type corresponding to a first type among a plurality of background types; and generate one or more composite images including at least a portion of a foreground object image based on the plurality of background types excluding the at least one target background type.

[0155] According to one embodiment, when the instructions are individually or collectively executed by at least one processor (120), the electronic device may: move the foreground object image so that the location of the bottom of the foreground object image corresponds to the location of the bottom of the original image; and generate one or more composite images including at least a portion of the foreground object image.

[0156] According to one embodiment, when the instructions are individually or collectively executed by at least one processor (120), the electronic device may be configured to: generate a second foreground object image by inpainting a defect in the foreground object image, if the foreground object image is determined to be a defective image; and generate one or more composite images including at least a portion of the second foreground object image.

[0157] According to one embodiment, when the instructions are individually or collectively executed by at least one processor (120), the electronic device may be configured to: receive one or more prompt elements for a foreground object image; and generate a second foreground object image by inpainting a defect in the foreground object image based on the one or more prompt elements.

[0158] According to one embodiment, when the instructions are individually or collectively executed by at least one processor (120), the electronic device may: determine a background image subregion to be used to generate one or more composite images from among the background image regions of the original image, when the type of the foreground object image corresponds to the first type, and generate one or more composite images including at least a portion of the foreground object image and the background image subregion.

[0159] According to one embodiment, a method for generating an image, performed by an electronic device (101), may include an operation (310) of determining an area of ​​a foreground object from an original image, an operation (320) of generating a foreground object image by separating the foreground object from the original image, an operation (330) of determining whether a type of the foreground object image corresponds to a first type, and an operation (340) of generating one or more composite images including at least a portion of the foreground object image if the type of the foreground object image corresponds to the first type.

[0160] According to one embodiment, the method for generating the image may further include an operation (350) of displaying one or more composite images through a display of the electronic device.

[0161] According to one embodiment, the operation (330) of determining whether the type of the foreground object image corresponds to the first type may include the operation (610) of determining whether the foreground object image is a defective image, and, if the foreground object image is determined to be a defective image, the operation (620) of determining that the type of the foreground object image corresponds to the first type.

[0162] According to one embodiment, the operation (340) of generating one or more composite images including at least a portion of a foreground object image may include the operation (710) of determining at least one target background type corresponding to a first type among a plurality of background types, and the operation (720) of generating one or more composite images including at least a portion of the foreground object image based on the plurality of background types excluding the at least one target background type.

[0163] According to one embodiment, the operation (340) of generating one or more composite images including at least a portion of a foreground object image may include the operation (810) of moving the foreground object image so that a position of the bottom of the foreground object image corresponds to a position of the bottom of the original image, and the operation (820) of generating one or more composite images including at least a portion of the foreground object image.

[0164] According to one embodiment, the operation (340) of generating one or more composite images including at least a portion of a foreground object image may include the operation (1110) of generating a second foreground object image by inpainting a defect in the foreground object image when the foreground object image is determined to be a defective image, and the operation (1120) of generating one or more composite images including at least a portion of the second foreground object image.

[0165] According to one embodiment, the operation (1110) of generating a second foreground object image may include the operation (1310) of receiving one or more prompt elements for the foreground object image, and the operation (1320) of generating the second foreground object image by inpainting a defect in the foreground object image based on the one or more prompt elements.

[0166] According to one embodiment, the operation (340) of generating one or more composite images including at least a portion of a foreground object image may include the operation (1410) of determining a background image subregion to be used to generate one or more composite images among a background image region of an original image, when the type of the foreground object image corresponds to a first type, and the operation (1420) of generating one or more composite images including at least a portion of the foreground object image and the background image subregion.

[0167] The embodiments described above may be implemented using hardware components, software components, and / or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone; however, one of ordinary skill in the art will recognize that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing unit may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

[0168] Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may, independently or collectively, command the processing device. The software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave, for interpretation by the processing device or for providing instructions or data to the processing device. The software may also be distributed over networked computer systems and stored or executed in a distributed manner. The software and data may be stored on a computer-readable recording medium.

[0169] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination, and the program commands recorded on the medium may be those specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands such as ROMs, RAMs, and flash memories. Examples of program commands include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc.

[0170] The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiment, and vice versa.

[0171] Although the embodiments described above have been described with limited drawings, those skilled in the art will appreciate that various technical modifications and variations can be applied based on the described embodiments. For example, appropriate results can still be achieved even if the described techniques are performed in a different order than described, and / or components of the described systems, structures, devices, circuits, etc. are combined or combined in a different manner than described, or are replaced or substituted with other components or equivalents.

[0172] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims described below.

Claims

1. In an electronic device (101), At least one processor (120) comprising a processing circuit; and A memory (130) comprising one or more storage media storing instructions, When the above instructions are individually or collectively executed by the at least one processor (120), the electronic device (101) causes: Determine the area of ​​the foreground object from the original image, Generating a foreground object image by separating the foreground object from the original image, Determine whether the type of the foreground object image corresponds to the first type, If the type of the foreground object image corresponds to the first type, generate one or more composite images including at least a portion of the foreground object image. To do, Electronic devices.

2. In paragraph 1, When the above instructions are individually or collectively executed by the at least one processor (120), the electronic device (101) causes: Displaying one or more of the composite images through the display of the electronic device. To do, Electronic devices.

3. In paragraph 1 or 2, The first type above is, A type in which the ratio of the size of the foreground object image to the size of the output area of ​​the display of the electronic device is below a threshold value. Electronic devices.

4. In any one of paragraphs 1 to 3, The first type above is, The above foreground object image is of a type that is not connected to the bottom of the original image. Electronic devices.

5. In any one of paragraphs 1 to 4, The first type above is, A type in which the distance between the bottom of the foreground object image and the bottom of the original image is greater than a preset distance. Electronic devices.

6. In any one of paragraphs 1 to 5, When the above instructions are individually or collectively executed by the at least one processor (120), the electronic device (101) causes: Determine whether the foreground object image above is a defective image, If the foreground object image is determined to be a defective image, the type of the foreground object image is determined to correspond to the first type. To do, Electronic devices.

7. In any one of paragraphs 1 to 6, When the above instructions are individually or collectively executed by the at least one processor (120), the electronic device (101) causes: determining at least one target background type corresponding to the first type among a plurality of background types; Generating one or more composite images including at least a portion of the foreground object image based on the plurality of background types excluding the at least one target background type. To do, Electronic devices.

8. In any one of paragraphs 1 to 7, When the above instructions are individually or collectively executed by the at least one processor (120), the electronic device (101) causes: Move the foreground object image so that the position of the bottom of the foreground object image corresponds to the position of the bottom of the original image, Generating one or more composite images including at least a portion of the foreground object image; To do, Electronic devices.

9. In any one of paragraphs 1 to 8, When the above instructions are individually or collectively executed by the at least one processor (120), the electronic device (101) causes: If the foreground object image is determined to be a defective image, a second foreground object image is generated by inpainting the defects in the foreground object image, Generating one or more composite images including at least a portion of the second foreground object image; To do, Electronic devices.

10. In any one of paragraphs 1 to 9, When the above instructions are individually or collectively executed by the at least one processor (120), the electronic device (101) causes: Receiving one or more prompt elements for the foreground object image, Generating a second foreground object image by inpainting defects in the foreground object image based on one or more of the prompt elements. To do, Electronic devices.

11. In any one of paragraphs 1 to 9, When the above instructions are individually or collectively executed by the at least one processor (120), the electronic device (101) causes: If the type of the foreground object image corresponds to the first type, determine a background image subarea to be used for generating one or more composite images among the background image areas of the original image, Generating one or more composite images including at least a portion of the foreground object image and a subregion of the background image To do, Electronic devices.

12. A method of generating an image, performed by an electronic device (101), An operation for determining the area of ​​a foreground object from an original image (310); An operation (320) of generating a foreground object image by separating the foreground object from the original image; An operation (330) for determining whether the type of the foreground object image corresponds to the first type; and An operation (340) of generating one or more composite images including at least a portion of the foreground object image, if the type of the foreground object image corresponds to the first type. Including, method.

13. In paragraph 12, An operation (350) of displaying one or more of the above composite images through a display of the electronic device. Including more, method.

14. In paragraph 12 or 13, The operation (330) of determining whether the type of the foreground object image corresponds to the first type is: An operation (610) for determining whether the foreground object image is a defective image; and If the foreground object image is determined to be a defective image, an operation (620) for determining that the type of the foreground object image corresponds to the first type. Including, method.

15. A computer program stored on a computer-readable recording medium for executing the method of any one of claims 12 to 14 in combination with hardware.

Images