Electronic device, method, and non-transitory computer-readable storage medium for acquiring image

Abstract

This electronic device may comprise: at least one camera; a display; a memory storing instructions; and at least one processor including processing circuitry. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to: while a preview image obtained through the at least one camera is displayed through the display, receive a user input for setting a value of a virtual aperture; obtain a preview bokeh image on the basis of providing the value of the virtual aperture and the preview image to a bokeh model; and display the preview bokeh image through the display.

Description

Electronic device, method, and non-transient computer-readable storage medium for acquiring an image

[0001] The present disclosure relates to an electronic device, a method, and a non-transient computer-readable storage medium for acquiring an image.

[0002] An electronic device may include a camera. The electronic device can acquire images through the camera. Various technologies are being developed to improve image quality. For example, the electronic device can change the camera's settings.

[0003] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art related to the present disclosure.

[0004] An electronic device is provided. The electronic device may include at least one camera. The electronic device may include a display. The electronic device may include a memory comprising one or more storage media for storing instructions. The electronic device may include at least one processor comprising processing circuitry. The instructions may cause the electronic device to receive user input for setting a virtual aperture value while displaying a preview image acquired through the at least one camera via the display, when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to acquire a preview bokeh image expressing a visual effect according to the value of the virtual aperture, based on providing the value of the virtual aperture and the preview image to a bokeh model, when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to display the preview bokeh image via the display, when executed individually or collectively by the at least one processor.

[0005] A method is provided. The method may be performed in an electronic device having at least one camera and a display. The method may include receiving user input for setting a value of a virtual aperture while displaying a preview image acquired through the at least one camera through the display. The method may include acquiring a preview bokeh image expressing a visual effect according to the value of the virtual aperture based on providing the value of the virtual aperture and the preview image to a bokeh model. The method may include displaying the preview bokeh image through the display.

[0006] A non-transient computer-readable storage medium is provided. The non-transient computer-readable storage medium may store one or more programs. The one or more programs may include instructions that cause the electronic device to receive user input to set a value of a virtual aperture while displaying a preview image acquired through the at least one camera on the display when executed by the electronic device having at least one camera and a display. The one or more programs may include instructions that cause the electronic device to acquire a preview bokeh image expressing a visual effect according to the value of the virtual aperture, based on providing the value of the virtual aperture and the preview image to a bokeh model when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to display the preview bokeh image on the display when executed by the electronic device.

[0007] FIG. 1 is a block diagram of an electronic device in a network environment according to one embodiment.

[0008] Figure 2 illustrates an example of a simplified block diagram of an electronic device.

[0009] Figure 3 illustrates examples of components used to obtain an image with a bokeh effect applied.

[0010] Figure 4 illustrates an example of a user interface displayed while running a camera application.

[0011] Figures 5a and 5b illustrate examples of images obtained based on a bokeh model.

[0012] Figure 6 illustrates an example of metadata for a bokeh image.

[0013] FIGS. 7a and 7b illustrate examples of images including visual objects corresponding to a light source.

[0014] FIG. 8a illustrates examples of operations of an electronic device for acquiring a preview bokeh image.

[0015] FIG. 8b illustrates examples of operations of an electronic device for acquiring bokeh images.

[0016] Throughout the drawings, the same reference numerals will be understood to refer to the same parts, components, and structures.

[0017] The terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit the scope of other embodiments. A singular expression may include a plural expression unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as generally understood by those skilled in the art described in this disclosure. Terms used in this disclosure that are defined in a general dictionary may be interpreted as having the same or similar meaning as they have in the context of the relevant technology, and are not to be interpreted in an ideal or overly formal sense unless explicitly defined in this disclosure. In some cases, even terms defined in this disclosure are not to be interpreted to exclude the embodiments of this disclosure.

[0018] In the various embodiments of the present disclosure described below, a hardware-based approach is described as an example. However, since the various embodiments of the present disclosure include techniques using both hardware and software, the various embodiments of the present disclosure do not exclude a software-based approach.

[0019] Terms used in the following description to refer to data (e.g., metadata), parameters (e.g., shutter speed, ISO (International Standards Organization, virtual aperture) values ​​(e.g., aperture value, virtual aperture value, shutter speed value, ISO value), modules (e.g., user interface module, brightness control module, depth module, brightness adjustment module), artificial intelligence models (e.g., first bokeh model, second bokeh model, single image depth model, multi-image depth model, image composite model), computational states (e.g., operation, process), objects (e.g., visual object, executable object, virtual object, icon, application), network entities, and device components are examples provided for the convenience of explanation. Accordingly, this disclosure is not limited to the terms described below, and other terms having equivalent technical meanings may be used. Furthermore, terms such as '...part', '...device', '...object', '...body' used below may refer to at least one shape structure or a unit that processes a function. It is possible.

[0020] Additionally, in this disclosure, expressions of "greater than" or "less than" may be used to determine whether a specific condition is satisfied or fulfilled; however, this is merely for the purpose of expressing an example and does not exclude descriptions of "greater than" or "less than." Conditions described as "greater than" may be replaced with "greater than," conditions described as "less than" may be replaced with "less than," and conditions described as "greater than and less than" may be replaced with "greater than and less than." Furthermore, "A" to "B" below refer to at least one of elements from A (including A) to B (including B). Below, "C" and / or "D" refers to including at least one of "C" or "D," i.e., {"C", "D", "C" and "D"}.

[0021] FIG. 1 is a block diagram of an electronic device in a network environment according to one embodiment.

[0022] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0023] The processor (120) can control at least one other component (e.g., hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., program (140)), and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., sensor module (176) or communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., central processing unit or application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., graphics processing unit, neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0024] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0025] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0026] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0027] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0028] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0029] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0030] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0031] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0032] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0033] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0034] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0035] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0036] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0037] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0038] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0039] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0040] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0041] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0042] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0043] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0044] The present disclosure may describe a method for obtaining an image to which a bokeh effect is applied. The bokeh effect may be referred to as, or correspond to, a blur effect represented (or processed) by a region within the image (e.g., an out-of-focus region). For example, the bokeh image (or preview bokeh image) may represent a first region that is perceived (or seen) as being in focus more clearly than a second region that is different from the first region. For example, the region perceived (or seen) as being in focus may be referred to as the focus region. For example, the degree of blur effect for the focus region may be relatively low. For example, the focus region may be relatively sharp. For example, the focus region may be emphasized more than the focus region and other regions. For example, in the bokeh image, a visual object (e.g., corresponding to a subject) within the focus region may be emphasized. For example, the bokeh image (or preview bokeh image) may be preferred by the user over an image to which no bokeh effect is applied. Users may perceive bokeh images (or preview bokeh images) as having higher artistic value than images without the bokeh effect applied.

[0045] The focus area may be related to the depth of field (DOE) of the image. In the present disclosure, the depth of field (DOE) may be referenced to or correspond to a range of depth values ​​that are perceived or seen as focused within the image. For example, depth values ​​of pixels within the focus area of ​​the image may be included in the depth. For example, the depth may be an area before and after a depth value corresponding to the focus point of the image, and may be perceived or seen as focused because the degree of blurring is relatively low. For example, the focus point may be referenced to or correspond to a region (or point) within the image that the electronic device (101) has set as the focus. For example, in the image, the sharpness of a first visual object having a depth value within the depth may be higher than the sharpness of a second visual object having a depth value outside the depth. For example, in the image, the sharpness of a first region corresponding to the depth may be higher than the sharpness of a second region different from the first region. As the depth of field becomes narrower, the range of depth that is clearly visible may be narrower. For example, if the depth value of a visual object corresponding to a subject (e.g., a person) is included in the depth of field, the area within the image occupied by the visual object may be expressed more clearly than other areas within the image. For example, the visual object may be emphasized in the image. As an example, the depth of field may be determined by the value of the aperture of the camera of the electronic device (101) (e.g., camera module (180), at least one camera (230) of FIG. 2). For example, the electronic device (101) may acquire an image with a depth of field by setting the value of the camera's aperture. The value of the aperture may correspond to the degree of aperture opening. As another example, the depth of field of the image may be determined, expressed, or acquired by providing the image acquired through the camera of the electronic device (101) to an artificial intelligence model.For example, the electronic device (101) can obtain an image in which the depth of field is expressed according to the value of the virtual aperture by setting the value of the virtual aperture. For example, even if the electronic device (101) sets the value of the virtual aperture, the aperture of the camera may not be closed or opened.

[0046] In embodiments of the present disclosure, the electronic device (101) may obtain an image with a bokeh effect applied by applying artificial intelligence. For example, the image with the bokeh effect applied may have a relatively shallow depth. The artificial intelligence model of the present disclosure (e.g., the first bokeh model (315), image synthesis model (321), single image depth model (325), multi-image depth model (327), and second bokeh model (329) of FIG. 3) may be characterized as being created through learning. Being created through learning may mean that an artificial intelligence model configured to perform a characteristic (or purpose) is created by the basic artificial intelligence model being learned using a plurality of learning data by a learning algorithm. Such learning may be performed on a device (e.g., the electronic device (101)) where the artificial intelligence according to the present disclosure is performed, and may be performed through a separate server and / or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples mentioned above.

[0047] An artificial intelligence model (e.g., the first bokeh model (315), image synthesis model (321), single image depth model (325), multi-image depth model (327), and second bokeh model (329) of FIG. 3) may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values ​​and can perform neural network operations through operations between the results of operations of the previous layer and the multiple weights. The multiple weights of the multiple neural network layers may be optimized by the learning results of the artificial intelligence model. For example, the multiple weights may be updated so that the loss value or cost value obtained from the artificial intelligence model during the learning process is reduced or minimized. According to one embodiment, the artificial intelligence model may be configured to perform not only a prediction operation but also a training operation using an artificial intelligence model trained on another device. For example, the artificial intelligence model may generate a neural network, train (or learn) the neural network, perform operations based on the neural network based on input data, generate an information signal based on the results of the operations, or retrain the neural network. According to one embodiment, at least part of the artificial intelligence model may be implemented by an external device (e.g., electronic device (102), electronic device (104), or server (108) of FIG. 1). For example, the electronic device (101) may receive a neural network generated or trained by the external device from the external device and store it in memory (e.g., memory (130) of FIG. 1, memory (210) of FIG. 2). For example, the electronic device (101) may transmit an image or information extracted from an image to the external device and receive the results of performing neural network operations on the image from the external device.For example, the electronic device (101) may receive a neural network retrained by an external device from an external device and update an artificial intelligence model stored in memory. The artificial neural network may include a Deep Neural Network (DNN), such as a Convolutional Neural Network (CNN), Region with Convolutional Neural Network (R-CNN), Region Proposal Network (RPN), Recurrent Neural Network (RNN), Stacking-based Deep Neural Network (S-DNN), State-Space Dynamic Neural Network (S-SDNN), Deconvolution Network, Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), Bidirectional Recurrent Deep Neural Network (BRDNN), Fully Convolutional Network, Long Short-Term Memory Network (LSTM), Classification Network, or Deep Q-Networks, but is not limited to the examples mentioned above.

[0048] According to embodiments of the present disclosure, an electronic device (101) may apply artificial intelligence technology to an image obtained through a camera (e.g., camera module (180), at least one camera (230) of FIG. 2). Based on applying artificial intelligence technology to the image, the electronic device (101) may obtain an image with a bokeh effect (or blur effect) applied (e.g., bokeh image, preview bokeh image). A method for obtaining a bokeh image (or preview bokeh image) may be executed within the electronic device (101). For example, the electronic device (101) may include components for obtaining a bokeh image (or preview bokeh image). Such components are described and illustrated in more detail with reference to FIG. 2 and / or FIG. 3.

[0049] FIG. 2 illustrates an example of a simplified block diagram of an electronic device (101). The device (101) may be one of various types of mobile devices, such as a laptop, smartphones with various form factors (e.g., a bar-type smartphone, a foldable-type (or multi-foldable-type) smartphone, or a rollable-type smartphone), a tablet, a cellular phone, and other similar computing devices. The electronic device (101) may be referred to as a user device, a multi-functional device, or a portable device.

[0050] Referring to FIG. 2, components of an electronic device (101) are described. The electronic device (101) may include at least one processor (200), memory (210), display (220), and / or at least one camera (230). For example, at least one processor (200), memory (210), display (220), and / or at least one camera (230) may be electrically and / or operably coupled with each other by a communication bus.

[0051] In the following, the hardware components being operatively coupled may mean that a direct or indirect connection between the hardware components is established via wired or wireless means so that a second hardware component is controlled by a first hardware component among the hardware components. The hardware components illustrated in FIG. 2 are illustrated based on different blocks, but the present disclosure is not limited thereto. For example, some of the hardware components illustrated in FIG. 2 (e.g., at least one processor (200), and at least a portion of memory (210)) may be included in a single integrated circuit such as a system on chip (SoC) or a system in package (SIP). The type and / or number of hardware components included in the electronic device (101) are not limited to those illustrated in FIG. 2. For example, the electronic device (101) may include only some of the hardware components illustrated in FIG. 2.

[0052] At least one processor (200) may include a hardware component for processing data based on executing instructions. At least one processor (200) may be configured to execute instructions stored in memory (210) individually or collectively. At least one processor (200) may include a processing circuit. For example, the hardware component for processing data may include an arithmetic and logic unit (ALU), a floating point unit (FPU), and a field programmable gate array (FPGA). For example, the hardware component for processing data may include a central processing unit (CPU), a graphic processing unit (GPU), a display processing unit (DPU), a neural processing unit (NPU), a tensor processing unit (TPU), an artificial intelligence engine, a digital signal processor (DSP), an application processor (AP), and / or a microcontroller (MCU). At least one processor (200) may include one or more cores. For example, at least one processor (200) may have the structure of a multi-core processor such as a dual core, quad core, or hexa core. At least one processor (200) may be configured to control memory (210), a display (220), and / or at least one camera (230). The at least one processor (200) of FIG. 2 may have substantially the same properties as the processor (120) of FIG. 1.

[0053] Memory (210) may include a hardware component for storing data and / or instructions that are input to and / or output from at least one processor (200). For example, instructions may represent operations and / or actions to be performed on the data by at least one processor (200) of the electronic device (101). Memory (210) may include one or more storage media. Memory (210) may include volatile memory, such as random-access memory (RAM), and / or non-volatile memory, such as read-only memory (ROM). Volatile memory may include at least one of dynamic RAM (DRAM), static RAM (SRAM), cache RAM, or pseudo SRAM (PSRAM). Non-volatile memory may include, for example, at least one of PROM (programmable ROM), EPROM (erasable PROM), EEPROM (electrically erasable PROM), flash memory, hard disk, compact disk, or EMMC (embedded multimedia card). The memory (210) of FIG. 2 may have substantially the same content as the memory (130) of FIG. 1.

[0054] The display (220) may include hardware components of an electronic device (101) used to display a screen. For example, the display (220) may include light-emitting elements and circuits (e.g., transistors) that control the light-emitting elements to emit light. For example, each of the light-emitting elements may include an OLED (organic light emitting diode) or a micro LED. However, it is not limited thereto. For example, the display (220) may include an LCD (liquid crystal display).

[0055] A display (220) of an electronic device (101) according to one embodiment may include a sensor (e.g., a touch sensor panel (TSP)) for detecting an external object (e.g., a user's finger) on the display (220). For example, based on the TSP, the electronic device (101) may detect an external object that is in contact with the display (220) or floating on the display (220). In response to the detection of the external object, the electronic device (101) may execute a function related to a specific visual object displayed at a location on the display (220) where the external object is in contact, among the visual objects displayed on the display (220). The display (220) of FIG. 2 may have substantially the same content as the display module (160) of FIG. 1.

[0056] According to one embodiment, at least one camera (230) of an electronic device (101) may be used to acquire an image. At least one camera (230) of an electronic device (101) may include one or more light sensors (e.g., a CCD (charged coupled device) sensor, a CMOS (complementary metal oxide semiconductor) sensor) that generate an electrical signal indicating the color and / or brightness of light. A plurality of light sensors included in at least one camera (230) may be arranged in the form of a two-dimensional grid (2-dimensional array). At least one camera (230) may acquire the electrical signal of each of the plurality of light sensors substantially simultaneously to generate an image comprising a plurality of pixels arranged in two dimensions corresponding to the light reaching the light sensors of the two-dimensional grid. For example, photo data captured using at least one camera (230) may mean a single image acquired from at least one camera (230). For example, video data captured using at least one camera (230) may refer to a sequence of multiple images obtained from at least one camera (230) along a specified frame rate. An electronic device (101) according to one embodiment may further include a flash light for outputting light in the direction in which at least one camera (230) is positioned to receive light. At least one camera (230) may include a lens for collecting light emitted from a subject that is the target of image capture. For example, the lens may include a wide-angle lens and / or a telephoto lens. At least one camera (230) may include an image signal processor. The image signal processor may be used to perform image processing on an image obtained through a light sensor.For example, image processing may include depth map generation, 3D modeling, image synthesis, and / or image compensation (e.g., noise reduction, resolution change, brightness adjustment, blur effect, sharpening effect). At least one camera (230) of FIG. 2 may have substantially the same content as the camera module (180) of FIG. 1 applied to it.

[0057] The electronic device (101) illustrated in the description of FIG. 2 may perform at least some of the operations illustrated in the descriptions of FIG. 3 through 8b. For example, the operations illustrated in the descriptions of FIG. 3 through 8b may be caused by (or within) the electronic device (101) under the control of at least one processor (200).

[0058] Figure 3 illustrates examples of components used to obtain an image with a bokeh effect applied.

[0059] Referring to FIG. 3, the electronic device (101) may include at least one camera (230), a preview module (301), and / or a capture module (302). The preview module (301) may be used to acquire or generate a preview bokeh image (303). The preview bokeh image (303) may be referenced as or correspond to a preview image with a bokeh effect applied. The capture module (302) may be used to acquire or generate a bokeh image (304). The bokeh image (304) may be referenced as or correspond to an image with a bokeh effect applied.

[0060] The preview module (301) may include a user interface module (311), a brightness control module (313), and / or a first bokeh model (315). The user interface module (311) may be used to perform functions related to a user interface associated with at least one camera (230). For example, the electronic device (101) may use the user interface module (311) to display the user interface through a display (e.g., a display (220)). For example, the user interface associated with at least one camera (230) may be displayed through the display (220) while running a camera application on the electronic device (101). While displaying the user interface, the electronic device (101) may receive user input to change a setting value associated with at least one camera (230). The user interface module (311) may be used to manage the changed setting value. For example, the user interface module (311) can provide the changed setting value to the brightness control module (313).

[0061] According to one embodiment, the electronic device (101) may receive user input to change the value of the aperture of at least one camera (230) while displaying a user interface. For example, the electronic device (101) may close or open the aperture of at least one camera (230) in response to user input to change the value of the aperture of at least one camera (230). For example, the electronic device (101) may close the aperture as the value of the aperture (e.g., F-number) increases. Closing the aperture may correspond to reducing the size of the aperture opening (e.g., the opening through which light passes). For example, as the aperture is closed, an image of lower brightness may be obtained. For example, as the aperture is closed, an image of deeper depth of field may be obtained. For example, the electronic device (101) may open the aperture as the value of the aperture decreases. Opening the aperture may correspond to widening the size of the aperture opening (e.g., the opening through which light passes). For example, the wider the aperture is opened, the brighter the image can be obtained. For example, the wider the aperture is opened, the shallower the image can be obtained.

[0062] According to one embodiment, the electronic device (101) may receive user input to set or change the value of a virtual aperture while displaying a user interface. The value of the virtual aperture may be referred to as a virtual F number. In response to the user input to change the value of the virtual aperture, the electronic device (101) may not directly change the aperture of at least one camera (230). For example, even if the value of the virtual aperture is changed, the actual aperture of at least one camera (230) may not be closed or opened. The electronic device (101) may perform functions such as closing or changing the aperture of at least one camera (230) based on setting the value of the virtual aperture. For example, the electronic device (101) may apply a bokeh effect to a preview image displayed through the display (220) based on setting the value of the virtual aperture. For example, the electronic device (101) can display a preview bokeh image (303) through a display (220) based on setting a virtual aperture value. For example, the electronic device (101) can change the depth of the preview image based on changing the virtual aperture value. For example, the electronic device (101) can acquire a bokeh image (304) based on setting a virtual aperture value. For example, the electronic device (101) can display a visual object corresponding to the set virtual aperture value through a user interface.

[0063] According to one embodiment, the electronic device (101) may receive user input to change the value of the shutter speed while displaying a user interface. The shutter speed may correspond to the time during which the light sensor (or image sensor) of at least one camera (230) collects light for at least one camera (230) to acquire an image. For example, the shorter the time during which the light sensor of at least one camera (230) collects light, the faster the shutter speed may be. For example, the faster the shutter speed, the clearer the fast-moving subject can be captured, but the brightness (or exposure) of the acquired image may be lower. For example, the slower the shutter speed, the higher the brightness of the acquired image may be, and the acquired image may include a high-intensity motion blur effect. The electronic device (101) may set the value of the shutter speed by the user. For example, the electronic device (101) may display a visual object corresponding to the set value of the shutter speed through the user interface.

[0064] The electronic device (101) may receive user input to change the value of the ISO (International Standards Organization) while displaying a user interface. For example, the ISO may represent the light sensitivity associated with the light sensor of at least one camera (230). For example, the brighter the image obtained, the higher the ISO, because the light sensor of at least one camera (230) receives light more sensitively. For example, the user may set the ISO value of the electronic device (101) relatively high to compensate for the light source in an environment with relatively little light source (e.g., a night environment). For example, the electronic device (101) may display a visual object corresponding to the set ISO value through the user interface.

[0065] A brightness control module (313) may be used to manage or control the brightness of a preview bokeh image (303). The brightness control module (313) may receive information about a user interface from a user interface module (311). Information about a user interface may include a virtual aperture value, a shutter speed value, and / or an ISO value. For convenience of explanation in this disclosure, a shutter speed value displayed on the user interface may be referred to as a shutter speed UX (user experience) value, and an ISO value displayed on the user interface may be referred to as an ISO UX value. In this disclosure, a shutter speed value applied to an image acquired through at least one camera (230) of the electronic device (101) may be referred to as a shutter speed system value, and an ISO value applied to an image acquired through at least one camera (230) of the electronic device (101) may be referred to as an ISO system value. The designation of terms is for convenience of explanation only and the embodiments are not limited to the designated terms.

[0066] According to one embodiment, in a first mode of an electronic device (101) in which the brightness value of a preview bokeh image (303) is changed, a brightness control module (313) may be used to determine, calculate, obtain, or generate a value of a parameter related to the brightness of the image. For example, the first mode may be referred to as a manual mode and / or an M mode. For example, in the first mode, the electronic device (101) may identify or determine the brightness value of the preview bokeh image (303) according to the value of a virtual aperture. For example, the larger the value of the virtual aperture, the smaller the value of the brightness may be. For example, the smaller the value of the virtual aperture, the larger the value of the brightness may be. The electronic device (101) may use the brightness control module (313) to determine, calculate, obtain, or generate a system value of shutter speed and / or a system value of ISO according to the determined brightness value. For example, the brightness control module (313) may calculate a parameter for expressing the determined brightness value. For example, the brightness control module (313) can calculate the system value of the shutter speed and / or the system value of the ISO. For example, the brightness control module (313) may include a look-up table. For example, the look-up table of the brightness control module (313) may indicate the relationship between the brightness value and the system value of the shutter speed and / or the relationship between the brightness value and the system value of the ISO. As an example, but not limited to, the look-up table of the brightness control module (313) may indicate the relationship between the virtual aperture value and the system value of the shutter speed and / or the relationship between the virtual aperture value and the system value of the ISO. For example, the electronic device (101) may change the value of the shutter speed from a previous value to a system value using the brightness control module (313). For example, the electronic device (101) may change the value of the ISO from a previous value to a system value using the brightness control module (313).For example, the electronic device (101) can provide the user with a user experience similar to using a digital single-lens reflex (DSLR) camera by changing the brightness of the preview bokeh image (303) according to the value of the virtual aperture set by the user.

[0067] According to one embodiment, the brightness control module (313) can determine the system value of the shutter speed after determining the system value of the ISO according to a lookup table.

[0068] According to one embodiment, the brightness control module (313) can provide a system value of shutter speed and / or a system value of ISO to at least one camera (230). At least one camera (230) can acquire at least one image according to the system value of shutter speed and / or the system value of ISO.

[0069] According to one embodiment, there may be cases where the value of the virtual aperture is relatively large, and the UX value of the ISO and / or the UX value of the shutter speed is relatively large. For example, the larger the value of the virtual aperture, the lower the system value of the ISO may be determined. For example, the greater the difference between the system value of the ISO and the UX value of the ISO, the lower the user's shooting experience may be. For example, the electronic device (101) may limit the range of change of the UX value of the ISO to prevent a reduction in the user's shooting experience. For example, the larger the value of the virtual aperture, the lower the system value of the shutter speed may be determined. For example, the greater the difference between the system value of the shutter speed and the UX value of the shutter speed, the lower the user's shooting experience may be. For example, the electronic device (101) may limit the range of change of the UX value of the shutter speed to prevent a reduction in the user's shooting experience. As an example of not limiting, if the UX value of the shutter speed is relatively large but the system value of the shutter speed is relatively low, the electronic device (101) may execute a function that provides a motion blur effect.

[0070] According to one embodiment, the brightness control module (313) can provide the first bokeh model (315) with a virtual aperture value, a brightness value, a system value of shutter speed, and / or a system value of ISO.

[0071] According to one embodiment, in a second mode of an electronic device (101) in which the brightness value of a preview bokeh image (303) is fixed to a specified value (e.g., 0 EV), a brightness control module (313) may be used to determine, calculate, obtain, or generate a UX value of shutter speed to be displayed through a user interface and / or a UX value of ISO to be displayed through a user interface. For example, the second mode may be referred to as an auto mode, an aperture mode, and / or an A mode. For example, while providing the second mode, the electronic device (101) may display the UX value of shutter speed and / or the UX value of ISO calculated by the brightness control module (313) through a user interface. For example, the electronic device (101) may determine the UX value of shutter speed and / or the UX value of ISO according to the value of a virtual aperture using a lookup table. For example, the electronic device (101) can provide the user with a user experience similar to using a digital single-lens reflex (DSLR) camera by displaying the UX value of the shutter speed and / or the UX value of the ISO through a user interface according to the value of the virtual aperture set by the user.

[0072] A first bokeh model (315) may be used to obtain a preview bokeh image (303) based on at least one image (e.g., including a preview image) obtained through at least one camera (230). For example, an electronic device (101) may obtain or generate a preview bokeh image (303) by providing a virtual aperture value and at least one image to the first bokeh model (315).

[0073] According to one embodiment, the first bokeh model (315) may be an artificial intelligence model learned using images obtained through a DSLR camera. The images obtained through the DSLR camera may include images obtained according to the aperture value of the DSLR camera. The images obtained through the DSLR camera may include bokeh effects. For example, the first bokeh model (315) may be learned to obtain a preview bokeh image (303) similar to an image captured through the DSLR camera based on at least one input image.

[0074] According to one embodiment, the preview bokeh image (303) may have a bokeh effect applied. For example, in the preview bokeh image (303), the sharpness of a first region corresponding to the depth may be higher than the sharpness of a second region different from the first region. For example, the smaller the value of the virtual aperture of the electronic device (101), the shallower the depth of the preview bokeh image (303). For example, pixels within the preview bokeh image (303) may have a blur effect applied. For example, the intensity of the blur effect in the first region corresponding to the depth may be lower than the intensity in the second region different from the first region.

[0075] According to one embodiment, the preview bokeh image (303) may include a focus point. For example, the focus point may be referenced to or correspond to a region (or point) set as a focus by the electronic device (101) in the image (e.g., preview bokeh image (303), bokeh image (304)). For example, the focus point may be set within a visual object corresponding to the subject based on the electronic device (101) identifying a visual object corresponding to the subject within the preview bokeh image (303). For example, the focus point may be set according to user input for determining the focus point while the electronic device (101) is displaying the preview bokeh image (303). For example, while the preview bokeh image (303) is being displayed through the display (220), a touch input to the display (220) may be received. For example, the focus location may be an area (or point) within the preview bokeh image (303) corresponding to the contact point of the touch input.

[0076] The depth of focus value may correspond to the focal position. For example, the depth of focus value may be referenced to or correspond to the depth value of the pixel at the focal position. For example, the depth of focus value may serve as a reference for depth. For example, among the pixels in the preview bokeh image (303), the sharpness of a pixel having a depth value equal to the depth of focus value may be relatively high. For example, the greater the difference between the depth value and the depth of focus value of a pixel in the preview bokeh image (303), the higher the intensity of the blur effect expressed (or processed) by that pixel may be. For example, the greater the depth value of a pixel in the preview bokeh image (303) than the depth of focus value, the higher the intensity of the blur effect expressed by that pixel may be. For example, the smaller the depth value of a pixel in the preview bokeh image (303) than the depth of focus value, the higher the intensity of the blur effect expressed by that pixel may be. For example, the closer the depth value of a pixel in the preview bokeh image (303) is to the depth of focus value, the lower the intensity of the blur effect expressed by that pixel may be.

[0077] According to one embodiment, the smaller the value of the virtual aperture, the smaller the depth of the preview bokeh image (303). The depth of field may be referenced or correspond to a range of depth values ​​that are perceived or seen as focused within the image (e.g., the preview bokeh image (303)). For example, the depth of field may be represented as a range between a front depth of field value and a rear depth of field value. For example, the front depth of field value may be smaller than the rear depth of field value. For example, the increase in the intensity of the blur effect according to the depth values ​​of pixels within the depth of field may be smaller than the increase in the intensity of the blur effect according to the depth values ​​of pixels outside the depth of field.

[0078] According to one embodiment, a first pixel and a second pixel may be included within the depth of the preview bokeh image (303). The depth value of the first pixel may be smaller than the focal depth value and larger than the foreground depth value. The depth value of the second pixel may be smaller than the depth value of the first pixel and larger than the foreground depth value. For example, the first intensity of the blur effect expressed by the first pixel may be lower than the second intensity of the blur effect expressed by the second pixel, but the difference between the first intensity and the second intensity may be relatively small. For example, the first pixel and the second pixel in the preview bokeh image (303) may be recognized as being in focus. A third pixel and a fourth pixel may be included outside the depth of the preview bokeh image (303). The depth value of the third pixel of the preview bokeh image (303) may be smaller than the foreground depth value. The depth value of the fourth pixel of the preview bokeh image (303) may be smaller than the depth value of the third pixel. For example, the third intensity of the blur effect expressed by the third pixel may be lower than the fourth intensity of the blur effect expressed by the fourth pixel. For example, in the preview bokeh image (303), the degree of blurring of the third pixel may be smaller than the degree of blurring of the fourth pixel. For example, the difference between the depth value of the first pixel and the depth value of the second pixel may correspond to or be equal to the difference between the depth value of the third pixel and the depth value of the fourth pixel. For example, the difference between the first intensity and the second intensity may be smaller than the difference between the third intensity and the fourth intensity.

[0079] According to one embodiment, a first pixel and a second pixel may be included within the depth of the preview bokeh image (303). The depth value of the first pixel may be greater than the focal depth value and smaller than the back depth of field value. The depth value of the second pixel may be greater than the depth value of the first pixel and smaller than the front depth of field value. For example, the first intensity of the blur effect expressed by the first pixel may be lower than the second intensity of the blur effect expressed by the second pixel, but the difference between the first intensity and the second intensity may be relatively small. For example, the first pixel and the second pixel in the preview bokeh image (303) may be recognized as being in focus. A third pixel and a fourth pixel may be included outside the depth of the preview bokeh image (303). The depth value of the third pixel of the preview bokeh image (303) may be greater than the back depth of field value. The depth value of the fourth pixel of the preview bokeh image (303) may be greater than the depth value of the third pixel. For example, the third intensity of the blur effect expressed by the third pixel may be lower than the fourth intensity of the blur effect expressed by the fourth pixel. For example, in the preview bokeh image (303), the degree of blurring of the third pixel may be smaller than the degree of blurring of the fourth pixel. For example, the difference between the depth value of the first pixel and the depth value of the second pixel may correspond to or be equal to the difference between the depth value of the third pixel and the depth value of the fourth pixel. For example, the difference between the first intensity and the second intensity may be smaller than the difference between the third intensity and the fourth intensity.

[0080] The capture module (302) may include an image synthesis model (321), a depth module (323), a second bokeh model (329), and / or a brightness adjustment module (331). The image synthesis model (321) may be used to acquire an intermediate image based on at least one image acquired through at least one camera (230). For example, the electronic device (101) may refer to at least one image as at least one frame. For example, at least one image may be in a raw file format. For example, an image in a raw file format may be referred to as an original image and may have a relatively high resolution. For example, the brightness of at least one image may differ from one another. For example, at least one image may include an image of -1 EV (exposure value), an image of -3 EV, and / or an image of -5 EV.

[0081] According to one embodiment, an electronic device (101) can acquire an intermediate image by providing at least one image to an image synthesis model (321). For example, the intermediate image may have undergone image processing on at least one image based on the image synthesis model (321). For example, the image synthesis model (321) may be an artificial intelligence model trained for multi-frame synthesis of at least one input image. For example, multi-frame synthesis may correspond to a technique for generating or acquiring one image (e.g., an intermediate image) by synthesizing at least one image. For example, the quality of the intermediate image may be higher than the quality of at least one image. For example, the resolution of the intermediate image may be higher than the resolution of at least one image. For example, the noise intensity of the intermediate image may be lower than the noise intensity of at least one image. For example, the intermediate image may be an image in which HDR (high dynamic range) has been performed. For example, the color gamut of the intermediate image may be wider than the color gamut of at least one image. For example, the electronic device (101) can provide an intermediate image to the second bokeh model (329).

[0082] The depth module (323) can be used to identify depth values ​​of pixels within an intermediate image. The depth module (323) can use at least one image obtained through at least one camera (230). For example, the electronic device (101) can obtain depth values ​​of pixels within an intermediate image by providing at least one image to the depth module (323).

[0083] The depth module (323) may include a single image depth model (325) and / or a multi-image depth model (327). The single image depth model (325) may be used to obtain, calculate, or determine depth values ​​of pixels of an intermediate image based on a single image. For example, the single image depth model (325) may be an artificial intelligence model trained based on a single artificial-intelligence depthmap algorithm. For example, the single artificial-intelligence depthmap algorithm may include a hair matting algorithm. For example, as the hair matting algorithm is included in the single artificial-intelligence depthmap algorithm, the performance of the multi-image depth model (327) (e.g., the accuracy of the depth values ​​of the pixels is improved) may be enhanced. For example, as the hair matting algorithm is included in the single artificial intelligence depth map algorithm, the boundary effect of the visual object (e.g., corresponding to the subject) of the bokeh image (304) obtained by the electronic device (101) can be relatively smooth or natural.

[0084] A multi-image depth model (327) may be used to obtain, calculate, or determine depth values ​​of pixels of an intermediate image based on two or more images. For example, the multi-image depth model (327) may be an artificial intelligence model trained based on a stereo artificial-intelligence depthmap algorithm. For example, the stereo artificial-intelligence depthmap algorithm may include a hair matting algorithm. For example, as the hair matting algorithm is included in the stereo artificial-intelligence depthmap algorithm, the performance of the multi-image depth model (327) (e.g., the accuracy of the depth values ​​of the pixels is improved) may be enhanced. For example, the electronic device (101) may provide the depth values ​​(or depth map) of the pixels of the intermediate image to the second bokeh model (329).

[0085] A second bokeh model (329) may be used to obtain a bokeh image (304) based on an intermediate image. For example, an electronic device (101) may obtain or generate a bokeh image (304) by providing depth values ​​of pixels of the intermediate image and / or the intermediate image to the second bokeh model (329).

[0086] According to one embodiment, the second bokeh model (329) may be an artificial intelligence model learned using images obtained through a DSLR camera. The images obtained through the DSLR camera may include images obtained according to the aperture value of the DSLR camera. The images obtained through the DSLR camera may include bokeh effects. For example, the second bokeh model (329) may be learned to obtain a bokeh image (304) similar to an image captured through the DSLR camera based on an input intermediate image.

[0087] According to one embodiment, the bokeh image (304) may be an image to which a bokeh effect has been applied. For example, in the bokeh image (304), the sharpness of a first region corresponding to the depth may be higher than the sharpness of a second region different from the first region. For example, the smaller the value of the virtual aperture of the electronic device (101), the shallower the depth of the bokeh image (304). For example, pixels within the bokeh image (304) may have a blur effect applied. For example, the intensity of the blur effect in the first region corresponding to the depth may be lower than the intensity in the second region different from the first region.

[0088] According to one embodiment, the bokeh image (304) may include a focus point. The depth of focus value may correspond to the focus point. For example, the depth of focus value may be referenced to or correspond to the depth value of the pixel at the focus point. For example, the depth of focus value may serve as a reference for depth. For example, among the pixels in the bokeh image (304), the sharpness of a pixel having a depth value equal to the depth of focus value may be relatively high. For example, the greater the difference between the depth value of a pixel in the bokeh image (304) and the depth of focus value, the higher the intensity of the blur effect expressed (or processed) by that pixel may be. For example, the greater the depth value of a pixel in the bokeh image (304) than the depth of focus value, the higher the intensity of the blur effect expressed by that pixel may be. For example, the smaller the depth value of a pixel in the bokeh image (304) than the depth of focus value, the higher the intensity of the blur effect expressed by that pixel may be. For example, the closer the depth value of a pixel in the bokeh image (304) is to the depth of focus value, the lower the intensity of the blur effect expressed by that pixel may be.

[0089] According to one embodiment, the capture module (302) may include a brightness control module (331). The brightness control module (331) may be used to change the brightness value of the bokeh image (304). For example, the brightness control module (331) may change the brightness value of the bokeh image (304) to a brightness value according to a virtual aperture value set by the user. For example, the brightness value according to the virtual aperture value may be determined by the brightness control module (313). For example, the brightness control module (331) may change the brightness value of the bokeh image (304) to a brightness value of the preview bokeh image (303). For example, the preview bokeh image (303) may be provided to the brightness control module (331).

[0090] Figure 4 illustrates an example of a user interface displayed while running a camera application.

[0091] Referring to FIG. 4, the electronic device (101) can display a user interface through a display (220). To display the user interface, the electronic device (101) may use the user interface module of FIG. 3 (e.g., user interface module (311)).

[0092] The user interface of the electronic device (101) may include a visual object (401). For example, the visual object (401) may represent the current value of a virtual aperture. The visual object (401) may be used to set the value of the virtual aperture. For example, the electronic device (101) may set or change the value of the virtual aperture based on receiving input regarding the visual object (401). For example, the electronic device (101) may display a visual object (410) on the user interface in response to input regarding the visual object (401). For example, the electronic device (101) may set or change the value of the virtual aperture with input regarding the visual object (410). For example, the electronic device (101) may change the appearance of the visual object (401) in response to input regarding the visual object (410). For example, the electronic device (101) may display a changed virtual aperture value on the visual object (401) in response to input to the visual object (410). For example, input to the visual object (410) may include touch input, scroll input, and / or swipe input. However, embodiments are not limited. For example, input to the visual object (410) may include voice input.

[0093] According to one embodiment, the user interface of the electronic device (101) may include a visual object (402). For example, the visual object (402) may represent a value of shutter speed (e.g., a UX value of shutter speed). The visual object (402) may be used to set the value of shutter speed. For example, the electronic device (101) may set or change the value of shutter speed based on receiving input to the visual object (402). For example, input to the visual object (402) may include touch input, scroll input, and / or swipe input. However, the embodiments are not limited. For example, input to the visual object (402) may include voice input.

[0094] According to one embodiment, the user interface of the electronic device (101) may include a visual object (403). For example, the visual object (403) may represent a value of ISO (e.g., a UX value of ISO). The visual object (403) may be used to set the value of ISO. For example, the electronic device (101) may set or change the value of ISO based on receiving input to the visual object (403). For example, input to the visual object (403) may include touch input, scroll input, and / or swipe input. However, the embodiments are not limited. For example, input to the visual object (403) may include voice input.

[0095] According to one embodiment, the user interface of the electronic device (101) may include a visual object (420). For example, the visual object (420) may be used to obtain an image (e.g., bokeh image (304)) corresponding to a preview image (e.g., preview bokeh image (303)). For example, the visual object (420) may be used to store an image corresponding to a preview image. For example, the visual object (420) may be used as a shooting input. For example, the electronic device (101) may utilize the functions of the capture module (302) illustrated in FIG. 3 based on receiving input for the visual object (420).

[0096] According to one embodiment, the appearance of a visual object (420) may change according to the value of a virtual aperture. For example, the appearance of the visual object (420) may change according to user input to the visual object (410). For example, the appearance of the visual object (420) may change while user input (e.g., swipe input, scroll input) to the visual object (410) is received. For example, the appearance of the visual object (420) may represent an aperture. For example, the size of the aperture opening represented may change according to user input (e.g., swipe input, scroll input) to the visual object (410). For example, the aperture opening represented by the visual object (420) may become larger according to user input (e.g., swipe input, scroll input) that causes the value of the virtual aperture to become smaller. For example, the appearance of the visual object (420) may be represented such that the aperture opening becomes larger as the value of the virtual aperture becomes smaller. For example, the appearance of the visual object (420) can represent an aperture having a larger size opening as the value of the virtual aperture set in the electronic device (101) becomes smaller.

[0097] For example, a visual object (421), a visual object (422), and / or a visual object (423) may be examples of a visual object (420). For example, the value of a virtual aperture corresponding to a visual object (421) may be smaller than the value of a virtual aperture corresponding to a visual object (422). For example, the value of a virtual aperture corresponding to a visual object (422) may be smaller than the value of a virtual aperture corresponding to a visual object (423). For example, the electronic device (101) may visually provide the user with the size of the value of a virtual aperture by changing the appearance of the visual object (420) according to the value of the virtual aperture. The user experience of the electronic device (101) may be enhanced.

[0098] FIGS. 5A and 5B illustrate examples of images (e.g., preview bokeh image (303), bokeh image (304)) obtained based on bokeh models (e.g., first bokeh model (315), second bokeh model (329)).

[0099] Referring to FIG. 5a, the image (501) may be an example of a preview bokeh image (303). The image (501) may be an example of a bokeh image (304). The image (501) may include a visual object (510), a visual object (520), and / or a visual object (530). The visual object (510) may correspond to a subject (e.g., a person).

[0100] According to one embodiment, when an image (501) is captured, the focus may be set on the face of the subject. In the image (501), the focus position (or focus) may be included in the area corresponding to the face within the visual object (510). A pixel having a focus depth value, which is the depth value of the focus position, may be included in the area corresponding to the face within the visual object (510). The smaller the difference between the depth value and the focus depth value of a pixel within the image (501), the higher the sharpness of that pixel may be. The larger the difference between the depth value and the focus depth value of a pixel within the image (501), the higher the intensity of the blur effect expressed by that pixel may be. For example, among the pixels within the image (501), the sharpness of the pixel corresponding to the focus depth value may be the highest. For example, among the pixels within the image (501), the sharpness of the pixel corresponding to the focus depth value may be the highest. For example, as the difference between the pixel depth value and the focus depth value of the pixels within the image (501) increases, the intensity of the blur effect expressed by that pixel may gradually increase.

[0101] According to one embodiment, the difference between the depth value and the focal depth value of pixels within the area corresponding to the right arm of the visual object (510) may be greater than the difference between the depth value and the focal depth value of pixels within the area corresponding to the face of the visual object (510). For example, the intensity of the blur effect expressed by the area corresponding to the right arm of the visual object (510) may be higher than the intensity of the blur effect expressed by the area corresponding to the face of the visual object (510).

[0102] According to one embodiment, the depth value of a first pixel within an area corresponding to the elbow of a visual object (510) may be smaller than the focal depth value. The depth value of a second pixel within an area corresponding to the finger of a visual object (510) may be smaller than the depth value of the first pixel. For example, the intensity of the blur effect expressed by the area corresponding to the elbow of a visual object (510) may be lower than the intensity of the blur effect expressed by the area corresponding to the finger of a visual object (510). For example, the image (501) may have a progressively different intensity of the blur effect expressed by the corresponding pixel depending on the depth values ​​of the pixels within the image (501).

[0103] According to one embodiment, the pixels (511) may be pixels within an area corresponding to the face of the visual object (510). Since the depth value of each pixel (511) has a value relatively close to the focal depth value, the clarity may be high. The visual object (520) may be positioned in front of the visual object (510). The pixels (521) may be pixels within an area occupied by the visual object (520). The difference between the depth value of each pixel (521) and the focal depth value may be greater than the difference between the depth value of each pixel (511) and the focal depth value. For example, the intensity of the blur effect expressed by the pixels (521) may be higher than the intensity of the blur effect expressed by the pixels (511).

[0104] According to one embodiment, the pixels (531) may be pixels within an area occupied by a visual object (530). The difference between the depth value and the focal depth value of each of the pixels (531) may be greater than the difference between the depth value and the focal depth value of each of the pixels (511). For example, the intensity of the blur effect expressed by the pixels (531) may be higher than the intensity of the blur effect expressed by the pixels (511).

[0105] According to one embodiment, the difference between the depth value of each pixel (531) and the focal depth value may be greater than the difference between the depth value of each pixel (521) and the focal depth value. For example, the intensity of the blur effect expressed by the pixels (531) may be higher than the intensity of the blur effect expressed by the pixels (521).

[0106] Due to the blur effect expressed by the visual object (530) positioned behind the visual object (510), as well as the blur effect expressed by the visual object (520) positioned in front of the visual object (510), the visual object (510) can be emphasized in the image (501) among the visual object (510), the visual object (520), and the visual object (530). Additionally, due to the blur effect expressed by the area corresponding to the arm of the visual object (510), the area corresponding to the face of the visual object (510) can be emphasized. The electronic device (101) can provide the user with an image (501) in which the area corresponding to the face of the visual object (510) is emphasized using a bokeh model (e.g., a first bokeh model (315), a second bokeh model (329)).

[0107] Referring to FIG. 5b, image (504) may be an image obtained by photographing a subject (e.g., a man) without using a bokeh model (e.g., a first bokeh model (315), a second bokeh model (329)). Image (505) may be an image obtained by photographing a subject (e.g., a man) using a bokeh model (e.g., a first bokeh model (315), a second bokeh model (329)). Image (505) may be an example of a preview bokeh image (303). Image (505) may be an example of a bokeh image (304).

[0108] The image (504) may include a visual object (540) corresponding to a subject. A first region occupied by the visual object (540) in the image (504) may be clearer than a second region within the image (504) that is different from the first region. Pixels (541) may be pixels adjacent to the boundary between the first region and the second region within the region occupied by the visual object (540). For example, pixels (541) may be pixels within the region occupied by the left part of the visual object (540) (e.g., left relative to the subject). For example, the intensity of the blur effect expressed (or processed) by the pixels (541) may be relatively low. For example, the intensity of the blur effect expressed by the pixels (541) may be substantially the same as the intensity of the blur effect expressed by other pixels within the visual object (540).

[0109] The image (505) may include a visual object (550) corresponding to a subject. The greater the difference between the depth value of a pixel within the image (505) and the depth of focus value corresponding to the focal position, the higher the intensity of the blur effect expressed by that pixel may be. As the difference between the pixel depth value and the depth of focus value increases, the intensity of the blur effect expressed by that pixel within the image (505) may gradually increase. In other words, if the depth value of the first pixel within the visual object (550) and the depth value of the second pixel within the visual object (550) are different, the intensity of the blur effect expressed by the first pixel and the intensity of the blur effect expressed by the second pixel may be different.

[0110] The pixels (551) may be pixels adjacent to the boundary between the first and second regions within the area occupied by the visual object (550). For example, the pixels (551) may be pixels within the area occupied by the left part of the upper surface of the visual object (550) (e.g., left relative to the subject). For example, the focal point in the image (505) may be included in the area corresponding to the face of the visual object (550). For example, a blur effect may be applied to the pixels (551) based on the difference between the depth values ​​of the pixels (551) and the focal depth value.

[0111] According to one embodiment, the positions of the pixels (551) may correspond to the positions of the pixels (541). The intensity of the blur effect expressed by the pixels (551) may be higher than the intensity of the blur effect expressed by the pixels (541). Because of the blur effect expressed by the pixels (551), the user may perceive the image (505) as more natural or smoother than the image (504).

[0112] FIG. 6 illustrates an example of metadata for a bokeh image (e.g., bokeh image (304)).

[0113] Referring to FIG. 6, the electronic device (101) can display metadata (610) of the bokeh image (304) through a display (220). For example, the metadata (610) can be stored in the file of the bokeh image (304) in the EXIF ​​(exchangeable image file format) format.

[0114] According to one embodiment, metadata (610) may represent attributes of the bokeh image (304). Metadata (610) may include data (620), data (630), and / or data (640). Data (620) may represent the capacity of the bokeh image (304) (e.g., 41.79 MB (megabyte)), the resolution of the bokeh image (304) (e.g., 4000x3000), and / or the number of pixels of the bokeh image (304) (e.g., 12 MP (megapixel)). Additionally, data (620) may represent the values ​​of parameters associated with at least one camera (e.g., at least one camera (230)) that was set to acquire the bokeh image (304). For example, the data (620) may represent a system value of ISO (e.g., 2500), a focal length of a lens of at least one camera (230) (e.g., 23mm), an exposure value of a bokeh image (e.g., 0.0EV), an aperture value of at least one camera (230) (e.g., F 1.7), and / or a system value of a shutter speed (e.g., 1 / 24s).

[0115] According to one embodiment, data (630) may indicate that a bokeh image (304) is acquired in an electronic device (e.g., electronic device (101)) in which a virtual aperture value is set. For example, data (630) may include a set virtual aperture value (e.g., F 16.0).

[0116] According to one embodiment, the data (640) may represent a UX value of ISO (e.g., 200) and / or a UX value of shutter speed (e.g., 1 / 10s) displayed in a user interface according to a user interface module (e.g., user interface module (311)) while the bokeh image (304) is being acquired.

[0117] FIGS. 7a and 7b illustrate examples of images including visual objects corresponding to a light source. Each of the images (701) and (702) in FIG. 7a may be an example of the preview bokeh image (303) and / or bokeh image (304) of FIG. 3. Each of the images (703) and (704) in FIG. 7b may be an example of the preview bokeh image (303) and / or bokeh image (304) of FIG. 3.

[0118] Referring to FIG. 7a, each of the images (701) and (702) may represent an image of an urban environment captured through an electronic device (101). The image (702) may represent an image of an urban environment captured according to a virtual aperture value greater than the virtual aperture value corresponding to the image (701).

[0119] The image (701) may include visual objects (710) corresponding to light sources. Additionally, the image (702) may include visual objects (720) corresponding to light sources. For example, a visual object (710-1) in the image (701) may correspond to a visual object (720-1) in the image (702). For example, the shape of the visual object (710-1) (e.g., circular) may be rounder than the shape of the visual object (720-1) (e.g., elliptical). For example, the intensity of the blur effect expressed (or processed) by the visual object (710-1) may be higher than the intensity of the blur effect expressed by the visual object (720-1). For example, the size of the visual object (710-1) may be larger than the size of the visual object (720-1). Although not shown in FIG. 7a, the intensity of the light splitting effect appearing in the visual object (710-1) may be lower than the intensity of the light splitting effect appearing in the visual object (720-1).

[0120] According to one embodiment, a visual object (710-2) in an image (701) may correspond to a visual object (720-2) in an image (702). For example, the shape of the visual object (710-2) may be rounder than the shape of the visual object (720-2). For example, the intensity of the blur effect expressed by the visual object (710-2) may be higher than the intensity of the blur effect expressed by the visual object (720-2). For example, the size of the visual object (710-2) may be larger than the size of the visual object (720-2). Although not shown in FIG. 7a, the intensity of the light splitting effect appearing in the visual object (710-2) may be lower than the intensity of the light splitting effect appearing in the visual object (720-2).

[0121] According to one embodiment, a visual object (710-3) in an image (701) may correspond to a visual object (720-3) in an image (702). For example, the shape of the visual object (710-3) may be rounder than the shape of the visual object (720-3). For example, the intensity of the blur effect expressed by the visual object (710-3) may be higher than the intensity of the blur effect expressed by the visual object (720-3). For example, the size of the visual object (710-3) may be larger than the size of the visual object (720-3). Although not shown in FIG. 7a, the intensity of the light splitting effect appearing in the visual object (710-3) may be lower than the intensity of the light splitting effect appearing in the visual object (720-3).

[0122] The features of the image (701) and / or image (702) described in FIG. 7a may appear or be caused by the bokeh model of FIG. 3 (e.g., first bokeh model (315), second bokeh model (329)) being learned using images acquired (or captured) through a digital single-lens reflex (DSLR) camera. For example, the images acquired through the DSLR camera may be images acquired by varying the aperture values ​​of the DSLR camera.

[0123] As a non-limiting example, the shape of a visual object (e.g., visual objects (710), visual objects (720)) corresponding to a light source within an image (e.g., image (701), image (702)) obtained by the electronic device (101) using a bokeh model may be round as the visual object is positioned closer to the center of the image. For example, the shape of a first visual object (e.g., corresponding to a light source) positioned closer to the center of the image may be circular, and the shape of a second visual object (e.g., corresponding to a light source) positioned closer to the edge of the image may be elliptical.

[0124] Referring to FIG. 7b, the image (703) may represent an image of a road environment captured by an electronic device (e.g., electronic device (101)). The image (704) may represent an image of a street light captured according to a virtual aperture value greater than the virtual aperture value corresponding to the image (703).

[0125] The image (703) may include a visual object (730) corresponding to a light source. For example, the visual object (730) may represent or correspond to an external object (e.g., a light source of a street light). The image (704) may include a visual object (740) corresponding to a light source. For example, the visual object (740) may represent or correspond to an external object (e.g., a light source of a street light). The visual object (730) may correspond to the visual object (740). For example, the external object (e.g., a light source of a street light) corresponding to the visual object (730) and the external object (e.g., a light source of a street light) corresponding to the visual object (740) may be the same. For example, the difference between the representation (or visual effect) of the visual object (730) and the representation (or visual effect) of the visual object (740) may be based on the difference between the value of the virtual aperture corresponding to the image (703) and the value of the virtual aperture corresponding to the image (704).

[0126] The intensity of the light splitting effect represented by the visual object (730) may be lower than the intensity of the light splitting effect represented by the visual object (740). The light splitting effect may be referred to as an optical phenomenon in which light rays spread out radially because light emitted from a light source is diffracted by an aperture. Due to the difference in the intensity of the light splitting effect applied to (or represented by) each of the visual object (730) and the visual object (740), the representation of the visual object (730) may be distinguished from the representation of the visual object (740). For example, the shape of the visual object (730) may be distinguished from the shape of the visual object (740). For example, the shape of the visual object (730) may be relatively round. For example, the visual object (730) may represent a phenomenon in which light from a light source spreads relatively uniformly in all directions. For example, the visual object (730) may be represented as a circle and / or an ellipse. For example, the shape of the visual object (740) may be a starburst shape. For example, the starburst shape may be referred to as a shape in which multiple rays of light radiate outward from the center of a light source. For example, the visual object (740) may represent multiple rays of light radiating from the light source relatively clearly. For example, the clarity of the rays represented by the visual object (740) may be higher than the clarity of the rays represented by the visual object (730). The starburst shape may be referred to by a star shape, an asterisk shape (e.g., '*' shape), a radial light ray pattern, a multi-directional light structure, a light ray pattern, and / or equivalent technical terms.

[0127] The features of the image (703) and / or image (704) described in FIG. 7b may appear or be caused by the bokeh model of FIG. 3 (e.g., first bokeh model (315), second bokeh model (329)) being learned using images acquired (or captured) through a digital single-lens reflex (DSLR) camera. For example, the images acquired through the DSLR camera may be images acquired by varying the aperture values ​​of the DSLR camera.

[0128] FIG. 8a illustrates examples of operations of an electronic device for acquiring a preview bokeh image. The electronic device of FIG. 8a may be an example of the electronic device (101) of FIG. 1 to 7b.

[0129] Referring to FIG. 8a, in operation 801, an electronic device (101) (e.g., at least one processor (200)) may receive user input for setting a virtual aperture value. For example, the electronic device (101) may receive user input while displaying a preview image acquired through at least one camera (e.g., at least one camera (230)) via a display (e.g., display (220)). For example, the electronic device (101) may receive user input for setting a virtual aperture value through a user interface displayed using a user interface module (e.g., user interface module (311)). For example, user input for setting a virtual aperture value may include input for a visual object within the user interface (e.g., visual object (401), visual object (410)). For example, descriptions of the visual object (401) and / or the visual object (410) of FIG. 4 may be referenced for user input to set the value of the virtual aperture.

[0130] In operation 803, an electronic device (101) (e.g., at least one processor (200)) may obtain a preview bokeh image (e.g., preview bokeh image (303)) based on providing a virtual aperture value and a preview image to a first bokeh model (e.g., first bokeh model (315)). The preview bokeh image (303) may express a visual effect according to the virtual aperture value. For example, the preview bokeh image (303) expressing the visual effect may be referred to as having the visual effect applied or processed. For example, the preview bokeh image (303) may be an image in which a bokeh effect is expressed (or applied). For example, in the preview bokeh image (303), the sharpness of a first region corresponding to the depth may be higher than the sharpness of a second region different from the first region. For example, the smaller the value of the virtual aperture of the electronic device (101), the shallower the depth of the preview bokeh image (303). For example, pixels within the preview bokeh image (303) may have a blur effect applied. For example, the intensity of the blur effect in the first region corresponding to the depth may be lower than the intensity in the second region different from the first region.

[0131] According to one embodiment, the preview bokeh image (303) may include a focus point. For the focus point, the descriptions of the focus point in FIG. 3 may be referenced. The depth of focus value may correspond to the focus point. For example, the depth of focus value may be referenced to or correspond to the depth value of the pixel at the focus point. For example, the depth of focus value may serve as a reference for depth. For example, among the pixels in the preview bokeh image (303), the sharpness of a pixel having a depth value equal to the depth of focus value may be relatively high. For example, each pixel in the preview bokeh image (303) may express a blur effect of high intensity as the difference between the depth value and the depth of focus value of each pixel is greater. For example, as the difference between the depth value and the depth of focus value of a pixel in the preview bokeh image (303) is greater, the intensity of the blur effect expressed by that pixel may be higher. For example, the greater the depth value of a pixel in the preview bokeh image (303) is compared to the depth of focus value, the higher the intensity of the blur effect expressed by that pixel may be. For example, the smaller the depth value of a pixel in the preview bokeh image (303) is compared to the depth of focus value, the higher the intensity of the blur effect expressed by that pixel may be. For example, the closer the depth value of a pixel in the preview bokeh image (303) is to the depth of focus value, the lower the intensity of the blur effect expressed by that pixel may be.

[0132] According to one embodiment, the intensity of the blur effect expressed by each pixel in the preview bokeh image (303) may be based on the value of a virtual aperture. For example, when the value of the virtual aperture is a first value, a pixel having a depth value in the preview bokeh image (303) may express a blur effect of a first intensity. For example, when the value of the virtual aperture is a second value smaller than the first value, the pixel having the depth value in the preview bokeh image (303) may express a blur effect of a second intensity higher than the first intensity.

[0133] According to one embodiment, a preview bokeh image (303) obtained according to the value of a virtual aperture can be compared with a preview image. For example, a high intensity of the blur effect may include a relatively high intensity of the blur effect and / or an absolutely high intensity of the blur effect. To compare the intensity of the blur effect, the intensity of the blur effect for a first pixel of the preview image and the intensity of the blur effect for a second pixel of the preview bokeh image (303) can be compared. For example, the preview image may correspond to the preview bokeh image (303). For example, the position of the first pixel in the preview image may correspond to the position of the second pixel in the preview bokeh image (303).

[0134] According to one embodiment, when the value of the virtual aperture is a first value, the difference between the pixel depth value of the preview bokeh image (303) expressing the blur effect of the reference intensity and the focal depth value of the preview bokeh image (303) may be a second value. For example, when the value of the virtual aperture is a third value smaller than the first value, the difference between the pixel depth value of the preview bokeh image (303) expressing the blur effect of the reference intensity and the focal depth value of the preview bokeh image (303) may be a fourth value. For example, the second value may be greater than the fourth value. For example, the reference intensity may correspond to the boundary of the depth of field. For example, the reference intensity may be referenced as the lowest intensity of the blur effect that is perceived or seen by the user as being in focus.

[0135] According to one embodiment, the brightness of the preview bokeh image (303) may be determined based on the value of a virtual aperture. For example, an electronic device (101) may determine the value of the virtual aperture. For example, the electronic device (101) may determine the value of the brightness of the preview bokeh image (303) according to the value of the virtual aperture. For example, the larger the value of the virtual aperture, the smaller the value of the brightness may be. For example, the smaller the value of the virtual aperture, the larger the value of the brightness may be. For example, the electronic device (101) may change the value of a camera parameter to express the determined value of brightness. For example, the camera parameter may include a parameter related to the brightness of the image. For example, the parameter related to the brightness of the image may include a system value of the shutter speed and / or a system value of ISO. For example, the descriptions of the brightness control module (313) of FIG. 3 may be referenced for the system value of the shutter speed and / or the system value of ISO.

[0136] In operation 805, an electronic device (101) (e.g., at least one processor (200)) can display a preview bokeh image (303) through a display (e.g., a display (220)).

[0137] FIG. 8b illustrates examples of operations of an electronic device for acquiring bokeh images.

[0138] Referring to FIG. 8b, in operation 821, an electronic device (101) (e.g., at least one processor (200)) may receive user input for acquiring a bokeh image (e.g., bokeh image (304)). For example, user input for acquiring a bokeh image (304) may be received while a preview bokeh image (e.g., preview bokeh image (303)) is displayed through a display (e.g., display (220)). For example, operation 821 may be an operation following operation 805. For example, the electronic device (101) may receive user input for acquiring a bokeh image (304) through a user interface displayed using a user interface module (e.g., user interface module (311)). For example, user input for acquiring a bokeh image (304) may include input for a visual object representing a shooting button (or capture button) (e.g., visual object (420), visual object (421), visual object (422), visual object (423) of FIG. 4).

[0139] In operation 823, an electronic device (101) (e.g., at least one processor (200)) can acquire an intermediate image through at least one camera (e.g., at least one camera (230)). For example, the electronic device (101) can acquire an intermediate image by providing at least one image acquired through at least one camera (230) to an image synthesis model (e.g., image synthesis model (321)). For example, the quality of the intermediate image may be higher than the quality of at least one image. For example, the resolution of the intermediate image may be higher than the resolution of at least one image. For the intermediate image, the descriptions of the image synthesis model (321) of FIG. 3 may be referenced.

[0140] In operation 825, an electronic device (101) (e.g., at least one processor (200)) can acquire a bokeh image (304) based on providing intermediate image and virtual aperture values ​​to a second bokeh model (e.g., second bokeh model (329)). For example, the virtual aperture value may be a virtual aperture value set in operation 801. For example, the virtual aperture value may be set through a user interface displayed using a user interface module (e.g., user interface module (311)).

[0141] According to one embodiment, the bokeh image (304) may express a visual effect according to the value of the virtual aperture. For example, the bokeh image (304) expressing the visual effect may be referred to as having the visual effect applied or processed. For example, the bokeh image (304) may be an image in which the bokeh effect is expressed (or applied). For example, in the bokeh image (304), the sharpness of a first region corresponding to the depth may be higher than the sharpness of a second region different from the first region. For example, the smaller the value of the virtual aperture of the electronic device (101), the shallower the depth of the bokeh image (304). For example, pixels within the bokeh image (304) may have a blur effect processed. For example, the intensity of the blur effect in the first region corresponding to the depth may be lower than the intensity in the second region different from the first region.

[0142] According to one embodiment, the bokeh image (304) may include a focus point. The depth of focus value may correspond to the focus point. For example, the depth of focus value may be referenced to or correspond to the depth value of the pixel at the focus point. For example, the depth of focus value may serve as a reference for depth. For example, among the pixels in the bokeh image (304), the sharpness of a pixel having a depth value equal to the depth of focus value may be relatively high. For example, each pixel in the bokeh image (304) may express a high-intensity blur effect as the difference between the depth value and the depth of focus value of each pixel is greater. For example, the greater the difference between the depth value and the depth of focus value of a pixel in the bokeh image (304), the higher the intensity of the blur effect expressed by that pixel may be. For example, the greater the depth value of a pixel in the bokeh image (304) is than the depth of focus value, the higher the intensity of the blur effect expressed by that pixel may be. For example, the smaller the depth value of a pixel in the bokeh image (304) is compared to the focal depth value, the higher the intensity of the blur effect expressed by that pixel may be. For example, the closer the depth value of a pixel in the bokeh image (304) is to the focal depth value, the lower the intensity of the blur effect expressed by that pixel may be.

[0143] According to one embodiment, the intensity of the blur effect expressed by each pixel in the bokeh image (304) may be based on the value of a virtual aperture. For example, when the value of the virtual aperture is a first value, a pixel having a depth value in the bokeh image (304) may express a blur effect of a first intensity. For example, when the value of the virtual aperture is a second value smaller than the first value, the pixel having the depth value in the bokeh image (304) may express a blur effect of a second intensity higher than the first intensity.

[0144] According to one embodiment, a bokeh image (304) obtained according to the value of a virtual aperture can be compared with an intermediate image. For example, a high intensity of the blur effect may include a relatively high intensity of the blur effect and / or an absolutely high intensity of the blur effect. To compare the intensity of the blur effect, the intensity of the blur effect for a first pixel of the intermediate image and the intensity of the blur effect for a second pixel of the bokeh image (304) can be compared. For example, the intermediate image may correspond to the bokeh image (304). For example, the position of the first pixel in the intermediate image may correspond to the position of the second pixel in the bokeh image (304).

[0145] According to one embodiment, when the value of the virtual aperture is a first value, the difference between the pixel depth value of the bokeh image (304) expressing the blur effect of the reference intensity and the focal depth value of the bokeh image (304) may be a second value. For example, when the value of the virtual aperture is a third value smaller than the first value, the difference between the pixel depth value of the bokeh image (304) expressing the blur effect of the reference intensity and the focal depth value of the bokeh image (304) may be a fourth value. For example, the second value may be greater than the fourth value. For example, the reference intensity may correspond to the boundary of the depth of field. For example, the reference intensity may be referenced as the lowest intensity of the blur effect that is perceived or seen by the user as being in focus.

[0146] According to one embodiment, the brightness of the bokeh image (304) may be determined based on the value of a virtual aperture. For example, an electronic device (101) may determine the value of a virtual aperture. For example, the electronic device (101) may determine the value of the brightness of the bokeh image (304) according to the value of the virtual aperture. For example, the larger the value of the virtual aperture, the smaller the value of the brightness may be. For example, the smaller the value of the virtual aperture, the larger the value of the brightness may be. For example, the electronic device (101) may change the value of a camera parameter to express the determined value of brightness. For example, the camera parameter may include a parameter related to the brightness of the image. For example, the parameter related to the brightness of the image may include a system value of shutter speed and / or a system value of ISO. For example, the descriptions of the brightness control module (313) of FIG. 3 may be referenced for the system value of shutter speed and / or the system value of ISO.

[0147] According to one embodiment, the electronic device (101) may store the acquired bokeh image (304) in memory (e.g., memory (210)). For example, the electronic device (101) may store the bokeh image in a raw file format. However, the embodiments are not limited. For example, the electronic device (101) may store the bokeh image (304) in various file formats (e.g., JPEG (joint photographic experts group)).

[0148] According to one embodiment, metadata of a stored bokeh image (304) may include depth values ​​of pixels within the bokeh image (304). For example, metadata may include a depth map of the bokeh image (304). For example, an electronic device (101) may execute an editing function to edit the stored bokeh image (304). For example, the electronic device (101) may change the value of a virtual aperture while executing the editing function. For example, the electronic device (101) may obtain a different bokeh image based on changing the value of the virtual aperture. For example, the different bokeh image may have a depth according to the changed value of the virtual aperture. For example, the electronic device (101) may obtain a different bokeh image by providing the set value of the virtual aperture and the stored bokeh image (304) to a second bokeh model (329).

[0149] In an embodiment according to the present disclosure, an electronic device (e.g., electronic device (101)) may display a user interface that can set a virtual aperture value through a display (e.g., display (220)). By changing the virtual aperture value through the user interface, the electronic device (101) may provide the user with a user experience of operating a DSLR camera. The electronic device (101) may acquire an image with a bokeh effect applied (e.g., preview bokeh image (303), bokeh image (304)) using a bokeh model (e.g., first bokeh model (315), second bokeh model (329)). The image with the bokeh effect may have a blur effect of different intensities processed according to the depth value of each pixel within the image. The image with the bokeh effect may include a gradual blur effect according to the depth value. The electronic device (101) may provide the user with an enhanced user experience through the image with the bokeh effect applied.

[0150] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure belongs from the description below.

[0151] The technical problems to be solved in this disclosure are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which this disclosure belongs.

[0152] As described above, an electronic device (e.g., electronic device (101)) may include at least one camera (e.g., at least one camera (230)). The electronic device may include a display (e.g., display (220)). The electronic device may include a memory (e.g., memory (210)) comprising one or more storage media for storing instructions. The electronic device may include at least one processor (e.g., at least one processor (200)) comprising processing circuitry. The instructions may cause the electronic device to receive user input for setting a virtual aperture value while displaying a preview image acquired through the at least one camera via the display when executed individually or collectively by the at least one processor. The above instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to obtain a preview bokeh image (e.g., a preview bokeh image (303)) expressing a visual effect according to the value of the virtual aperture, based on providing the value of the virtual aperture and the preview image to a bokeh model (e.g., a first bokeh model (315)). The above instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to display the preview bokeh image through the display.

[0153] According to one embodiment, each pixel in the preview bokeh image expressing the visual effect can express a blur effect of higher intensity as the difference between the depth value of each pixel and the depth of focus value corresponding to the focus point of the preview bokeh image increases. The intensity of the blur effect expressed by each of the pixels in the preview bokeh image may be based on the value of the virtual aperture.

[0154] According to one embodiment, when the value of the virtual aperture is a first value, a pixel having a depth value within the preview bokeh image can express a blur effect of a first intensity. When the value of the virtual aperture is a second value smaller than the first value, a pixel having a depth value within the preview bokeh image can express a blur effect of a second intensity higher than the first intensity.

[0155] According to one embodiment, the preview bokeh image may include a visual object (e.g., a visual object (510)). Pixels within an area of ​​the preview bokeh image occupied by the visual object may include a first pixel having a depth value smaller than the focal depth value and a second pixel having a depth value smaller than the depth value of the first pixel. The intensity of the blur effect expressed by the first pixel may be lower than the intensity of the blur effect expressed by the second pixel.

[0156] According to one embodiment, the preview bokeh image may include a first visual object (e.g., visual object (510)) located at the focal position and a second visual object (e.g., visual object (520)) located in front of the first visual object. The intensity of the blur effect expressed by pixels in a first region within the preview bokeh image occupied by the first visual object may be lower than the intensity of the blur effect expressed by pixels in a second region within the preview bokeh image occupied by the second visual object.

[0157] According to one embodiment, the instructions may cause the electronic device to determine the brightness value of the preview bokeh image based on the value of the virtual aperture when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to change the value of a camera parameter to express the brightness value based on the value of the virtual aperture when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to display the preview bokeh image having the brightness value through the display based on the changed value of the camera parameter based on the value of the virtual aperture when executed individually or collectively by the at least one processor.

[0158] According to one embodiment, the instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to display a visual object (e.g., visual object (420, 421, 422, 423)) that is used as a shooting input while displaying the preview image and represents an aperture through the display. The instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to change the appearance of the visual object by changing the size of the aperture opening represented by the visual object according to the value of the virtual aperture, based on receiving the user input for setting the value of the virtual aperture.

[0159] According to one embodiment, the bokeh model can be trained using images acquired through a digital single-lens reflex (DSLR) camera.

[0160] According to one embodiment, the preview bokeh image expressing the visual effect may include a visual object corresponding to a light source. The visual object corresponding to the light source can clearly express a plurality of light rays emitted from the light source as the value of the virtual aperture increases.

[0161] According to one embodiment, the instructions may cause the electronic device to receive other user input to acquire a bokeh image (e.g., bokeh image (304)) while displaying the preview bokeh image, when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to acquire an intermediate image through the at least one camera based on receiving the other user input, when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to acquire a bokeh image expressing a different visual effect according to the value of the virtual aperture, based on providing the intermediate image and the value of the virtual aperture to a second bokeh model (e.g., second bokeh model (329)), when executed individually or collectively by the at least one processor.

[0162] According to one embodiment, each pixel in the bokeh image expressing the other visual effect can express a blur effect of higher intensity as the difference between the depth value of each pixel in the bokeh image and the second depth of focus value corresponding to the second focal position of the bokeh image increases. The intensity of the blur effect expressed by each pixel in the bokeh image may be based on the value of the virtual aperture.

[0163] According to one embodiment, the bokeh image may include a visual object (e.g., a visual object (510)). Pixels within an area of ​​the bokeh image occupied by the visual object may include a first pixel having a depth value smaller than the second focal depth value and a second pixel having a depth value smaller than the depth value of the first pixel. The intensity of the blur effect expressed by the first pixel may be lower than the intensity of the blur effect expressed by the second pixel.

[0164] According to one embodiment, the bokeh image may include a first visual object (e.g., visual object (510)) located at the second focal position and a second visual object (e.g., visual object (520)) located in front of the first visual object. The intensity of the blur effect expressed by pixels in a first region within the bokeh image occupied by the first visual object may be lower than the intensity of the blur effect expressed by pixels in a second region within the bokeh image occupied by the second visual object.

[0165] According to one embodiment, the intermediate image can be obtained by providing at least one image obtained through the at least one camera to an image synthesis model (e.g., image synthesis model (321)).

[0166] According to one embodiment, the instructions may cause the electronic device to acquire at least one image through the at least one camera when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to identify depth values ​​of pixels of the intermediate image by providing the at least one image to a depth model (e.g., single image depth model (325), multi image depth model (327)) when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to acquire the bokeh image based on further providing the depth values ​​of the pixels of the intermediate image to the second bokeh model when executed individually or collectively by the at least one processor.

[0167] According to one embodiment, the second bokeh model can be learned using images acquired through a digital single-lens reflex (DSLR) camera.

[0168] According to one embodiment, the metadata of the bokeh image (e.g., metadata (610)) may include the value of the virtual aperture.

[0169] A method performed in an electronic device (e.g., electronic device (101)) having at least one camera (e.g., at least one camera (230)) and a display (e.g., display (220)) as described above may include receiving user input for setting a value of a virtual aperture while displaying a preview image acquired through the at least one camera through the display. The method may include acquiring a preview bokeh image (e.g., preview bokeh image (303)) that expresses a visual effect according to the value of the virtual aperture, based on providing the value of the virtual aperture and the preview image to a bokeh model (e.g., first bokeh model (315)). The method may include displaying the preview bokeh image through the display.

[0170] According to one embodiment, each pixel in the preview bokeh image expressing the visual effect can express a blur effect of higher intensity as the difference between the depth value of each pixel and the depth of focus value corresponding to the focus point of the preview bokeh image increases. The intensity of the blur effect expressed by each of the pixels in the preview bokeh image may be based on the value of the virtual aperture.

[0171] According to one embodiment, when the value of the virtual aperture is a first value, a pixel having a depth value within the preview bokeh image can express a blur effect of a first intensity. When the value of the virtual aperture is a second value smaller than the first value, a pixel having a depth value within the preview bokeh image can express a blur effect of a second intensity higher than the first intensity.

[0172] According to one embodiment, the preview bokeh image may include a visual object (e.g., a visual object (510)). Pixels within an area of ​​the preview bokeh image occupied by the visual object may include a first pixel having a depth value smaller than the focal depth value and a second pixel having a depth value smaller than the depth value of the first pixel. The intensity of the blur effect expressed by the first pixel may be lower than the intensity of the blur effect expressed by the second pixel.

[0173] According to one embodiment, the preview bokeh image may include a first visual object (e.g., visual object (510)) located at the focal position and a second visual object (e.g., visual object (520)) located in front of the first visual object. The intensity of the blur effect expressed by pixels in a first region within the preview bokeh image occupied by the first visual object may be lower than the intensity of the blur effect expressed by pixels in a second region within the preview bokeh image occupied by the second visual object.

[0174] According to one embodiment, the method may include an operation of determining a value of brightness of the preview bokeh image based on the value of the virtual aperture. The method may include an operation of changing a value of a camera parameter for expressing the value of brightness based on the value of the virtual aperture. The method may include an operation of displaying the preview bokeh image having the value of brightness through the display based on the value of the virtual aperture and the changed value of the camera parameter.

[0175] According to one embodiment, the method may include an operation of displaying a visual object (e.g., visual object (420, 421, 422, 423)) that is used as a shooting input and represents an aperture, through the display while displaying the preview image. The method may include an operation of changing the appearance of the visual object by changing the size of the aperture opening represented by the visual object according to the value of the virtual aperture, based on receiving the user input for setting the value of the virtual aperture.

[0176] According to one embodiment, the bokeh model can be trained using images acquired through a digital single-lens reflex (DSLR) camera.

[0177] According to one embodiment, the preview bokeh image expressing the visual effect may include a visual object corresponding to a light source. The visual object corresponding to the light source can clearly express a plurality of light rays emitted from the light source as the value of the virtual aperture increases.

[0178] According to one embodiment, the method may include receiving another user input to acquire a bokeh image (e.g., bokeh image (304)) while displaying the preview bokeh image. The method may include acquiring an intermediate image through the at least one camera based on receiving the other user input. The method may include acquiring the bokeh image expressing a different visual effect according to the value of the virtual aperture based on providing the intermediate image and the value of the virtual aperture to a second bokeh model (e.g., second bokeh model (329)). Pixels within the bokeh image may have a higher intensity blur effect processed as the difference between the depth value of each pixel within the bokeh image and the second depth of focus value corresponding to the second focal position of the bokeh image increases.

[0179] According to one embodiment, each pixel in the bokeh image expressing the other visual effect can express a blur effect of higher intensity as the difference between the depth value of each pixel in the bokeh image and the second depth of focus value corresponding to the second focal position of the bokeh image increases. The intensity of the blur effect expressed by each pixel in the bokeh image may be based on the value of the virtual aperture.

[0180] According to one embodiment, the bokeh image may include a visual object (e.g., a visual object (510)). Pixels within an area of ​​the bokeh image occupied by the visual object may include a first pixel having a depth value smaller than the second focal depth value and a second pixel having a depth value smaller than the depth value of the first pixel. The intensity of the blur effect expressed by the first pixel may be lower than the intensity of the blur effect expressed by the second pixel.

[0181] According to one embodiment, the bokeh image may include a first visual object (e.g., visual object (510)) located at the second focal position and a second visual object (e.g., visual object (520)) located in front of the first visual object. The intensity of the blur effect expressed by pixels in a first region within the bokeh image occupied by the first visual object may be lower than the intensity of the blur effect expressed by pixels in a second region within the bokeh image occupied by the second visual object.

[0182] According to one embodiment, the intermediate image can be obtained by providing at least one image obtained through the at least one camera to an image synthesis model (e.g., image synthesis model (321)).

[0183] According to one embodiment, the method may include the operation of acquiring at least one image through the at least one camera. The method may include the operation of identifying depth values ​​of pixels of the intermediate image by providing the at least one image to a depth model (e.g., single image depth model (325), multi image depth model (327)). The method may include the operation of acquiring the bokeh image based on further providing the depth values ​​of the pixels of the intermediate image to the second bokeh model.

[0184] According to one embodiment, the second bokeh model can be learned using images acquired through a digital single-lens reflex (DSLR) camera.

[0185] According to one embodiment, the metadata of the bokeh image (e.g., metadata (610)) may include the value of the virtual aperture.

[0186] In a computer-readable storage medium in which one or more programs are stored as described above, the one or more programs may include instructions that cause the electronic device to receive user input to set a value of a virtual aperture while displaying a preview image acquired through the at least one camera (e.g., at least one camera (230)) and a display (e.g., display (220)) when executed by the electronic device (e.g., electronic device (101)). The one or more programs may include instructions that cause the electronic device to acquire a preview bokeh image (e.g., preview bokeh image (303)) expressing a visual effect according to the value of the virtual aperture, based on providing the value of the virtual aperture and the preview image to a bokeh model (e.g., first bokeh model (315)) when executed by the electronic device. The above one or more programs may include instructions that cause the electronic device to display the preview bokeh image through the display when executed by the electronic device.

[0187] According to one embodiment, each pixel in the preview bokeh image expressing the visual effect can express a blur effect of higher intensity as the difference between the depth value of each pixel and the depth of focus value corresponding to the focus point of the preview bokeh image increases. The intensity of the blur effect expressed by each of the pixels in the preview bokeh image may be based on the value of the virtual aperture.

[0188] According to one embodiment, when the value of the virtual aperture is a first value, a pixel having a depth value within the preview bokeh image can express a blur effect of a first intensity. When the value of the virtual aperture is a second value smaller than the first value, a pixel having a depth value within the preview bokeh image can express a blur effect of a second intensity higher than the first intensity.

[0189] According to one embodiment, the preview bokeh image may include a visual object (e.g., a visual object (510)). Pixels within an area of ​​the preview bokeh image occupied by the visual object may include a first pixel having a depth value smaller than the focal depth value and a second pixel having a depth value smaller than the depth value of the first pixel. The intensity of the blur effect expressed by the first pixel may be lower than the intensity of the blur effect expressed by the second pixel.

[0190] According to one embodiment, the preview bokeh image may include a first visual object (e.g., visual object (510)) located at the focal position and a second visual object (e.g., visual object (520)) located in front of the first visual object. The intensity of the blur effect expressed by pixels in a first region within the preview bokeh image occupied by the first visual object may be lower than the intensity of the blur effect expressed by pixels in a second region within the preview bokeh image occupied by the second visual object.

[0191] According to one embodiment, the one or more programs may include instructions that cause the electronic device to determine the brightness value of the preview bokeh image based on the value of the virtual aperture when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to change the value of a camera parameter for expressing the brightness value based on the value of the virtual aperture when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to display the preview bokeh image having the brightness value through the display based on the changed value of the camera parameter based on the value of the virtual aperture when executed by the electronic device.

[0192] According to one embodiment, the one or more programs may include instructions that cause the electronic device to display a visual object (e.g., visual object (420, 421, 422, 423)) that is used as a shooting input and represents an aperture, when executed by the electronic device while displaying the preview image. The one or more programs may include instructions that cause the electronic device to change the appearance of the visual object by changing the size of the aperture opening represented by the visual object according to the value of the virtual aperture, based on receiving the user input for setting the value of the virtual aperture when executed by the electronic device.

[0193] According to one embodiment, the bokeh model can be trained using images acquired through a digital single-lens reflex (DSLR) camera.

[0194] According to one embodiment, the preview bokeh image expressing the visual effect may include a visual object corresponding to a light source. The visual object corresponding to the light source can clearly express a plurality of light rays emitted from the light source as the value of the virtual aperture increases.

[0195] According to one embodiment, the one or more programs may include instructions that cause the electronic device to receive other user input to acquire a bokeh image (e.g., bokeh image (304)) while displaying the preview bokeh image when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to acquire an intermediate image through the at least one camera based on receiving the other user input when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to acquire a bokeh image expressing a different visual effect according to the value of the virtual aperture based on providing the intermediate image and the value of the virtual aperture to a second bokeh model (e.g., second bokeh model (329)) when executed by the electronic device. The pixels in the bokeh image may have a higher intensity blur effect processed as the difference between the depth value of each pixel in the bokeh image and the second focal depth value corresponding to the second focal position of the bokeh image increases.

[0196] According to one embodiment, each pixel in the bokeh image expressing the other visual effect can express a blur effect of higher intensity as the difference between the depth value of each pixel in the bokeh image and the second depth of focus value corresponding to the second focal position of the bokeh image increases. The intensity of the blur effect expressed by each pixel in the bokeh image may be based on the value of the virtual aperture.

[0197] According to one embodiment, the bokeh image may include a visual object (e.g., a visual object (510)). Pixels within an area of ​​the bokeh image occupied by the visual object may include a first pixel having a depth value smaller than the second focal depth value and a second pixel having a depth value smaller than the depth value of the first pixel. The intensity of the blur effect expressed by the first pixel may be lower than the intensity of the blur effect expressed by the second pixel.

[0198] According to one embodiment, the bokeh image may include a first visual object (e.g., visual object (510)) located at the second focal position and a second visual object (e.g., visual object (520)) located in front of the first visual object. The intensity of the blur effect expressed by pixels in a first region within the bokeh image occupied by the first visual object may be lower than the intensity of the blur effect expressed by pixels in a second region within the bokeh image occupied by the second visual object.

[0199] According to one embodiment, the intermediate image can be obtained by providing at least one image obtained through the at least one camera to an image synthesis model (e.g., image synthesis model (321)).

[0200] According to one embodiment, the one or more programs may include instructions that cause the electronic device to acquire at least one image through the at least one camera when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to identify depth values ​​of pixels of the intermediate image by providing the at least one image to a depth model (e.g., single image depth model (325), multi image depth model (327)) when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to acquire the bokeh image based on further providing the depth values ​​of the pixels of the intermediate image to the second bokeh model when executed by the electronic device.

[0201] According to one embodiment, the second bokeh model can be learned using images acquired through a digital single-lens reflex (DSLR) camera.

[0202] According to one embodiment, the metadata of the bokeh image (e.g., metadata (610)) may include the value of the virtual aperture.

[0203] The electronic devices according to the various embodiments disclosed in this document may be of various forms. The electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, electronic devices, or consumer electronics. The electronic devices according to the embodiments of this document are not limited to the devices described above.

[0204] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0205] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0206] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101) of FIG. 1). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0207] According to one embodiment, the method according to the various embodiments disclosed herein may be provided as included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created in a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0208] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device, At least one camera; display; Memory comprising one or more storage media for storing instructions; and It includes at least one processor comprising a processing circuit, and When the above instructions are executed individually or collectively by the at least one processor, the electronic device: While displaying a preview image acquired through at least one camera through the display, receiving user input to set a virtual aperture value, Based on providing the value of the virtual aperture and the preview image to the bokeh model, a preview bokeh image expressing a visual effect according to the value of the virtual aperture is obtained, and Causing the above preview bokeh image to be displayed through the above display, Electronic device.

2. In Claim 1, Each pixel in the preview bokeh image expressing the above visual effect expresses a higher intensity blur effect as the difference between the depth value of each pixel and the depth of focus value corresponding to the focus point of the preview bokeh image increases, and The intensity of the blur effect expressed by each of the pixels in the above preview bokeh image is based on the value of the above virtual aperture, Electronic device.

3. In Claim 2, When the value of the virtual aperture is the first value, a pixel having a depth value within the preview bokeh image expresses the blur effect of the first intensity, and When the value of the virtual aperture is a second value smaller than the first value, the pixel having the first depth value within the preview bokeh image expresses the blur effect of a second intensity higher than the first intensity. Electronic device.

4. In Claim 2, The above preview bokeh image includes a visual object, and The pixels within the area within the preview bokeh image occupied by the visual object include a first pixel having a depth value smaller than the focal depth value and a second pixel having a depth value smaller than the depth value of the first pixel, and The intensity of the blur effect expressed by the first pixel is lower than the intensity of the blur effect expressed by the second pixel. Electronic device.

5. In Claim 2, The above preview bokeh image includes a first visual object located at the focal position and a second visual object located in front of the first visual object, and The intensity of the blur effect expressed by pixels in the first region within the preview bokeh image occupied by the first visual object is lower than the intensity of the blur effect expressed by pixels in the second region within the preview bokeh image occupied by the second visual object. Electronic device.

6. In Claim 1, When the above instructions are executed individually or collectively by the at least one processor, the electronic device: Based on the above value of the virtual aperture: Determine the brightness value of the above preview bokeh image, and Changing the value of the camera parameter to express the above value of the brightness, and Causing to display the preview bokeh image having the brightness value based on the changed value of the camera parameter through the display. Electronic device.

7. In Claim 1, When the above instructions are executed individually or collectively by the at least one processor, the electronic device: While displaying the above preview image, a visual object that is used as a shooting input and represents the aperture is displayed through the above display, and Based on receiving the user input for setting the value of the virtual aperture, changing the size of the opening of the aperture represented by the visual object according to the value of the virtual aperture causes the appearance of the visual object to be changed. Electronic device.

8. In Claim 1, The above bokeh model is trained using images acquired through a DSLR (digital single-lens reflex) camera, Electronic device.

9. In Claim 1, The preview bokeh image expressing the above visual effect includes a visual object corresponding to a light source, and The visual object corresponding to the light source expresses a plurality of rays emitted from the light source more clearly as the value of the virtual aperture increases. Electronic device.

10. In Claim 1, When the above instructions are executed individually or collectively by the at least one processor, the electronic device: While displaying the above preview bokeh image, receive other user input to acquire the bokeh image, and Based on receiving the other user input mentioned above, an intermediate image is acquired through the at least one camera, and Based on providing the intermediate image and the value of the virtual aperture to the second bokeh model, causing to obtain the bokeh image expressing a different visual effect according to the value of the virtual aperture, Electronic device.

11. In Claim 10, Each of the pixels in the bokeh image expressing the above other visual effect expresses a blur effect of high intensity as the difference between the depth value of each of the pixels in the bokeh image and the second depth of focus value corresponding to the second focal position of the bokeh image is greater, and The intensity of the blur effect expressed by each of the pixels in the bokeh image is based on the value of the virtual aperture. Electronic device.

12. In Claim 11, The above bokeh image includes a visual object, and Pixels within the region within the bokeh image occupied by the visual object include a first pixel having a depth value smaller than the second focal depth value and a second pixel having a depth value smaller than the depth value of the first pixel, and The intensity of the blur effect expressed by the first pixel is lower than the intensity of the blur effect expressed by the second pixel. Electronic device.

13. In Claim 11, The bokeh image above includes a first visual object located at the second focal position and a second visual object located in front of the first visual object, and The intensity of the blur effect expressed by pixels in the first region within the bokeh image occupied by the first visual object is lower than the intensity of the blur effect expressed by pixels in the second region within the bokeh image occupied by the second visual object. Electronic device.

14. A method performed in an electronic device having at least one camera and a display, The operation of receiving user input to set a virtual aperture value while displaying a preview image acquired through at least one camera through the display, Based on providing the value of the virtual aperture and the preview image to a bokeh model, the operation of acquiring a preview bokeh image that expresses a visual effect according to the value of the virtual aperture, and A method including the operation of displaying the above preview bokeh image through the above display. method.

15. In a non-transient computer-readable storage medium storing one or more programs, When the above one or more programs are executed by an electronic device having at least one camera and a display, While displaying a preview image acquired through at least one camera through the display, receiving user input to set a virtual aperture value, Based on providing the value of the virtual aperture and the preview image to the bokeh model, a preview bokeh image expressing a visual effect according to the value of the virtual aperture is obtained, and Instructions for causing the electronic device to display the above preview bokeh image through the display, Non-transient computer-readable storage media.

Images